Avangrid Uses Azure Synapse to Power NYSEG and RG&E IEDR Data Compliance

Designed around Azure Synapse Analytics, the cloud-native platform was engineered to ingest, validate, and process billions of records across disparate internal systems to satisfy New York’s Integrated Energy Data Resource (IEDR) mandate. Yet as with any vendor case study, the real story for enterprise architects and IT administrators lies in separating the marketing gloss from the underlying engineering realities—and understanding where a utility’s internal Azure pipeline ends and the state’s centralized energy platform begins.

What Avangrid and NTT DATA Built on Microsoft Azure​

According to NTT DATA’s case study, Avangrid—which through its networks business operates eight electric and natural gas utilities serving more than 3.4 million customers across New York and New England —used New York’s IEDR regulatory mandate as a catalyst for broader enterprise data modernization and democratization. Rather than treating regulatory reporting as a one-off compliance chore, NYSEG and RG&E partnered with NTT DATA to construct a centralized Integrated Energy Data Management platform hosted on Microsoft Azure.

The reported workflow covers the full utility data lifecycle required for regulatory handoffs:

• Data Discovery and Ingestion: Pulling high-volume, complex datasets from multiple disparate utility source systems into a unified cloud environment.
• Transformation and Analytics: Utilizing Microsoft’s Azure Synapse Analytics to process, structure, and analyze what NTT DATA reports as billions of records across multiple input streams.
• Integrated Validation: Applying automated quality and standardization checks so outgoing datasets conform to strict regulatory delivery specifications.
• Governance and Secure Delivery: Enforcing security and data governance controls to protect sensitive customer and grid information before packaging datasets for internal teams and external stakeholders, specifically the New York State Energy Research and Development Authority (NYSERDA).

Shawn Ras, Senior Manager for Data Project at Rochester Gas & Electric, noted in NTT DATA’s report that the IEDR mandate posed a distinct challenge because it required delivering large volumes of complex data across multiple system input streams against a rigorous specification network. According to Ras, the data structuring and quality model developed during the engagement allowed the utility to deliver results to NYSERDA with accuracy and confidence in its internal processes.

Crucial Context: Utility Data Pipelines vs. the Statewide IEDR Platform​

One of the easiest traps in reading enterprise IT case studies is conflating a participant’s internal compliance platform with the public program it feeds. A careful review of public filings and program documentation from NYSERDA makes an essential architectural distinction clear: NYSEG and RG&E’s Azure Synapse platform is a utility-side data preparation and management engine, not the statewide IEDR platform itself.

According to NYSERDA’s official program documentation, the statewide Integrated Energy Data Resource was established by the New York Public Service Commission (PSC) under an order issued on February 11, 2021 (Case 20-M-0082). Supporting New York’s Climate Leadership and Community Protection Act—which targets 70% renewable electricity by 2030 and a zero-emission electricity sector by 2040—the statewide IEDR serves as a centralized public and stakeholder repository that collects, integrates, and shares energy data from electric, gas, and steam utilities across New York.

Furthermore, NYSERDA’s March 2024 completion announcement for Phase 1 of the statewide platform confirms that the state’s central IEDR portal is designed and operated by a completely different vendor coalition:

• Program Sponsor: NYSERDA, appointed by the Public Service Commission to oversee and facilitate the statewide program.
• Solution Architect and Development Team: Led by E Source, working alongside partners UtilityAPI, Flux Tailor, and TRC Companies.
• Program Manager: Deloitte Consulting LLP, handling project management and administrative oversight.
• Utility Data Advisor: Pecan Street Inc., advising NYSERDA and the Department of Public Service on utility-sourced data.

Understanding this two-tier ecosystem explains why NYSEG and RG&E needed an enterprise-grade engine like Azure Synapse Analytics on their side of the fence. Every major investor-owned utility in New York—collectively referred to in regulatory filings as the Joint Utilities—must extract data from its own aging meter data management systems, geographic information systems (GIS), customer billing databases, and interconnection queues, and then scrub and format that data to match the exact schemas demanded by the state’s IEDR development team.

Summary: While NYSERDA and E Source run the statewide portal that end users interact with, individual utilities like NYSEG and RG&E must build and maintain their own heavy-duty cloud pipelines on platforms like Microsoft Azure to feed that central repository accurately and on schedule.

Why Regulatory Data Demands Enterprise Cloud Scale​

Why would a regional utility need to process "billions of records" in Azure Synapse just to satisfy a state data portal? A closer look at NYSERDA’s regulatory timeline and data cadence reveals just how massive utility datasets become when smart meters, grid topology, and customer tariffs collide.

According to NYSERDA’s IEDR resource documentation, the program was structured across two major development phases designed to enable approximately 50 distinct clean-energy use cases:

1. Phase 1 (Initial Public Version and Minimum Viable Product): The Initial Public Version launched on March 31, 2023, delivering consolidated utility hosting capacity maps alongside data on installed and planned distributed energy resources (DERs) from utility interconnection queues. The Minimum Viable Product (MVP), completed on March 27, 2024, brought the total to eight active use cases by adding environmental and terrain DER siting assessments, an Electrical Infrastructure Assessment Tool (EIAT), cross-utility rate plan filtering, tariff book access, and a sandbox environment using simulated meter data for customer billing access.
2. Phase 2 Expansion: Approved for funding by the Public Service Commission in January 2024 and officially commenced on April 1, 2024, Phase 2 was projected to span 30 to 36 months after the first quarter of 2024, targeting more than 40 additional use cases. NYSERDA noted in March 2024 that Phase 2 was planned to expand into granular electricity consumption data, full Green Button Connect customer data sharing with approved third-party energy companies, renewable installations, greenhouse gas emissions, and energy efficiency metrics.

From a data engineering standpoint, the most revealing detail on NYSERDA’s resource page is its disclosure regarding data latency and update frequencies. During the MVP rollout, NYSERDA acknowledged that from the time utility data was received by the statewide IEDR to the time it became available to users, latency could exceed one month "due to initial coordination efforts to create alignment in file contents and formatting."

Furthermore, utilities are required to supply data on varying timetables:

• Installed and Planned DERs: Updated monthly.
• Photovoltaic (PV) and Energy Storage System (ESS) Hosting Capacity: Updated annually, or more frequently.
• Electric Vehicle (EV) Load Capacity: Updated annually.
• Rate Plan Data: Aperiodic updates required within two business days of publication to the Department of Public Service website for new or modified rates.

When a utility has to reconcile feeder-level hosting capacity, interconnection queues, tariff updates within 48 hours, and advanced metering infrastructure (AMI) interval reads across millions of endpoints, schema drift and malformed records are the enemy. Using Azure Synapse Analytics allows data engineering teams to combine enterprise data warehousing (via dedicated or serverless SQL pools) with Apache Spark-based big data processing and data integration pipelines—all within a single analytical workspace.

Security, Governance, and the "Custodian" Mandate​

Another critical factor driving Avangrid’s focus on cloud governance is the legal sensitivity of utility customer data. In its case study, NTT DATA emphasizes that integrated governance and security capabilities were built into the Azure platform to protect sensitive information and streamline delivery.

In New York, this is not merely a best practice; it is a strict legal mandate. According to NYSERDA’s summary of the Public Service Commission’s Second IEDR Order (issued October 13, 2023), the commission officially designated the statewide IEDR platform as a "data custodian." Remarkably, that order required New York’s Joint Utilities to transfer Customer Data Sets to the IEDR Administrator without individual customer consent, ruling that the transfer represents a regulated exchange between two data custodians.

Because those transfers involve sensitive billing, consumption, and account records, the Public Service Commission mandated that all activities comply with its Data Access Framework (DAF)—a regulatory rulebook originally ordered in April 2021 to govern how energy service entities access data while preventing unauthorized disclosures. For a utility operating on Microsoft Azure, preparing those transfers requires airtight role-based access control, auditing, data masking for non-production environments, and verifiable encryption in transit and at rest before a single file ever leaves the utility’s tenant.

An Objective Look at What the Case Study Leaves Out​

As veteran IT observers know, vendor case studies are written to celebrate project milestones—and NTT DATA and Avangrid have collaborated closely across multiple digital initiatives, including Avangrid’s Digital Summit showcasing advanced analytics and AI. While the report confirms that NYSEG and RG&E stood up a production Azure Synapse environment to handle their IEDR lifecycle, enterprise architects evaluating this deployment should note what is not disclosed in the public documentation:

• No Granular Azure Architecture Inventory: Beyond naming Microsoft Azure and Azure Synapse Analytics, NTT DATA does not specify whether the ingestion layer relies on Azure Data Factory or Synapse Pipelines, how data lake storage is tiered (such as Azure Data Lake Storage Gen2 or Delta Lake tables), or whether Microsoft Purview is used for the stated governance layer.
• Unquantified Performance and Cost Metrics: While the report cites "billions of records" and "on-time dataset delivery", it offers no baseline benchmarks, query execution improvements, latency reduction figures, or cloud compute cost analyses. (While regulatory filings from the February 2021PSC order established a combined $12 million Phase 1 regulatory cost cap for NYSEG and RG&E with costs deferred for future rate recovery, neither Avangrid nor NTT DATA has disclosed the specific contract cost of the Azure build.)
• Microsoft’s Evolving Analytics Roadmap: From a broader Microsoft ecosystem perspective, choosing Azure Synapse Analytics provides a proven, battle-tested PaaS foundation for enterprise SQL and Spark workloads. However, Windows and Azure administrators will also note that Microsoft has spent the past several years heavily steering new greenfield enterprise analytics workloads toward Microsoft Fabric, which unifies Synapse-style data engineering, Data Factory, and Power BI atop OneLake. For utilities that began designing their Phase 1 and Phase 2 IEDR architectures between 2021 and 2024, Azure Synapse Analytics was the logical, mature enterprise standard—though long-term roadmaps may eventually face questions about Fabric migration or interoperability.

Takeaways for Enterprise Cloud and Data Architects​

Even with the typical omissions of a vendor case study, the NYSEG and RG&E deployment highlights a pragmatic lesson for IT leaders in any heavily regulated industry—whether energy, healthcare, or finance. When regulators mandate complex, ongoing data transparency initiatives like New York’s IEDR, bolting ad-hoc extraction scripts onto legacy on-premises databases only creates technical debt and compliance risk.

By using a compulsory regulatory mandate to justify building a modern, unified cloud data foundation on Microsoft Azure, enterprise IT teams can solve two problems at once: satisfying external auditors with repeatable, validated data pipelines today, and unlocking clean, governed datasets for internal analytics and grid modernization tomorrow.