PG&E: Where big data meets the Internet of Things
PG&E energy source: Ivanpah, the world’s largest solar generation facility, in the Mojave Desert
Image: Getty
Share on LinkedIn
Share on Xing

PG&E: Where big data meets the Internet of Things

Jessica Twentyman — May 2014

How the Californian gas and electricity company is maximizing the business value of the data mountain generated by its state-wide smart meter deployment.

At Californian utility provider Pacific Gas and Electric Company (PG&E), executives like to boast that they were already working on the smart grid concept long before the term became the cool industry buzzword.

Getting an early head start seems to have paid off for the company: today, PG&E is the largest US utility to have installed smart meters across its entire service territory, which covers 70,000 square miles and 9.4 million residential and commercial properties. Now, the focus is on extracting business value from the data that those smart meters deliver, a process that is already well underway according to Jim Meadows, who previously led PG&E’s 12-year rollout program of smart meters, before moving into his current role of director of smart grid research.

E Jim Meadows PGE
Jim Meadows, director of smart grid research, PG&E

Analyzing smart meter data effectively is important not only for driving the kind of behavioral changes among customers required to achieve better energy efficiency, but also for providing them with a more transparent and reliable service, he says. The smart meter technology measures energy use in hourly or quarter-hourly increments, allowing customers to track energy usage throughout the billing month and thus enabling greater customer control over electricity costs. Usage data is collected through a wireless communications network and transmitted to PG&E’s information system where the data is stored and used for billing and other business purposes.

“When people understand how they use gas or electricity, then they have clear direction on how to optimize their own usage, with positive effects for their monthly bills and the environment. And smart meters mean more visibility for us into our own operations, too, as well as lower costs in meter reading and management,” says Meadows.
Managing peaks

Take, for example, the company’s SmartRate program for customers: those that opt for this tariff get discounts on their bills if they can cut their consumption on what the company terms ‘critical peak days.’

There can be a maximum of 15 critical peak days per year, Meadows explains, but their timing obviously varies from year to year. During a sudden heatwave, for example, PG&E will alert customers to the fact it’s a critical period by text message, email or phone call, so that those customers who might otherwise be tempted to crank up their air-conditioning systems can resist the urge, and take a trip to the beach or shopping mall instead, in return for credit discounts on non-critical days.

And every customer now has access to their own PG&E My Energy online portal, which allows them to analyze their own time-specific energy usage, see how they compare within their own neighborhood and set usage goals.

“We made a conscious decision early on to build a platform where data could be cleansed and perfected in a single place and made ready for business users in different ways.”

But 9.4 million meters, each producing raw usage data, once hourly every day of the year, produces a mountain of data — around 2 terabytes per month or 100 billion readings per year, in fact. So an essential element of the company’s wider smart-meter rollout has been the implementation of an analytics platform, two years in the building, to house all that data.

PG&E’s Interval Data Analytics (IDA) platform, based on a data warehouse from Teradata, is accessed by different areas of the business through a variety of analytical tools from providers that include SAS Institute and Tableau Software.

“We’re doing our best to focus the company, and all of its different lines of business, on a single data platform for this interval data,” he says. “Historically, all of the different departments would have taken their own approach and built their own analytic environments but we made a conscious decision early on not to go in that direction but to build a platform where data could be cleansed and perfected in a single place and made ready for presentation to business users in a wide range of different ways.”
Smart streetlights

Demand among line-of-business users to IDA has been high, he reports, requiring a careful process of prioritization. “We definitely have to do this in an organized way, because there’s no way we could accommodate all possible data use-cases overnight.” Instead, PG&E has gathered together seven of the company’s most senior officers to form a governing panel that decides on which use-case to accommodate next, based on its value to the business.

IDA will also provide the backbone for much of Meadows’ ongoing work at PG&E in his new research director role. “Our regulator has set aside money for true applied research into energy efficiency, which means we’ll get to dabble in new data collection and analysis techniques and to collaborate on analysis with state municipalities,” he says.

A particular target of this collaborative analysis will be on ‘smart streetlight’ projects, he says: there are around 800,000 streetlights in PG&E’s service area, with around one-quarter owned by the utility directly and the rest owned by municipal government. “By sharing our data, and analyzing it together, we’ll be able to detect imbalances at very local, micro levels and correct them, as well as optimize usage across the entire streetlight estate.”

“With the level of smart-meter analysis now available to us, you’ll see us find a whole bunch of new projects to work on over the next three to five years,” Meadows promises. “But not just for our own benefit — our customer interactions will be way smarter, too.”

First published May 2014
Share on LinkedIn
Share on Xing

    Your choice regarding cookies on this site

    Our website uses cookies for analytical purposes and to give you the best possible experience.

    Click on Accept to agree or Preferences to view and choose your cookie settings.

    This site uses cookies to store information on your computer.

    Some cookies are necessary in order to deliver the best user experience while others provide analytics or allow retargeting in order to display advertisements that are relevant to you.

    For a full list of our cookies and how we use them, please visit our Cookie Policy

    Essential Cookies

    These cookies enable the website to function to the best of its ability and provide the best user experience for you. They can still be disabled via your browser settings.

    Analytical Cookies

    We use analytical cookies such as those used by Google Analytics to give us information about the way our users interact with - this helps us to make improvements to the site to enhance your experience.

    For a full list of analytical cookies and how we use them, visit our Cookie Policy

    Social Media Cookies

    We use cookies that track visits from social media platforms such as Facebook and LinkedIn - these cookies allow us to re-target users with relevant advertisements from

    For a full list of social media cookies and how we use them, visit our Cookie Policy