Mobius extracted gas commodity information from 62 different operator sites for a global petrochemical information provider.

The Business Need

One of the world’s largest information providers had been providing competitive insights to companies in the global petrochemical market encompassing fast-growing energy and fertilizer division. This was achieved by delivering trusted pricing data and high-value news, analysis and independent consulting services thereby enabling customers to make well-informed trading and planning decisions. They needed technical expertise to collect and aggregate data from 62 different operator sites related to gas commodity, on a real-time basis.

Challenges we faced

The major challenge was to develop source-specific crawlers as different operators provide data in different measurement units, like million cubic meters (MCM) - a volume measurement and kilowatt-hours (KWH) - an energy value measurement.

They required technical expertise to aggregate natural gas commodity data like production, demand and unit capacity, amount of gas stored, outflow, import and export from the Transmission System Operators (TSO), Storage System Operators (SSO) and LNG System Operators(LSO), on a real-time basis.

Operators update the data on their websites on a daily basis or intraday basis, at different times of the day. The client wanted the process to query operator websites for new data regularly.

How we solved the problem

Mobius process incorporated an automated solution to scrape and aggregate the data on an hourly basis. Regular checks were performed to ensure all data were being captured and normalized. The normalized data were then validated i.e. checked for zero values or significant day-on-day fluctuations. Mobius process also retained all the historical data for a requested period, for diagnostics and refreshing purpose. The automated process scraped various online data on an hourly basis and published the latest data to the client’s SFTP in two different measurement units – MCM and MWH.


Mobius process ensured an accuracy of more than 98% in delivering consistent real-time data. Highly customized scripts guaranteed high quality and quick turnaround time. The aggregated data exceeded all service level expectations.

62 different operator sites

Different measurement units – MCH & MWH

Accuracy of more than 98%

High quality & quick turnaround time assured with automated solution

Looking out for a similar solution?