Enabled one of the world’s largest information provider to deliver information that helped in making informed decisions

Casestudies > Information & Media > Real time commodity data

Business Need

One of the world's largest information provider for the petrochemical market and a fast-growing energy and fertilizer division that provided companies in the global commodity market a competitive advantage by delivering trusted pricing data and high-value news, analysis and independent consulting services, enabling customers to make better informed trading and planning decisions. Their service request was to collect gas commodity data from 62 different operator websites. They required technical expertise for aggregation of natural gas commodity data from the Transmission System Operators (TSO), Storage System Operators (SSO) and LNG System Operators (LSO), on a real-time basis.


  • Data was required to be provided in several different formats such as spreadsheets, PDFs and HTML. Different operators provide data in different measurement units, including million cubic meters (MCM) - a volume measurement, and kilowatt-hours (KWH) - an energy value measurement
  • Operators update the data on their websites on a daily basis or intraday basis, at different times of the day. The client wanted the system/process to query operator websites for new data regularly
  • The commodity data is inclusive of production, demand, capacity of unit, amount of gas stored, outflow, import and export of gas. The challenge was to develop source-specific crawlers that can aggregate the required data in near real-time without any hassles


A three way methodology was adopted -

Data checking
    • Mobius system/process included regular checks to ensure that all data are being captured and subsequently being normalized. The process also ensured that all normalized data values are valid, including checking for zero values or significant day-on-day increases/decreases
Data storage
    • In addition to harvesting, normalizing and delivering latest data to the client, the process ensured that Mobius retained all historical data for a period of five years, for the purpose of diagnostics and refreshing
Data display and distribution
  • Data delivered has to be in two measurement units, MCM and MWH. Client has provided the conversion factors to facilitate this conversion. Mobius should scrape data from the site every hour and publish the latest data to their SFTP. If there is any change in data, that data has to be scrapped and published in a new version


  • Mobius was able to deliver the data in real time consistently with an accuracy of more than 98%
  • Highly customized scripts guaranteed quality and a quick turnaround time was achieved
  • Aggregated data exceeded all sort of service level expectations

Find out how Mobius can help your business

Write to support@mobiusservices.com. Our team will get in touch with you.

© 2018 Mobius Knowledge Services. All Rights Reserved.