As with many industries, data science is transforming the energy vertical, providing insights into cost reductions in down markets and allowing oil producers to adjust to market demands in boom times.

Recent declines in oil prices have hit the world economy hard. Alberta, Canada’s major oil region, has witnessed increased unemployment due to declining commodity prices. In January 2016, Saudi Arabia increased the price of gasoline for its citizens by 50 percent given the situation. With major fluctuations in prices and the high cost of energy projects, quality information has never mattered more.

The energy industry uses data science to cut costs, optimize investments and reduce risk. Reducing costs with data science is a popular application in the industry: much work has focused on improving maintenance and equipment monitoring. Optimizing investment decisions takes several forms including better internal resource allocation and assisting investors. Data science also contributes to improving public safety by providing better monitoring and oversight.

Delivering innovation by borrowing ideas from other sectors

Transferring ideas and techniques across industries is a tried-and-true innovation method. The energy industry has recently started to adopt the survival analysis concept from the medical field. In medicine, survival analysis is a statistical method to estimate survival rates for patients based on their condition, treatments and related matters. In the oil and gas sector, this concept has been applied to field equipment.

Survival analysis is used to predict the maintenance requirements for field equipment such as compressors through monitoring and modeling. Instead of taking an oil well offline for three days to repair damage from equipment failure, proactive action enabled by data science can reduce downtime to a single day, he says. Saving a day of downtime is valuable. A day’s production at a small site – 1,000 barrels of oil – represents $30,000 of revenue at current prices.

British Petroleum (BP), the U.K.-based energy company, has long been a leader in IT and related disciplines. The company’s drive to invest in this area is driven by several factors. In terms of safety, the company’s 2010 Deepwater Horizon disaster led to $18 billion fine in 2015 and other damage to the environment. Preventing such a disaster through better information is important rationale for the company. In 2013, the company established a Center for High- Performance Computing in Houston, Texas to connect with leading American institutions such as Rice University.

BP’s commitment to improvement through analytics shows an end to end commitment. The process starts with investment in high quality data and monitoring capabilities.

Data analytics in the field. The BP Well Advisor provides operational support for oil sites. This information is fed into several dashboards at the production site and at the corporate offices. The Well Advisor is now in use at more than one hundred 100 offshore wells.

Improving production. BP builds models and analytics to improve the efficiency of its refineries. This approach optimizes the refinery’s production capabilities. Analytics plays a role in directly improving production.

Partnerships and talent. BP’s direct investments in technology are only part of the data story. The company also works closely with IBM to improve its capabilities. BP has also recognized the importance of staff – Charles Cai, head of data science technology at BP, has been recognized as one of the Top 50 U.K. Data Leaders.

Not every energy firm operates at BP’s scale with operations scattered around the world. Fortunately, there are other ways to get started in analytics.

Inside the data science toolkit

Before we dive into tools and techniques, it is vital to start with the business problem. Typical business problems in energy include predicting production, improving field efficiency and understanding geological activities. Large firms such as BP and Halliburton have adopted data science methods already. I see a great opportunity for small companies with less complex data to achieve wins by bringing one or two specialized data scientists on board.

In oil and gas, you have a wide variety of data to work with and it takes time to bring this all together. I have seen projects where some data are in Oracle databases, other databases have drilling data and there are yet other systems for economic and seismic data. Bringing all this data together requires tools such as Hadoop and NoSQL.

Regarding specific tools, it will depend on the complexity of the problem. If you are working on a problem with over 50 variables, I suggest looking into machine learning tools. Random Forest, produced by Salford Systems, is one option to consider. For other projects, the data science toolkit includes tools such as R and Python. Tibco and Tableau are useful visualization tools to present the data.

Opportunities for data analytics

Consulting firms and analysts have long added value to the industry through their specialized knowledge – the same holds true for analytics and energy.  Organizing data and presenting it in a useful way is another way to add value with data science.

The market pays a premium for explanatory data visualizations because many organizations lack in-house capabilities for these activities. In the energy sector, there are still plenty of data opportunities. It starts with implementing systems and processes for data collection, cleaning and storage. Hiring a data scientist to build the architecture and guide the implementation is one way to start. With that in place, you can start to generate predictive insights.

The future for energy data science

The use of data science and analytics is expected to grow in the energy industry. In a low oil price environment, management will seek cost reduction insights from data. During growth periods, data science will guide management decision making with better insights to improve production and adjust to market demand. The continued growth of data science tools and vendors will also support the trend.