The oil and gas industry is experiencing multi-dimensional landscape changes, including economical, production and consumption areas. The impact of these changes starts with the upstream business also referred to as exploration and production (E&P) industry.
The E&P industry is one of the most complex operations among any of the industry verticals. This industry has been at the forefront of massive data collection historically, well before the term Big Data has been in use. The data collected is from instruments, sensors, computer software and hand-written notes for conducting effective and efficient operations.
On a daily basis this data collection can range from a few terabytes to a few petabytes, depending upon the phase of oil well life cycle and the complexity and modernity of the oil field. The overarching goal of the E&P industry is to produce maximum oil at the lowest cost possible with the highest degree of safety for all stakeholders. Another important aspect for E&P industry is that the above goal must be achieved within the ever-increasing costly restrictions imposed by the global, regional and local regulatory authorities. Thus, the E&P industry has to remain innovative and competitive at all times.
Forward looking E&P companies must adopt Big Data, the second industrial revolution to remain competitive and innovative. The industry has a strong foundation for the seven pillars of Big Data namely, Volume, Velocity, Variety, Veracity, Virtual, Value and Variability.
The meta-data associated with tools used in exploration and drilling could be in few megabytes while the metrics it collects daily could range from megabytes to gigabytes to petabytes, covering a wide range of volume through the lifecycle.
The data comes at different velocity, some are collected and processed in batch mode, while some data may be collected in real-time and need for actionable insights to be real-time as well.
As mentioned earlier, the data comes in highly structured numerical format to grossly unstructured hand-written reports on an irregular to regular basis, thus the variety of data in E&P covers all types of data including audio, video, text, numerical, etc.
The fourth pillar is the veracity commonly referred as the truth in data. For E&P, this is critical pillar because the data that is collected through thousands of sensors is important, requiring regular monitoring, analyzing and recalibration so the metrics remains relevant. The impact of weather, temperature, pressure, fluids and other activities on these sensors could pose challenges for keeping the truth in this data collection. Similarly, the handwritten drilling reports and other structure and unstructured reports must speak to veracity.
Additionally, the data in the E&P industry is in multiple locations and due to the mobility gap moving this data is a big challenge, so one has to leverage the virtual pillar of Big Data. The Virtual pillar addresses data duplication and lost in transformation related problems while enabling better data governance.
The variability occurs in these five pillars, across the phases of oil well life cycle. For example, the exploration data is generated in terabytes to petabytes but does not need to be analyzed in real-time, hence has low velocity. The seismic analysis is critical for the E&P industry thus has a significant value compared to many other phases of life cycle. During the drilling phase the data may be generated in real-time and requires actions to be taken in real-time for optimal operations, thus the velocity is very high.
The last pillar is value. If there is no value in the E&P data then none of these pillars of big data matter much. Figure 1 shows the relative comparison of these pillars. Figure 1 A relative comparison of 7 pillars of Big Data in E&P industry. In summary, the 7 pillars of Big Data can be described as shown in Figure 2.
The four enablers to create the value from E&P Big Data are data, technologies, advanced analytics and actionable insights. These enablers are important for E&P because the industry has significantly inefficient processes and lacks automation and real-time insights for the amount of data it generates across the phases of an oil wells life cycle. The E&P industry holds a significant amount of dark data that has significant hidden value that can make the E&P industry innovative and competitive. To do this one needs a next generation E&P platform.
Halliburton addresses this problem by building industry leading highly integrated and modular E&P Big Data Platform, DecisionSpace Enterprise software. This platform has 4 Foundation layers and that enables us to create value from E&P Big Data. The 4 foundation layers include:
Information Foundation
A comprehensive E&P data management platform that incorporates leveraging the well known E&P databases like OpenWorks software, Recall, etc. and the big data stores using Hadoop, NoSQL etc.
Integration Foundation
This is an E&P domain knowledge enhanced data processing applications layer where it leverages traditional data processing tools and technology, as well as complex emerging technologies including MPP, IN-Memory, HPC, Hadoop, X-SQL, etc.
Application Foundation
As discussed earlier the layer provides a strong, reliable foundation to build rich applications specific to the phase of the oil well life cycle of interest, but leveraging all the data available. This layer provides us the ability to add algorithms, models, analysis, machine learning, artificial intelligence libraries, etc. to easily create prescriptive, predictive, and cognitive analytics.
Ecosystems Foundation
It provides an easy to use, easy to deploy, role based dashboards for key performance indicators, alarms for outliers events and easy to address actionable insights. It also provides for creating new patterns from pipelining the virtual data through the Information and Integration foundation that results in creating innovative products and services for E&P industry. This also enables us to leverage emerging technology revolutions like mobile, which provides insights anywhere.
The integrated and modular platform helps us to leverage historical Big Data and provide real-time predictive analytics from a single platform, thereby increasing efficiency in creating value from Big Data. The DecisionSpace platform is used to explore hidden inefficiencies in E&P Dark Data that is sitting in data silos in various locations. The new patterns from combining data from multiple diverse sources for diverse variety becomes a smoother, easier process thus providing significant return on innovation through E&P Big Data.
