Digital transformation: Harnessing the power of data science

The oil and gas industry has a problem with data scientists. Why? Because most of them do not understand the industry domain. Like other market sectors that are seeking to reduce costs and increase revenue, the oil and gas industry is also looking at, and investing in, automation, AI, and machine learning as potential solutions. According to John Chappell, director of business development—energy, at BlockApps, the industry is “looking at a lot of change, as society undergoes its energy transition and moves towards net-zero.” 

A major component of that change, he notes, will be a technology revolution “filled with edge computing, IoT devices, digital twins, AI, and blockchain technologies.” Cindy Corpis, chief executive officer of SearchPeopleFree, sees this as a natural step forward in the industry, noting, “The volume of data in the oil and gas business has increased tremendously, as information technology has advanced. Managing this data and utilizing it as a strategic asset has a substantial impact on the company’s financial performance.” 

That’s why, she says, business intelligence (BI) tools are no longer capable of performing the necessary level of analysis required for predictive models, which are based on statistics. She adds that when the oil price dropped, courtesy of the coronavirus pandemic, it “compelled oil and gas businesses to look beyond standard methods, in order to improve performance and reduce costs.” 

The role of talent. Because this drive is replete with potential risks—the ones that come from the potential calamitous safety hazards inherent in the industry—it’s important to recruit, train and retain data scientists who understand the potential industry dangers and can interpret data accurately. General knowledge data scientists might be able to survive in other industries, but in the oil and gas industry, they need higher education, training, and experience. As a result of specialized industry experience, access to 24/7 data and performing the right kind of analysis, these data scientists are more likely to be able to predict potential hazards and take action before disasters occur, Fig. 1. 

Fig. 1. The technology revolution is changing the oil and gas industry, creating an unprecedented need for highly trained and specialized data scientists.

According to the 2021 Global Energy Talent Index (GETI) report, the biggest challenge the sector faces is a looming talent crisis. The report was compiled by Airswift, the global workforce solutions provider for the energy, process and infrastructure sectors, and Energy Jobline, a job site for the energy and engineering industries. The survey indicated that eight out of ten oil and gas professionals feel less secure in their jobs than they did in 2020, and 42% believe the sector has contracted over the prior 12 months. Yet, two-thirds expect advances in engineering to open-up important opportunities in the next several years. The 2022 report also defines the talent crisis as one of the industry’s most critical challenges, Fig. 2. 

Those advances have been discussed for some time. In March 2019, Nathan Meehan, then-president of Gaffney, Cline and Associates (and a World Oil editorial advisor), noted in an article titled The End of Petroleum Engineering as We Know It, “Petroleum engineering jobs in the future are likely to be smaller in number and radically different from those of today. The next generation of petroleum engineers will have to address demands for sustainability, lower carbon intensity, and needs for radical productivity improvements, which only artificial intelligence (AI) and digital can drive. This suggests that we will need to revisit university education for petroleum engineers and all aspects of career development and training.” 

Fig. 2. The oil and gas industry relies on specialized talent, who can predict potential hazards and take action before crisis occurs.

Automation and offshore drilling platforms. Automation is essential to the future of the industry. Most offshore production platforms are manned. If equipment fails, somebody will repair it. If companies already have the data that are constantly being gathered via automation and have undertaken the appropriate analysis, they will then be able to predict issues ahead of time. For example, they will be able to determine how many hours it will take for the pressure on the vessel to reach its maximum, if a process fails. This information, in turn, provides the ability to take preventative action before problems arise. Automation, AI and machine learning offer the ability to fix a problem before it happens. 

Automation also benefits the industry in the long term. Petroleum employees begin their process by looking at seismic data to determine the drilling plan and how to make it most efficient ahead of drilling the wells, producing the oil and then establishing a platform or facility at the site to process the oil. All these operations are incredibly cost-intensive. Being able to use automation to make minor changes in each of these areas results in more cost efficiency and substantial savings in the industry. 

Industry-specific issues. The importance of having data engineers specifically and expertly trained for this industry can’t be understated when it comes to safety issues. A general computer engineer, data engineer, or data scientist doesn’t have the background to understand the safety issues inherent in the petroleum industry. They look at databases and just see data. They have no insights and very little ability to see how parameters A and B are related. 

For example, in the case of a distillation column, there will be a certain amount of pressure, at which point certain products will need to be filtered out. An oil and gas data scientist will understand that workers cannot increase the pressure after a certain point without risking safety. 

Chappell also notes that “the supply chain is congested with many different data sources. This puts tracking initiatives into many different silos, making it a challenge for businesses to effectively organize and understand their data. This lack of consistency leaves companies struggling to standardize their outputs, complicating the record-keeping process. As they move away from siloed models to a more tailored and integrated approach through digital transformation, being able to understand these new data sources and solutions will play a foundational role as the world moves to new energy models.” 

Turning to industry leaders. The solution to industry issues will come from collaboration and partnerships. One way to accomplish this is for industry leaders to work with colleges and universities that offer data sciences and petroleum engineering courses and help them rethink the curriculum, Fig. 3. The option for students to minor in statistics and other data science courses could be helpful, as they enter the oil and gas industry. 

Fig. 3. To meet the industry’s talent crisis, industry leaders will need to partner with colleges and universities to better prepare graduates for real-world challenges.

A 2017 industry study by the Rand Corporation of employers and colleges in Ohio, Pennsylvania and West Virginia, highlighted the problems associated with recruiting the necessary graduates in the oil and gas industry. According to the report, few employers provided in-depth, ongoing or curricular support to colleges. Sixty-four percent of employers reported that they did not offer any instructional support to colleges, and 55% of employers offered no material or fiscal support to colleges for workforce development. Only 17% offered scholarships, while another mere 17% reported providing supplies and materials for hands-on training. In addition, few employers offered other incentives, such as cash support for low-income students, equipment for realistic, high-fidelity training, or laboratory centers to share space and equipment. 

Along with more collaboration between employers and colleges, there also needs to be an emphasis on in-depth training of data scientists for the industry. While there are many masters programs where future industry data scientists can train, there are far more bootcamps that don’t require much effort to be accepted into, and who churn out graduates within a matter of months or even weeks. That approach might work for other industries, but it does not work for oil and gas. One reason is that the courses barely scratch the surface of the technical requirements needed. 

Currently, there is no standardized process when it comes to data science studies specific to the oil and gas industry, and there are very few students studying data science at the masters’ level. And those who do are still not well-versed in the specifics required for the oil and gas domain. To enter the industry, graduates need to have a firm grasp of statistics, mathematics, machine learning and probabilities, as well as knowledge of databases, such as SQL, and cloud technologies, such as AWS. Courses undertaken should also foster a knowledge of languages, including R and Python. According to Chappell, there also needs to be specific training in new areas, “such as blockchain at home to stay competitive and ensure the next generation is getting the training they need to be a part of the solution. This means investing in data education at all levels.” 

In addition, companies within the industry should be offering incentives, by funding packages of courses for new petroleum engineers to undertake, and actively seeking to hire these students. It’s that financial incentive that is key, according to Janette Marx, CEO at Airswift. She noted in the GETI report that while the industry is resilient, “the bigger long-term challenge is a reduction in available capital, with investors looking to their own reputations and diverting funds towards sectors like renewables.” 

Marx also added that the key to overcoming these obstacles requires firms in the industry to “show their commitment to innovation and to people, demonstrate support for environmental measures and technological advancements and, crucially, ensure that investors and the workforce alike hear the message.” 

Similarly, the Rand study recommended a stronger connection between “the actors and institutions in the workforce development system.” It noted that colleges should prioritize “the development of work-based learning opportunities for students, such as internships or cooperatives, which provide hands-on experiential learning at a worksite.” A concluding recommendation stated, “In crafting a broader workforce development system, we need to be mindful of the full array of occupations required to sustain a thriving regional oil and natural gas economy.” 

The future is now. Even though oil prices rebounded from the pandemic lows and geopolitics are contributing to rising prices, like all other industries, oil and gas will continue to need newly trained talent. With the right education and training at the university level, graduates will be better prepared to enter the field. They will understand the basic concepts of how the industry works; be able to use their knowledge to interpret data to ensure that safety standards are maintained; and also make cost-effective decisions regarding seismic data, and drilling and completion of wells in this incredibly cost-intensive industry. 

Said Chappell, “The industry will need to appeal to new graduates, not as traditional oil and gas companies, but as integrated energy companies with a commitment to a lower-carbon world. This challenge is what will draw the brightest into our industry, and the multitude of perspectives that this new talent pool will bring can only drive innovation.” 

To do this, the study of data science should be included in petroleum engineering courses. Times are changing, the focus is shifting toward more automation, and graduates need to take advantage of machine learning. A college graduate with a degree in petroleum engineering will be in a better position to help the industry long-term, if they also are skilled in data science.  

About the Authors

Vivek K. Prajapati

Petroleum Engineer

Vivek K. Prajapati is a Houston-based petroleum engineer with 15 years of industry experience. Mr. Prajapati holds a master’s degree in petroleum engineering. For more information contact vprajapati.ttu@gmail.com