Leveraging AI to solve novel challenges

Artificial intelligence (AI) is deeply embedded in human life, from unlocking smart phones to self-driving cars. While AI is defeating chess masters and diagnosing disease better than doctors, it’s important to get past the hype and leverage this technology to meet industry challenges. At this point, many readers may have been involved in either successful or disappointing AI initiatives, and experience has shown that most cases display common threads explaining either outcome. There are several key learnings for successfully leveraging AI technologies to solve challenging scientific and industrial problems, particularly in the oil and gas industry. 

For example, using AI for petrophysical analysis is a daunting challenge. However, one sub-problem might be using gamma rays to identify rock types. This is a well-defined problem with clear physical boundaries, inputs and outputs, readily solvable using off-the-shelf classification algorithms. This approach can be repeated for many components of the analysis to create an AI assistant that handles much of the routine or well-defined portions of the workflow, allowing petrophysicists to focus their attention on analysis that cannot be easily defined or automated. 

Fig. 1. Framework of a domain-centric approach when applying AI technologies to solve sanding challenges in wells.

Solving complex challenges does not depend on having complex technology. Instead, solutions demand a diverse team bringing the right experience and technology, working together to understand problems from the top down, and solving them from the bottom up. The power of AI comes in several forms: 

• The ability to perform mundane tasks rapidly and indefinitely. 
• The capacity to combine many simple tasks in an intelligence framework that provides higher-order output. 
• Where a closed-form solution is uncertain or unknown, AI can bridge the gap and learn relationships. 
• Where humans struggle with complexity, AI excels at integrating many dimensions and mixed forms of data. 
• Cloud scalability enables greater exploration of the solution space and the targeting of increasingly larger, more complex challenges. 

ROADMAP FOR SUCCESS 

AI has great potential for improving how humans work and live, yet many challenges to successfully deploying AI solutions remain. While this is still an evolving field, the technology and know-how to solve some complex challenges already exists. Successful projects are planned from the start, with certain key considerations that provide a solid foundation for the entire project. 

Getting aligned on goals. AI projects affect many parts of an organization, from data acquisition and security to the end-user. A project may have a single sponsor, but all stakeholders should be included from the outset to ensure agreement on the problem definition and alignment on the objectives. The problem definition should be clear, with the business case and impact explicit, even if it’s not quantitative. Specific outcomes should be listed and agreed upon. The project team should identify issues around digital readiness, stakeholder alignment, and expectations early on, working constructively with the project sponsor and key stakeholders to find solutions before commencing work. 

Set clear expectations. The significant hype around AI often leads to ambitious projects with sky-high expectations. Managing these expectations involves educating clients and team members to differentiate hype from reality and ensure project goals are achievable. It is also important to understand and convey a client organizations’ readiness in the context of expectations. Organizations with inconsistent data collection and management practices, or insufficient data coverage, will see their ambitions limited or timelines extended to account for these drawbacks. It is essential to remain solution-oriented during the process, as pushing back on unrealistic expectations can be challenging, and clients must continue to view the team as a partner for success rather than a roadblock. 

Use the right tools for the job. Revisiting hype: it is important to use the right tool for the job and not get distracted by buzzwords, industry trends, or someone’s predisposition toward a particular algorithm or solution. In addition to being faster and easier to implement, the simpler the solution, the more predictable its behavior. While algorithms, such as reinforcement learning, sound exciting because they are designed for advanced problem classes, most complex industrial or scientific workflows must be broken down into a series of sub-problems to solve. Each of these sub-problems should be evaluated and solved, based on their merit, not on predisposition to leverage specific technology. If technology is the product on offer, then appropriate problem classes and use cases should be targeted from the outset. 

Leverage domain expertise. Domain experts are often the best source of information. They are great resources for understanding the problem and the intended usage of the deliverable. There are situations where these experts may not embrace AI solutions, due to lack of familiarity. However, giving experts a seat at the table will not only help to gain their support, but it will also result in an arguably better outcome. 

Solutions informed by domain knowledge are generally more robust, provide greater transparency, and lead to enhanced trust and adoption by the end-users. For example, a reservoir engineer can help a data scientist engineer predictive features for an oil rate forecast tool based on saturation, porosity and connectivity data about which the data scientist alone may have no knowledge. 

Create a roadmap. AI projects tend to be complicated and involve a diverse audience; a roadmap to keep the team oriented will provide an important benchmark to measure progress against while ensuring delivery is on track. The roadmap should be built backward to ensure all solutions map directly to outcomes. Roadmaps and milestones should focus on deliverables, centered around what will be done and why, as opposed to how it will be done. The roadmap should be refined with increasing levels of detail—as more is learned throughout the project—to identify bottlenecks and challenges. This iterative process is intentionally designed to allow a diverse audience of stakeholders to quickly agree on goals and kick off the project while providing a mechanism for future realignment as the situation evolves. 

Build on small successes. When building or refining the roadmap, it’s important to design milestones to deliver small successes early on. Complex AI projects often face a host of issues, and reaching preliminary results earlier provides a forum for feedback and discussion. 

Tangible results build momentum for tackling larger and longer-time frame aspects in later phases. It also builds confidence that the team can deliver results and meet expectations. Stakeholders will be asked to justify the project at many junctures; milestone deliverables support them and show that the team is a partner committed to the project’s success. 

CASE STUDY 

Sand is a natural byproduct of oil and gas production and can typically be minimized, using completion or stabilization techniques. However, in certain situations, sand production can only be managed, not prevented. In these cases, operators attempt to maximize production rate while keeping sand production below a threshold. This is challenging, because there are no analytical or physics-based solutions for predicting sand production as a function of well control parameters, such as rate or drawdown. 

The current best practice is largely operator-specific with custom tools, heuristics from historical data, and specialized experts providing experience-based guidance. Without an existing framework, the challenge was to build a sand management platform leveraging machine learning on historical data, integrating operator knowledge, and providing an end-to-end workflow to streamline analysis. 

Phased development. Given this was a first-of-its-kind solution, the initial phases of the project focused heavily on understanding the analysis workflow, structuring operator knowledge, and ensuring alignment on goals and expectations. Once the full scope and ambition of the project were understood, the project was broken down into five phases over three years. 

At inception, high-level goals were agreed upon, while each phase involved a detailed and defined work scope, leaving relevant high-level goals in the backlog. This approach allowed a complex problem with an unproven workflow to be tackled in stages, with short-term objectives defined, based on the situation, while keeping the project oriented toward delivering on long-term objectives. This also enabled the delivery of a minimum viable product (MVP) in phase two, helping to maintain stakeholder support for continuing development of an integrated end-to-end sand management platform. 

Domain-centric approach. Sand management requires a domain-centric approach when applying AI technologies, Fig. 1. Not only do predictive features need to be rooted in underlying principles, but expert knowledge is central to diagnosing sanding risk factors for the solution workflow. 

The project team worked closely with domain experts to understand the workflow and break it down into a series of smaller-scale, solvable problems. Each step was then mapped to a problem class, and an appropriate algorithm was selected. For example, sand rate prediction uses a statistical approach for robustness in a dynamic and data-sparse environment. 

Finally, the numeric portion of the AI workflow was integrated with the symbolic aspect to create a hybrid solution. The numeric portion is typical, in that features are generated and a prediction results. The symbolic aspect leverages an expanded set of features to evaluate expert-defined knowledge statements for specific sanding risk categories and quantify overall sanding risk. In this way, domain expertise incorporates directly into the workflow. 

Platform flexibility. Many aspects of this project—having never been demonstrated before—were unproven and required ongoing research to refine the approach. To conform to the roadmap and continue demonstrating success, a flexible platform was designed to enable ongoing workflow refinement while still delivering new features and capabilities in a stable software product. Three key areas requiring flexibility included predictive features, recommended production and corresponding sanding rate prediction, and expert knowledge definitions. 

Fig. 2. User interface of platform that allows users to directly upload custom predictive features, customize parameters and manage a knowledge base for flexibility specific to each well’s unique situation.

For predictive features, the platform allows users to directly upload custom predictive features such as raw data, user-calculated features or even machine learning model output, Fig. 2. For the prediction, various rates, search algorithms, and feature aggregation options were provided, along with more than 300 customizable parameters. Finally, for knowledge flexibility, a self-contained knowledge editor with creation, customization and knowledge base management capabilities was created. This knowledge editor allows users to define, test and refine knowledge statements at a well-by-well level, providing high-resolution knowledge customization specific to each wells’ unique situation. 

This project was challenging for many reasons, including the highly ambitious scope, the novelty of the problem, complex workflows and stakeholder diversity. While there were challenges on the journey, having a supportive client, an experienced project team, engaged stakeholders and a focus on process improvement, this complex scientific challenge was successfully addressed using AI. The above recommendations were key to project success, as they ensured stakeholders were aligned on goals, deliverables and expectations while maintaining the ability to cooperatively calibrate priorities throughout the project lifecycle.  

About the Authors

Michael Krause

AI Solutions at Beyond Limits

Michael Krause is director at AI Solutions at Beyond Limits, an artificial intelligence engineering company creating software solutions that go beyond conventional AI. Dr. Krause specializes in industrial AI, with experience from bespoke AI solutions at small businesses to digital transformation at large enterprises. He holds a PhD in energy resources engineering from Stanford University.