November 2014

Energy issues

Use of “Big Data” defines industry’s generations

William J. Pike / World Oil


In late August, one of my nephews enrolled in the engineering school of a well-known Texas university. He is considering a degree in petroleum engineering. Being the only member of the extended family actively engaged in the oil and gas industry, I was asked to advise my nephew on the merits, and the difficulties, of pursuing a career in the industry. The merits were not too difficult. But, I had to do a lot of thinking about the difficulties because, in terms of technological and operational issues, today’s industry is definitely not the one I grew up in.

But, while advances in technology define the new oil and gas environment, it’s the generation of data, and its use and storage, that really divide the new generations from mine. In the larger context, it’s the cell phone generation versus the Princess phone generation. The fact that all you younger readers are now Googling “Princess phone” on your mobile phones defines the divide nicely. You are accessing information from a large database using technology designed to access and manage data almost instantly. We only dreamed of that in the past, if we thought of it at all.

Historical data practices. As late as the mid-1980s, the bulk of our data existed on paper. Up to that date, the majority of the data that my drilling operations generated were contained in the morning reports. These were accumulated in red, two-hole-punch binders. That was our data collection system. I suspect/know that the morning report that I called in was also recorded on paper. Where the information went after its initial use is a mystery to me. I would guess that it is now long gone.

Not that it matters. The data we collected, most at a single point in the day, was elemental, including such things as weight-on-bit, depth, mud weight, rpm, hours drilling in the last 24, penetration rate and similar, single-point information. Of course, we also generated mud logging data and logs, which were used, together with our elemental recorded data, to design offset wells—by hand, no less. As well as I remember, that was as far as data collection and integration went in the drilling industry prior to the mid-1980s.

Things were not much better in the production arena. Well test data from stationary and portable test separators were recorded on circular charts, with various colored pens drawing squiggly lines. Fluid levels were determined by firing a 10-gauge or 45-caliber explosive pulse into the annulus, to create a reflective wave that bounced off casing collars, allowing you to calculate the depth to the fluid on a paper chart. Daily production numbers were recorded by hand on field sheets by pumpers. Like the paper drilling information, it is hard to tell where this paper trail ended, but I suspect that the data are long gone.

Evolution of Big Data. My nephew’s world will be different. It will be defined by “Big Data,” the new catch phrase for the data revolution. The central element of most exploration, development and operational programs these days, Big Data comprises a multi-step data process: data collection, processing, storage and integration in multi-disciplinary solutions environments. 

Far from my “red folder” system, data today are collected from numerous sources, continually. Electronic instrumentation and smart devices are everywhere—downhole, in surface facilities, on export systems, and on and on and on—collecting data in real time, and tons of it. Data overload has been an increasing burden for at least two decades. That overload is complicated by inconsistencies in the format of the data collected. In short, the complexity and intensity of data collection, alone, are enough to make your knees weak.

Once collected, unimaginable quantities of these data must be stored. This requirement has spurred significant advances in storage capability and capacity, so much so that storage has now been fairly well eliminated as the bottleneck in Big Data. It is interesting to enter a sophisticated data storage site and attempt to estimate how much paper data could be stored in the same space. I am guessing it is on the order of 1% to 5% of the digital data stored in the same space.

And, then, there is the data processing and integration so fundamental to real-time operations.  In my time, all the paper data went to the engineering staff, to be analyzed and integrated. I always believed data processing and integration to be a thankless, wearisome job, based on the general demeanor of those who ended up with that responsibility. But, that is what it was. The term, real time, was generally used to denote actual time, as in “this weekend, I got some ‘real’ time off.” Data management and processing in real time never occurred to us, even when we received our first, primitive calculators in the early 1980s.

The problem with responding to my nephew is not that I don’t understand how the upstream world has changed with Big Data. I do. But that is a long way from being able to use Big Data, much like it is a long way from understanding lunar landings to being able to land a module on the moon.

So, Robert, I am afraid I can’t offer too much advice on the challenges of working in the industry today, especially when it comes to the dominant role of Big Data. But, if you want to know how to kick-start an 8½ × 10 Ajax, or shoot a fluid level, or strap out of the hole without RFID tags for backup, give me a call.

Oh, and if you think I am being nostalgic, I am not. I am envious. wo-box_blue.gif 

About the Authors
William J. Pike
World Oil
William J. Pike has 47 years’ experience in the upstream oil and gas industry, and serves as Chairman of the World Oil Editorial Advisory Board.
Connect with World Oil
Connect with World Oil, the upstream industry's most trusted source of forecast data, industry trends, and insights into operational and technological advances.