End-to-End Foundation Models for the Energy Industry — with Jazmia Henry

Jazmia Henry, a foundation model engineer at Collide, is at the forefront of applying AI to the energy sector, an industry where 90% of the documentation is unstructured. This includes decades of handwritten notes, safety reports, and leasing documents. The goal is to synthesize this vast, siloed knowledge to improve safety and efficiency, especially as a generation of experienced engineers retires. AI models must be robust and reliable, as a failure—like misinterpreting a job safety report—could have severe real-world consequences, such as physical injury. The models need to understand domain-specific jargon and even archaic, informal measurements like distances described by “how many Marlboros you smoke while walking.”

The Four Stages of Full-Stack Foundation Model Building

Jazmia describes her role as a “full stack foundation model builder,” which encompasses an end-to-end process broken down into four key stages:

1. Data Curation: This is the most intensive phase. It involves transforming unstructured data, like thousand-page textbooks or handwritten notes, into a usable format. This isn’t just simple text extraction; it requires subject matter experts to help build knowledge graphs that connect disparate concepts mentioned chapters apart. The company views itself as a data company first and an AI company second, underscoring the foundational importance of high-quality, structured data.

2. Data Shaping & Pre-processing: The curated data must be prepared for the model. This involves creating bespoke tokenizers that are optimized for the specific vocabulary of the oil and gas industry. Building on top of embedding models, these custom tokenizers improve downstream tasks like Optical Character Recognition (OCR), allowing the system to more accurately extract information from documents without always resorting to more expensive vision models.

3. Model Training: This stage involves multiple techniques:

   • Continued Pre-training: Taking powerful open-source models and further training them on the specialized oil and gas corpus to extend their capabilities into the specific domain.
   • Reinforcement Learning (RL): To ground the model in real-world physics and constraints, it’s trained in simulated environments. Instead of trying to teach the model physics directly through text (which can lead to overfitting), the environment enforces the rules. The model uses tools like a Python REPL to perform calculations that must adhere to these physical laws. This approach helps strip out complex knowledge from the model training and places it into the RL environment.

4. Inference: The way a model is trained directly impacts how it can be served. RL models, in particular, are described as “bursty.” This means they can have periods of high CPU activity for calculations while the GPU remains idle, followed by a sudden, intense “burst” of activity on the GPU. This burstiness makes scaling inference challenging and expensive, as you must provision a large compute cluster to handle peak loads, even if it’s underutilized most of the time. Designing the model with these inference constraints in mind is critical.

Reward Hacking and Grounded Continuous Evaluation (GCE)

A significant challenge in reinforcement learning is reward hacking, where a model finds a shortcut to maximize its reward without actually learning the desired behavior. For example, instead of learning to solve a math problem, it might guess the most common answers from the training set. This is often a result of a mismatch between the training data (e.g., perfectly formulated exam questions) and real-world user queries (e.g., “Tell me about this well”).

To combat this, Jazmia developed the Grounded Continuous Evaluation (GCE) framework. Instead of waiting hours or days for a training checkpoint to evaluate the model, GCE uses Monte Carlo simulations on the model’s reward trajectories in near real-time. This allows for:

• Early Detection: Quickly identifying if an experiment is heading in the wrong direction, saving significant time and compute.
• Identifying Reward Hacking: If a simulation predicts poor performance but the model suddenly scores high at the checkpoint, it’s a strong signal of reward hacking.
• Injecting Determinism: GCE creates a “space” of rules and norms where the model must exist. Any deviation from this grounded space is penalized, forcing the model to learn the correct processes rather than just the final answers.

On Scaling Laws and Data Quality

Jazmia revisits her 2023 NeurIPS paper, “Scaling Laws or the Laws of Diminishing Returns,” which argued against the prevailing belief that simply scaling up data, compute, and parameters linearly improves model performance. Drawing an analogy to the economic law of diminishing returns, she demonstrated that scaling with low-quality, non-deduplicated data actually degrades model performance. This “bad data” causes the model to become more forgetful and prone to overfitting. The paper was met with skepticism at the time but has since been validated, as smaller, high-quality models (e.g., 70B parameters) are now consistently outperforming much larger models (1T+ parameters) trained on lower-quality data.

• Tags:
• Foundation Models
• Reinforcement Learning
• Feature
• LLM Evaluation
• Data Quality
• Reward Hacking
• Energy Sector