TABLE OF CONTENTS
FEATURED
Context Engineering: From Prompt Engineering to Infrastructure
Mubbashir Mustafa
11 min read
In 2023, prompt engineering was the hottest role in tech. Companies hired prompt engineers at six-figure salaries to craft the perfect instructions for language models. Conferences dedicated entire tracks to prompt optimization. "Prompt engineering" had its own Wikipedia page, its own certifications, and a cottage industry of courses promising to unlock AI's potential through better prompting.
By 2025, the role was largely irrelevant. Not because prompting doesn't matter, but because it doesn't scale. A well-crafted prompt can make a single interaction better. It cannot solve the systemic challenges of running AI agents across an enterprise: maintaining context across conversations, integrating data from dozens of systems, ensuring consistency across hundreds of daily decisions, and governing agent behavior at organizational scale.
The industry term for what replaced it is "context engineering." It emerged in mid-2025 when Shopify CEO Tobi Lutke and former Tesla AI director Andrej Karpathy independently endorsed the concept, triggering rapid adoption across the AI ecosystem. Within months, dbt, Elastic, Cognizant, Confluent, Anthropic, and LangChain had all reframed their technical messaging around context engineering. Adoption grew 400% year-over-year. The shift wasn't marketing. It was a recognition that production AI requires a fundamentally different discipline than prototype AI.
What Prompt Engineering Actually Did (and Its Limits)
Prompt engineering optimized the input to a language model for a specific interaction. Write a clear system prompt. Provide few-shot examples. Structure the output format. Add chain-of-thought instructions. These techniques genuinely improve model outputs for individual queries.
The limits become visible in production. Consider an enterprise customer service agent. A prompt engineer crafts a system prompt that instructs the agent to be helpful, cite sources, and escalate complex issues. In a demo, this works well. In production, the agent encounters a customer who mentions a product that was renamed three months ago. The agent doesn't know about the rename because the prompt doesn't contain that information. It hallucinates the old product details because that's what its training data contains.
Prompt engineering can't solve this because the problem isn't the prompt. The problem is that the agent lacks access to current enterprise knowledge. No amount of prompt optimization gives the agent information it doesn't have.
The pattern extends across enterprise use cases. An agent that needs to reason about cross-system dependencies can't do it through prompt instructions alone. An agent that needs to maintain context across a multi-day workflow can't store state in a prompt. An agent that needs to respect role-based access controls can't enforce them through system message instructions. These are infrastructure problems wearing a prompt engineering mask. Learn more
What Context Engineering Actually Means
Context engineering is the discipline of designing and building the systems that provide AI agents with the right information, at the right time, in the right format, with the right access controls. It's not about crafting better prompts. It's about building the infrastructure that determines what goes into the prompt, and what surrounds the prompt in the broader system architecture.
The distinction maps to three layers of capability.
System-level context design determines what an agent knows about the world. Which enterprise systems can it query? What data does it have access to? How is that data structured, filtered, and prioritized? This layer is pure infrastructure: integration connectors, data pipelines, knowledge graphs, and entity resolution systems. It answers the question "what information exists for this agent to use?"
Retrieval context engineering determines how the agent finds relevant information for a specific query. This layer combines vector search (semantic similarity), graph traversal (relationship-aware retrieval), and semantic layer queries (business-logic-consistent data retrieval). It answers the question "given what this agent needs to do right now, which information is most relevant?"
State and memory management determines what the agent remembers across interactions. Short-term memory for the current conversation. Long-term memory for patterns and preferences observed across many interactions. Shared memory for context that multiple agents need to access. This layer answers the question "what does this agent know about what happened before?" Learn more
Prompt engineering operates within the second layer. Context engineering designs all three.
Why the Shift Happened in 2025
Three production realities forced the shift from prompting to infrastructure.
Agents replaced chatbots. In 2023, most enterprise AI was conversational: a user types a question, the model responds. Prompting works well for this interaction pattern. By 2025, enterprises were deploying autonomous agents that take actions across systems, run background processes, and coordinate with other agents. These agents need persistent state, cross-system data access, and governance controls that prompt instructions can't provide.
Scale broke manual approaches. A prompt engineer can optimize interactions for one agent serving one use case. When an enterprise runs 20 agents across 10 departments, the prompt engineering approach requires 20 separate optimization efforts, each with its own context assembly logic, each maintained independently. Context engineering consolidates this into shared infrastructure that every agent benefits from. The math is straightforward: if each agent requires two weeks of prompt optimization and ongoing maintenance, 20 agents require 40 engineer-weeks of initial optimization and a permanent maintenance burden. If those 20 agents share context infrastructure, the infrastructure investment is made once and benefits every agent.
Accuracy demands increased. Early enterprise AI applications were advisory: "here's a summary, a human will review it." Newer applications are operational: "execute this trade, file this ticket, update this record." The accuracy bar for operational AI is orders of magnitude higher than for advisory AI. Meeting that bar requires grounding in enterprise data, not better prompting. Learn more
Context Engineering in Practice
Consider a concrete example: an enterprise AI agent handling internal IT support.
Under the prompt engineering approach, the agent has a system prompt that says "You are an IT support agent. Help employees resolve technical issues. Escalate to the engineering team when you can't resolve an issue directly." The agent receives the user's question, generates a response based on its training data, and hopes for the best.
Under the context engineering approach, the agent operates within a rich infrastructure layer. When an employee submits an IT issue, the system first identifies the employee (pulling their role, team, location, and device profile from HR and asset management systems). It then identifies the relevant systems (checking the service registry and dependency graph for any ongoing incidents that might be related). It retrieves relevant knowledge (searching internal documentation, past incident reports, and runbook entries). It checks permissions (verifying what actions this agent is authorized to take for this user's role and this category of issue). Only then does it assemble the prompt, with all of this context included.
The difference in output quality is not marginal. The prompt-engineered agent might suggest restarting the application. The context-engineered agent knows that the application depends on a database that's currently in a degraded state due to a maintenance window, that the employee's team was notified about this maintenance window yesterday, and that the expected resolution time is 45 minutes. It provides this context and suggests waiting rather than troubleshooting a problem that will resolve itself.
This isn't a better prompt. It's a better system.
The contrast becomes even starker in multi-step workflows. Consider the same IT support agent handling a request to provision a new development environment. Under prompt engineering, the agent follows scripted instructions: "create a VM with these specs." Under context engineering, the agent checks the requesting developer's team (from the HR system), their project allocation (from the project management system), the team's standard environment configuration (from the infrastructure registry), the current cloud capacity and cost allocation (from the cloud management platform), and the approval requirements for this resource type (from the governance framework). It then provisions the environment with the correct configuration, charges it to the right cost center, and notifies the relevant approvers. Each piece of context comes from a different system. The agent doesn't need to be told where to look. The context infrastructure routes the right information to the right agent at the right time.
What Infrastructure Context Engineering Requires
Building context engineering capabilities requires four infrastructure components. Learn more
A real-time integration layer that connects enterprise systems and maintains current data. Not batch exports. Not static snapshots. Live connections with configurable synchronization frequencies. When data changes in a source system, the change propagates to the knowledge layer within minutes (or seconds, for time-sensitive data types). Every enterprise integration platform claims real-time capability. The test is whether entity resolution works across systems: can the platform correctly identify that "jsmith@company.com" in your email system, "John Smith" in your HR system, and "JS-42871" in your ticketing system are the same person?
A knowledge graph that models entities and relationships across the enterprise. The knowledge graph is what makes cross-system reasoning possible. It's the data structure that answers "which services depend on this database," "who owns the on-call rotation for this team," and "what is this customer's complete interaction history across support, sales, and billing." Vector databases store documents. Knowledge graphs store reality.
A semantic layer that enforces consistent business definitions across data sources. "Active customer" means different things in your CRM, your billing system, and your product analytics. The semantic layer reconciles these definitions so that when an agent queries "active customers," it gets a consistent answer regardless of which underlying system provides the data. Without a semantic layer, agents inherit the definitional chaos of your enterprise data, and their outputs reflect that chaos.
A governance layer that controls what context each agent can access. Not every agent should see every piece of enterprise data. A customer-facing agent shouldn't access internal financial data. An engineering agent shouldn't see employee compensation information. Context engineering includes designing the access control rules that determine which context flows to which agents, and enforcing those rules at the infrastructure level rather than through prompt instructions. Learn more
The governance layer is where context engineering diverges most sharply from prompt engineering. In a prompt engineering world, access controls are instructions in the system prompt: "do not access financial data." These instructions are suggestions, not enforcement. Models can and do ignore them under certain conditions. In a context engineering world, the agent literally cannot access financial data because the integration layer doesn't provide it. The agent's context window contains only the data it's authorized to see. You can't hallucinate from data you never received.
The Measurement Gap: How to Know if Context Engineering Is Working
Organizations transitioning from prompt engineering to context engineering need clear metrics to validate the investment. Three measurements capture the impact.
Grounding rate measures what percentage of an agent's outputs are traceable to source data. A prompt-engineered agent with a 40% grounding rate produces outputs where 60% of the content is generated from the model's training data (or fabricated). A context-engineered agent with an 85% grounding rate produces outputs overwhelmingly derived from actual enterprise data. Track grounding rate per agent and set minimum thresholds by use case: operational agents that take actions should have grounding rates above 90%.
Context assembly latency measures how long it takes to gather and deliver relevant context to an agent for each query. If context assembly adds three seconds to every interaction, user experience suffers. Well-designed context infrastructure delivers assembled context in under 500 milliseconds for most queries. If latency is high, the bottleneck is usually in the integration layer (slow API calls to source systems) or in the knowledge graph (inefficient relationship traversal). Both are solvable with infrastructure optimization.
Context utilization measures what percentage of the context provided to an agent actually influences its output. If you're assembling 8,000 tokens of context but the agent's response only references 1,500 tokens, you're either providing too much context (which increases cost and can confuse the model) or the wrong context. Optimizing context utilization means providing precisely the information the agent needs and nothing more.
The Adoption Timeline
The shift to context engineering is following a predictable adoption curve. In 2025, early adopters (primarily large technology companies and well-funded startups) built context engineering infrastructure internally. In 2026, mid-market enterprises are evaluating platforms that provide these capabilities as managed services. By 2027, context engineering will be the default approach for any serious enterprise AI deployment. Learn more
The early movers are already building competitive advantages. Companies that adopted context engineering in 2025 are now 12-18 months ahead of those still relying on prompt optimization. Their agents have broader enterprise context, higher accuracy, and faster deployment cycles. As AI becomes a standard enterprise capability rather than a differentiator, the gap between context-engineered and prompt-engineered AI programs will determine which enterprises extract real operational value and which remain stuck in the pilot phase.
Three indicators suggest your organization is ready for the shift. First, you're running more than three AI agents in production, and prompt optimization is producing diminishing returns. Second, your agents need data from multiple enterprise systems, and each integration is a custom project. Third, you've had accuracy or hallucination incidents that better prompting couldn't prevent.
If any of these apply, the bottleneck isn't your prompts. It's your infrastructure. Context engineering is the discipline. Grounding infrastructure is what you build. The companies that recognized this early, including LinkedIn (78% accuracy improvement with knowledge graph integration), organizations using dbt's Semantic Layer (83% accuracy), and enterprises using Looker's semantic layer (two-thirds error reduction), are already seeing the results.
The question isn't whether to shift from prompt engineering to context engineering. The question is whether to build the infrastructure yourself (12-18 months, $2-5M in engineering cost) or adopt a platform that provides it as a managed capability. For most enterprises, the math favors the platform. Your competitive advantage isn't in building knowledge graph infrastructure. It's in the agents and applications you build on top of it. Learn more
The companies that get context engineering right in 2026 will have an 18-month head start on those that wait. That head start compounds: more connected systems mean smarter agents, smarter agents mean faster deployment of new use cases, and faster deployment means organizational learning that accumulates over time. Context engineering isn't a technology trend to monitor. It's an infrastructure decision to make.
The good news: you don't have to build context engineering infrastructure from scratch. Platforms exist that provide the integration layer, knowledge graph, semantic layer, and governance framework as managed capabilities. The investment shifts from building plumbing to building agents that deliver business value. That's the right allocation of engineering talent for any enterprise where AI infrastructure is a means to an end, not the end itself.
Context engineering requires infrastructure, not better prompts. Rebase provides the foundation: a live knowledge graph connecting 100+ enterprise systems, real-time synchronization, and relationship-aware retrieval. Move from prompting to context engineering: rebase.run/demo.
Related reading:
AI Grounding Infrastructure: The Operating System for Enterprise AI
Context Engine vs RAG: What's the Difference?
What is a Context Engine?
Enterprise AI Infrastructure: The Complete Guide
AI Agent Orchestration: The Enterprise Guide
Ready to see how Rebase works? Book a demo or explore the platform.
![](data:image/svg+xml,<svg display="block" role="presentation" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><g d="M 0 24 L 0 0 L 24 0 L 24 24 Z M 22.223 0 L 1.772 0 C 0.792 0 0 0.773 0 1.73 L 0 22.266 C 0 23.222 0.792 24 1.772 24 L 22.223 24 C 23.203 24 24 23.222 24 22.27 L 24 1.73 C 24 0.773 23.203 0 22.223 0 Z M 7.12 20.452 L 3.558 20.452 L 3.558 8.995 L 7.12 8.995 Z M 5.339 7.434 C 4.195 7.434 3.272 6.511 3.272 5.372 C 3.272 4.233 4.195 3.309 5.339 3.309 C 6.478 3.309 7.402 4.233 7.402 5.372 C 7.402 6.506 6.478 7.434 5.339 7.434 Z M 20.452 20.452 L 16.894 20.452 L 16.894 14.883 C 16.894 13.556 16.87 11.845 15.042 11.845 C 13.191 11.845 12.909 13.294 12.909 14.789 L 12.909 20.452 L 9.356 20.452 L 9.356 8.995 L 12.769 8.995 L 12.769 10.561 L 12.816 10.561 C 13.289 9.661 14.452 8.709 16.181 8.709 C 19.786 8.709 20.452 11.081 20.452 14.166 Z" fill="transparent" height="24px" id="ubvjAgDes" width="24px"><path d="M 0 24 L 0 0 L 24 0 L 24 24 Z" fill="transparent" height="24px" id="qp5kjBCzR" width="24px"/><path d="M 22.223 0 L 1.772 0 C 0.792 0 0 0.773 0 1.73 L 0 22.266 C 0 23.222 0.792 24 1.772 24 L 22.223 24 C 23.203 24 24 23.222 24 22.27 L 24 1.73 C 24 0.773 23.203 0 22.223 0 Z M 7.12 20.452 L 3.558 20.452 L 3.558 8.995 L 7.12 8.995 Z M 5.339 7.434 C 4.195 7.434 3.272 6.511 3.272 5.372 C 3.272 4.233 4.195 3.309 5.339 3.309 C 6.478 3.309 7.402 4.233 7.402 5.372 C 7.402 6.506 6.478 7.434 5.339 7.434 Z M 20.452 20.452 L 16.894 20.452 L 16.894 14.883 C 16.894 13.556 16.87 11.845 15.042 11.845 C 13.191 11.845 12.909 13.294 12.909 14.789 L 12.909 20.452 L 9.356 20.452 L 9.356 8.995 L 12.769 8.995 L 12.769 10.561 L 12.816 10.561 C 13.289 9.661 14.452 8.709 16.181 8.709 C 19.786 8.709 20.452 11.081 20.452 14.166 Z" fill="rgb(234, 241, 255)" height="24px" id="W5lcXBfNC" width="24px"/></g></svg>)