The landscape of Artificial Intelligence development is undergoing a fundamental shift as industry leaders move away from the trial-and-error nature of "prompt engineering" toward a more disciplined, architectural approach known as "context engineering." While early Retrieval-Augmented Generation (RAG) systems focused almost exclusively on document retrieval—chunking text and using vector searches to find relevant passages—experts now argue that the user’s question itself must be treated as a structured piece of context. This evolution, spearheaded by figures such as Shopify CEO Tobi Lütke and AI researcher Andrej Karpathy, positions question parsing not merely as a preliminary step, but as a critical "brick" in the foundation of enterprise-grade document intelligence.
The Emergence of Context Engineering
In the mid-2020s, the AI community reached a consensus: the bottleneck in LLM performance was no longer just the model’s parameters, but the quality and structure of the data fed into the context window. Tobi Lütke famously proposed the term "context engineering" in June 2025, suggesting it as a more accurate replacement for the often-misunderstood "prompt engineering." This sentiment was echoed by Andrej Karpathy, who noted that as LLMs become more like operating systems, the way we structure inputs—both from documents and from users—requires a formal engineering discipline.
Traditional RAG systems often fail because they treat the user’s query as a simple string for cosine similarity searches. For example, when an insurance analyst asks for a "maximum coverage amount" while explicitly warning the system not to confuse it with a "deductible," a standard vector search might pull in lines containing both terms indiscriminately. This is not a failure of the retrieval algorithm or the LLM’s generative capabilities; it is a context-engineering failure on the question side. The system fails to isolate the signals within the query—the topic, the negative cue, and the expected structural shape of the answer.
The Four Pillars of Enterprise Document Intelligence
To address these failures, the Enterprise Document Intelligence framework organizes the RAG pipeline into four distinct "bricks": document parsing, question parsing, retrieval, and generation. The second brick, question parsing, is unique because it acts as both a consumer and a writer of context.
As a consumer, the question parser makes its own LLM call to analyze the user’s string. This call is highly constrained, utilizing a context window assembled from four typed slots: the system instructions, the user string, the project schema, and the domain-specific vocabulary. By excluding document text and conversation memory from this stage, the system ensures that the parsing is deterministic, reproducible, and free from the bias of irrelevant data.
As a writer, the question parser transforms a raw string into a "typed row" within a database. This row, often referred to as a ParsedQuestion, serves as a contract for all downstream processes. It breaks the query into specific fields such as keywords, intent, and retrieval hints. This structured output ensures that the retrieval and generation modules receive only the information they need, preventing "context bloat" and reducing the likelihood of hallucinations.
A Chronology of Strategy: The LangChain Taxonomy
The formalization of context engineering was further refined by LangChain, which categorized the practice into four canonical strategies: write, select, compress, and isolate. These strategies are now being applied to question parsing to ensure maximum efficiency.
- Write (The ParsedQuestion Contract): Instead of summarizing a question into a new string, the system writes a typed row with named fields. This allows for rigorous testing and auditing. For instance, a schema linter can flag a field if it is never read by a downstream call, ensuring the pipeline remains lean.
- Compress (The Retrieval Brief): The retrieval module does not need to know the final answer’s shape or the suggested model tier. Compression in this context means dropping unnecessary fields to ensure the retrieval detectors (keyword matchers and embedding scorers) focus only on finding the right anchors in the text.
- Select (The Generation Brief): Selection involves picking the right template for the LLM to use. A dispatcher analyzes the parsed question to decide on the chunking strategy and model tier (e.g., opting for a faster, smaller model for simple factual questions versus a more robust model for complex synthesis).
- Isolate (The Clarification Request): When a question is too vague or references missing information, the "isolate" strategy prevents low-quality context from poisoning the pipeline. The system pauses and issues a
ClarificationRequestto the user, ensuring that retrieval only proceeds once the intent is clear.
Case Studies in Structured Question Parsing
The practical application of these strategies is best observed through various industry-specific scenarios. These examples illustrate how different signals within a question trigger specific technical behaviors in the RAG pipeline.
Insurance: Handling Negative Cues
When asked, "What is the maximum coverage amount? Don’t confuse it with the deductible," the parser extracts "maximum coverage amount" and "deductible" as keywords but identifies the intent as "factual." While current schemas are still evolving to handle negative cues as dedicated fields, the structured approach allows the system to log these terms separately, providing a trace for developers to audit why a specific number was chosen during the generation phase.
Legal: Section-Specific Retrieval
In legal inquiries such as "Does the indemnification clause survive termination?", the parser identifies a section_hint. This triggers a section_filter_active flag, instructing the retrieval module to ignore the rest of the document and focus exclusively on the pages identified in the Table of Contents (TOC) as part of the "Indemnification" section. This significantly reduces noise and increases the accuracy of the final answer.
Finance: Layout-Aware Inquiries
For finance questions like "What was the total revenue for Q3 2024, broken down by region?", the parser identifies a layout_hint for a "table." This activates a two-hop retrieval pattern: first, finding the relevant table region, and second, reading the specific rows and headers. This prevents the LLM from misinterpreting tabular data as standard paragraph text.
Medical: The Clarification Loop
In the medical field, precision is paramount. If a user asks, "Is warfarin contraindicated with the current medication list?" without providing the list, the system utilizes the "isolate" strategy. It writes a ClarificationRequest asking the user to specify which document contains the medication list. This prevents the system from making a high-stakes guess based on incomplete data.
Technical Implications and System Architecture
The decision to separate the question parser’s output into four distinct pieces—the parsed row, the retrieval brief, the generation brief, and the clarification request—is driven by operational necessity rather than aesthetic preference.
Engineers note that a merged payload, where all data is sent to every module, creates dangerous "couplings." If the retrieval module begins reading fields meant for the generation module, the system becomes fragile. A change in the generation logic could inadvertently break the retrieval logic. By enforcing typed separation, developers can ensure that a change to one part of the schema has a limited "blast radius."
Furthermore, this separation optimizes caching. Each piece of the parsed question has its own cache key. A simple change to a user’s intent might not require a full re-parsing of the keywords, allowing the system to reuse previously computed data and reduce latency.
Broader Impact on Enterprise AI
The move toward context engineering represents a maturation of the AI industry. As enterprises move from experimental chatbots to mission-critical document intelligence tools, the need for auditability and reliability becomes non-negotiable.
By treating the user’s question as a structured data object, organizations can implement rigorous quality control. Domain experts can audit the ParsedQuestion rows to ensure the system is interpreting industry-specific terminology correctly without ever having to read the underlying Python code.
As we look toward the future, the vocabulary of "prompts" is likely to fade, replaced by a sophisticated language of "briefs," "hints," and "activation flags." This shift ensures that LLMs are no longer treated as "black boxes" that respond to magic spells, but as predictable components in a well-engineered software architecture. The discipline of context engineering, particularly on the question side, is the key to unlocking the full potential of RAG in the enterprise.
