RL vs. GEPA: Prompting works to extract knowledge

Every organization runs on artifacts. Documents, past agent traces, tickets, pull requests, and the state of tools like a chat workspace, an ERP, or a code repository accumulate over time, and together they encode the semantics of a domain: how things work here, what was decided before, which facts and conventions hold. An agent dropped into that setting needs that knowledge to act well, but it is often scattered across an enterprise and not stored in a way the agent can absorb.

The task for agentic knowledge engines is therefore to learn semantics from existing artifacts and expose that knowledge to agents in a form they can use. The mechanism is simple: one model reads the source material and creates a "cheat sheet" that carries the useful knowledge forward so a downstream model performs better with it than without.

In recent blog posts, we've seen different teams take complex, hard-to-interpret approaches to improving the quality of knowledge distillation. We find that while these methods do work well, equivalently good results can be achieved without sacrificing interpretability.

Two ways to raise cheat-sheet quality

We see two broad levers to improve knowledge extraction:

Reinforcement learning tweaks a model's weights to make it better at summarization.In a recent post by Applied Compute, reinforcement learning was used to improve a model that distilled knowledge out of short stories to answer multiple-choice questions. We take those published results as the reference for what RL achieves here.

Prompt optimization leaves the weights alone and changes what we ask for. We search over the instruction that tells the model how to act, and keep the instructions that work best. The specific optimizer we use is GEPA, which proposes instruction edits, tests them for accuracy, and reflects on the failures to propose the next edit. Nothing about the model changes; only the text of the instruction moves.

The two levers differ in cost and in what they leave behind. Reinforcement learning needs a training loop and produces a new set of weights. Prompt optimization needs only inference and produces a new instruction which is fully interpretable by human reviewers.

We ask how far the second approach gets, and what advantages we get by choosing prompt optimization.

Method

Task and data. QuALITY (Pang et al. 2022), is a dataset of public-domain short stories of 1,800 to 6,000 words with human-written four-way multiple-choice questions. We used the official train and dev splits, deduplicated to 150 train and 115 dev unique articles, roughly 17 questions per document.

Protocol. A summarizer reads the full document and writes a summary about 20 times shorter than the original story. A separate frozen reader model answers the document's questions using only that summary. Downstream accuracy is the reward.

Models. gpt-4.1-nano serves as both the summarizer and the frozen reader, one small off-the-shelf model in both roles, chosen to sit near Applied Compute's small-model setup.

Splits and optimizer. 30 training documents for optimization, split 20/10 into the optimizer's train and validation sets. 60 held-out documents scored identically across every condition. The prompt optimizer, GEPA, ran on its light preset with a 420 metric-call budget and a reflection model equal to the summarizer being optimized.

Measuring cheat-sheet quality, and the setup

We cannot reproduce the reference setup from Applied Compute exactly, and almost everything that fixes an absolute accuracy number differs between the two experiments: a different slice of QuALITY documents, a different reader model, a different model writing the cheat sheet. Each of those shifts raw accuracy on its own, independent of how good the cheat sheet is, so a percentage from one setup cannot be laid beside a percentage from the other. We therefore score every experiment by the fraction of the achievable performance gap that the approach closes.

Gap closed = (accuracy − floor) / (ceiling − floor).

The floor is the frozen reader answering with no context at all. The ceiling is the same reader given the full document. A cheat sheet that closes 70 percent of the gap recovers 70 percent of the accuracy the full document would have provided.

The point of the normalization is that floor and ceiling are measured inside each setup. Two experiments that cannot be run identically can still be compared on how much of their own achievable gap each closes, which is the metric the post reports and the one we adopt.

Every model we use is off-the-shelf and callable through an ordinary API, with no training infrastructure, which is deliberate: part of the question is whether the result needs an RL stack at all, or reproduces with commodity models and a search anyone can run.

The result

Prompt optimization improved the summarizer's instructions to maximize downstream accuracy. Gap closed rose from 55.8 percent under a naive prompt to 71.9 percent under the optimized one. Because both conditions were scored on the same 60 held-out documents, the per-document scores pair directly: the accuracy improvement is 4.1 points, with a bootstrap 95 percent confidence interval of 1.7 to 6.5 and p near 0.0004. The optimized prompt was better on 31 documents, worse on 15, tied on 14.

Set against the reported RL result from Applied Compute, the two land in about the same place: 71.9 percent gap closed for prompt optimization, 72.1 for reinforcement learning.

Why steering with language is enough here

The result is less surprising than it first looks, and spelling out why also says when the approach should be expected to work.

The summarizer is already a capable, instruction-following model. The ceiling condition shows the information is recoverable, and the model produces fluent summaries under the naive prompt already. What the task asks is narrower: point an existing capability at a specific downstream use, answering questions from the summary later, rather than install a capability the model lacks.

For directing behavior a model already has, natural language is the native interface. Instruction-tuned models are built to map written directions onto behavior, so an edit to the prompt moves along the same axis the model was trained to respond to.

It also matters that the failures costing accuracy on this task were failures of specification, not capability. The naive prompt left much of what the reader needed unsaid, so the summaries omitted detail the questions turned on. A missing specification is exactly what a written instruction fixes directly.

Overall, reasoning about documents, or other enterprise work products more broadly, is not a new capability that needs to be trained into LLMs. Rather, by directing AI to think the right way through clear instructions, we can close the gap with complex, expensive methods like RL.

Reading the optimized prompt

Parity in accuracy is one half of the argument. The other half is that we can say why the optimized prompt helped, which is not something trained weights allow. RL folds its improvement into millions of parameters that cannot be inspected. Prompt optimization writes its improvement into an instruction string, so the change is available in plain text and its effect can be traced to a specific edit and checked.

Here that is concrete.The optimized instruction, reproduced in full in the appendix, is legible enough to say exactly how it adds value: it points the summary at answering later questions rather than conveying the gist, asks to keep specific facts, names, numbers, causes, and motivations, tells the model not to invent and to flag uncertain inferences, and states an explicit target length. A human can read every one of those decisions and keep, cut, or edit any of them. None of that is legible in a set of updated weights.

Secondarily, while further tuning may be needed, a prompt optimization is fundamentally portable — we can use the same prompt with a different language model in the future.

This is the practical difference for humans in the loop: GEPA hands back a paragraph you can read, question, and carry to the next model, while the RL hands back an opaque checkpoint.

Appendix

A. Raw accuracy

gpt-4.1-nano as summarizer and reader, 60 held-out documents:

Gap closed, prompt optimization against the Applied Compute RL result. Their figures are read from the published QuALITY chart (floor about 36 percent, baseline about 50.5 percent, RL about 60.5 percent, ceiling about 70 percent):

B. Prompts

Baseline prompt

Produce a short summary that preserves as much information and key content as possible.

Optimized instruction

Produce a comprehensive yet concise summary of the given document, approximately 200-300 words, that captures all crucial information necessary for accurate understanding and detailed question answering. The summary should include:

- The main plot points, their sequence, and outcomes.

- Character motivations, feelings, relationships, and key actions.

- Thematic messages, societal or political implications, and underlying messages.

- Critical references, symbols, or moments that influence interpretation.

- The overall tone, mood, or underlying messages that define the work.

Pay careful attention to domain-specific details relevant to the content, such as references to scientific theories, historical figures, technological concepts, or societal constructs, and include subtle nuances that clarify characters' perspectives and the overarching themes. The summary should be as thorough as possible within the given length constraints, emphasizing factual correctness and relevance to potential questions about the material.