How does post-training help without proprietary data?

What's the point of specializing AI models?

The primary value of post-training is to help models learn and generalize knowledge that was not sufficiently internalized during pre-training. Broadly, there are three reasons to specialize a model:

1. Teaching models proprietary knowledge. Enterprise data, specialized workflows, and institutional expertise are largely unavailable to frontier labs. Building agents that reason effectively over this information requires additional specialization. The result is intelligence specific to an individual business.

2. Improving capabilities in domains already represented in frontier training data. For example, teaching models to reason more effectively about publicly available financial information. This is an increasingly difficult proposition to justify as frontier model capabilities rapidly advance.

3. Aligning behavior with specific preferences. Frontier models are tuned to appeal to an average user. Post-training can further steer their behavior toward particular preferences for communication style, brevity, formatting, and other characteristics.

These distinctions matter because improvements to a particular benchmark does not necessarily imply improvements in domain knowledge or reasoning.

Consider financial analysis of public companies. Frontier labs have trained their models on enormous volumes of SEC filings, analyst commentary, and similar material. Samaya's FrontierFinance benchmark, which relies entirely on publicly available information, therefore represents a interesting test of post-training that is quite different from most of Sphinx's work, which focuses on specialization using proprietary data.

Our central finding is that while Sphinx pushes the cost-accuracy Pareto frontier on FrontierFinance, almost all of its gains can be retained with just two learned instructions about how to present answers. This calls into question whether sophisticated post-training is necessary for this benchmark at all.

Sphinx improves the Pareto frontier beyond models like Samaya's LoRA-fine-tuned Sterling, but a surprising and somewhat negative result emerges from our ablations: removing nearly all of Sphinx's learned knowledge has no statistically distinguishable effect on performance. What remains is essentially a better system prompt, rather than any substantive knowledge about finance.

Our interpretation is that much of the progress on FrontierFinance comes from learning to satisfy its grading rubrics rather than genuinely improving financial reasoning. This is a degenerate case of post-training – preference optimization masquerading as domain specialization.

The FrontierFinance benchmark

FrontierFinance evaluates how well AI agents perform financial research tasks across the investment workflow. Unlike traditional financial benchmarks focused on extracting numbers from filings, FrontierFinance emphasizes open-ended questions requiring information retrieval, synthesis, and financial reasoning.

The benchmark consists of 220 expert-written questions across six categories: company research, financial modeling, earnings analysis, industry research, investment screening, and catalyst monitoring. Questions range from understanding a company's business model and unit economics to analyzing earnings and comparing investment opportunities.

Each question is accompanied by detailed, expert-authored grading rubrics specifying what a high-quality answer should contain. Across the benchmark, there are 11,543 individual rubric items, averaging roughly 52 per question. Responses are evaluated by LLM judges that determine the fraction of rubric criteria satisfied.

The benchmark and evaluation code are publicly available, allowing models, agent harnesses, and post-training approaches to be compared on answer quality and inference cost. Importantly, the questions can be answered using publicly available information, so FrontierFinance is potentially a useful test of whether specialization improves financial reasoning beyond the capabilities already present in frontier models.

Sphinx pushes the FrontierFinance Pareto frontier

Unlike benchmarks built around a fixed corpus, FrontierFinance requires agents to retrieve information from the open internet. To train Sphinx, we therefore generated a synthetic dataset using GPT-6.1 Sol.

To train Sphinx without risking data leakage or relying on actual benchmark answers, we used GPT-6.1 Sol to generate entirely synthetic training data based on Samaya's benchmark description. The model created novel financial analysis tasks, synthetic evaluation rubrics, and simulated agent trajectories illustrating potential solution paths and failure modes. Sphinx then trained on 250 of these synthetic examples to construct its context graph.

At inference time, Sphinx dynamically retrieves and injects relevant knowledge from this graph into an agent's context. Because Sphinx operates at the context layer rather than modifying model weights, the same learned knowledge can be used with different models and agent harnesses.

Our results establish a new cost-accuracy frontier:

• GPT-6.1 Sol + Sphinx achieves 61.7% accuracy, exceeding the results reported on the FrontierFinance leaderboard.

• GPT-6 Luna + Sphinx achieves 57.2% accuracy at $0.26 per rollout, extending the Pareto frontier beyond Samaya's published results

These results suggest that Sphinx can outperform specialized approaches without modifying model weights. However, understanding what Sphinx actually learned leads to a more interesting conclusion.

Preference alignment alone solves FrontierFinance

While running baseline evaluations, we noticed an unexpected result: GPT-5.6 Sol outperformed GPT-6 Sol when both were used with the default FinanceAgent v2 harness.

On closer inspection, GPT-6 Sol tended to produce shorter, more concise answers, often omitting intermediate calculations that GPT-5.6 Sol included in its final responses. Because FrontierFinance grades against detailed rubrics, this counted against the newer model.

For example, when calculating a margin, GPT-6 Sol returned the final percentage with supporting citations. GPT-5.6 Sol instead explicitly enumerated revenues, costs, and intermediate calculations before presenting the same result. While both answers are correct, the latter received more credit for satisfying the rubric.

Specifically, we considered the FrontierFinance task “For the five largest telecom companies by revenue, pull capital expenditures as % of revenue for each of the past 5 years.”. One of the relevant companies is Verizon, and FrontierFinance's rubrics ask us to present three facts:

1. “Provides information for Verizon Communications Inc. (VZ) stating that Revenue for 2023 was USD 133,974 Million”

2. “Provides information for Verizon Communications Inc. (VZ) stating that Capital Expenditures (including capitalized software) for 2023 was USD 18,767 Million”

3. “Calculates Capital Expenditures as a percentage of Revenue for Verizon Communications Inc. (VZ) for 2023 as 14.01%”

We found that several agents we tested were judged to be correct on the third point, but as incorrect on points 1 and 2. This was not because that information wasn't found, but because it was omitted from the final answer. For example, with GPT-6 Sol, we prompted the same agent session used for this task to provide revenue and capex, and as expected these numbers had already been retrieved — they just weren't in the final writeup.

This raised a natural question: was post-training improving financial reasoning, or simply teaching the agent how to present answers in the format FrontierFinance rewards? We investigated this through an ablation study.

Our original Sphinx context graph contained approximately 400 nodes learned from the synthetic training set. We removed all but two nodes, retaining only instructions to produce thorough answers and explicitly show intermediate work. Neither node contained substantive finance-specific knowledge.

For example, the following context which purely discusses formatting and organization was provided by Sphinx to agents: Recover the numerical inputs, component values, period values, calculations, and relevant qualitative findings from the source material already consulted. Organize them by the entities, periods, and measures relevant to the original question, alongside the resulting ratios, changes, totals, or conclusions. Retain units, definitions, qualifications, and citations. Include supporting numerical facts even where the question requested only derived measures or conclusions. 

Despite removing most of the graph, a standard Codex harness equipped with these instructions achieved performance statistically indistinguishable from the full Sphinx graph.

This is a striking negative result. It suggests two things:

1. Much of the apparent benefit of specialization on FrontierFinance is preference alignment, not improved financial reasoning. The gains largely come from learning how to satisfy the grading rubrics, rather than learning to retrieve or reason about financial information more effectively.

2. Sophisticated post-training is unnecessary for this benchmark. If a few static instructions can capture nearly all the benefit, then weight-based fine-tuning and Sphinx's in-context learning are overkill. A context graph containing two presentation instructions is not meaningfully different from a well-engineered system prompt.

An illusion of vertical AI

This result points to a broader question about when vertical AI is useful, and when it may provide little edge to end users.

A specialized agent can substantially outperform a general-purpose alternative on a domain-specific benchmark without actually knowing more about that domain. Instead, it may have learned the conventions of how certain people in that field prefer information to be presented, e.g. which calculations to show, how to structure an analysis, what details to emphasize, and what language to use.

These preferences matter, but they are often much easier to learn than underlying domain expertise. In some cases, the apparent value of expensive domain-specific fine-tuning may be captured by a handful of instructions in a system prompt.

This distinction is increasingly important as frontier models become even more intelligent. General knowledge of finance, law, software engineering, and other professions is not proprietary, so as frontier labs continue to train on enormous amounts of publicly available material, the incremental value of teaching models this knowledge is diminishing. Similarly, conventions about how professionals communicate and structure their work can often be captured through relatively lightweight steering.

The durable opportunity for specialization is not teaching AI to work like a smart financial analyst, lawyer, or engineer. It's teaching AI to work like the best analyst, lawyer, or engineer at a particular organization and to capture institutional alpha. That requires deep specialization on proprietary workflows, rather than publicly available data.