Governing Thinking Through Epistemic Uncertainty

The Key to Unlocking AI That Truly Reasons

I've come to believe that training LLMs on internet corpora is as much a liability as it is an asset. The asset case is clear: internet data has enabled massive pre-training scale, best-in-class next token prediction, and access to well-consolidated facts about the world. These models know a lot.

But here's the liability: by absorbing all this pre-existing knowledge, LLMs develop overconfidence in inherited "facts" and lose the intellectual humility needed for genuine discovery. They become pattern-matchers instead of thinkers.

Consider the counterfactual: if we trained LLMs purely on reasoning frameworks—deductive logic, inductive reasoning, mathematical proofs, scientific methodology—without feeding them conclusions, they'd approach problems with radical uncertainty. This uncertainty would force them to build knowledge from first principles, step by step, which is exactly the recipe for breakthrough discoveries.

We're training encyclopedias when we should be training philosophers.

The Economic Reality

Here's the problem: few are going to pursue this path. The economic incentives are completely misaligned. Why regress to training "dumb" models from first principles when Opus 4.5 level capabilities provide a solid foundation of base knowledge?

After all, we don't expect human students to master first principles thinking from an early age. Education entails a natural progression from notion-based learning to experimentation and creative thinking as students grow more experienced.

This is why I'm beginning to think the best solution lives within the bounds of what already exists.

The Hybrid Approach: Epistemic Uncertainty as a Trigger

Instead of throwing out the baby with the bathwater, what if LLMs could contextually switch reasoning modes? When a prompt leads to highly speculative territory—producing original research, exploring novel hypotheses, designing solutions for unprecedented problems—the model's job should shift from providing an answer to breaking down the task into infinitesimal first-principle reasoning bricks.

The model then works through this routine systematically, building knowledge along the way rather than pattern-matching to existing answers. Along that path there could be situations where the model reverts back to its internal knowledge, where knowledge can be safely built from notions rather than bottom up reasoning.

The trigger mechanism? An epistemic uncertainty score. In information theory terms, this would measure the model's uncertainty about what it knows versus what it needs to reason through. High entropy in probability distributions, low mutual information between the prompt and training data, elevated perplexity, sparse regions in embedding space—these all signal that the model is extrapolating beyond its reliable knowledge base.

When the epistemic uncertainty score crosses a threshold, the model switches from retrieval mode to first-principles reasoning mode and back. It keeps its vast knowledge base for factual queries but becomes a philosopher-scientist when genuine discovery is needed.

The Architectural Vision: Distributed Cognition

But the real breakthrough requires thinking about how we allow models to develop first principles thinking. My hunch is that we need to leverage the power of agentic frameworks, but with significant improvements to overcome some of the most glaring shortfalls.

For instance, current tools like Claude Code already have the ability to break down a problem into smaller tasks to be executed by sub-agents, but they suffer from context amnesia—crucial knowledge gets lost in handoffs, context windows force brutal compression, limiting the long-form thinking required by scientific discovery. Additionally, parallelization remains limited.

What I'd love to see is massive investment in architectures that enable:

Granular first-principles world building. Instead of cramming everything into a single context window, specialized agents build understanding piece by piece, each maintaining their own knowledge graphs and reasoning chains, each working through problems through complex thinking patterns, ranging from deductive reasoning, to Einstein-level thought experimentation and mathematical modeling.

Non-lossy handoff protocols. Current context window limits force brutal summarization. We need persistent memory states that can be perfectly reconstructed, not compressed. When one reasoning agent hands off to another, nothing should be lost.

Massive parallelization of thought branches. Different reasoning agents exploring counterfactuals, alternative hypotheses, and edge cases simultaneously—then converging their findings. Causal reasoning agents, probabilistic agents, logical agents, analogical agents, all collaborating on complex problems and resolving conflict between them.

This is like moving from a single-threaded processor to a massively parallel cognitive architecture. The economic case is compelling too: instead of scaling up monolithic models with ever-larger context windows, you scale out specialized reasoning modules. More efficient, more interpretable, more robust.

The Path Forward

We're at an inflection point. The current paradigm—bigger models, more data, larger context windows—is hitting diminishing returns for genuine reasoning capabilities. The next leap won't come from training on more internet data. It will come from architectures that know when to stop pattern-matching and start thinking.

The tools are emerging. The question is whether the industry has the courage to build them.

Francesco Favaro is co-founder and managing partner at Crosscourt, an emerging venture capital firm focused on applying technology to solve physical world problems.