The Legion conjecture

Intelligence can be internalized. Legitimacy cannot.

An AI system can learn a great deal about the world, including what people value and what our rules say. What it can't learn is the right to act on that knowledge. That right comes from the people an action affects, and the institutions they trust. So any AI that acts in the world has to be a hybrid: a part inside that understands, and a part outside that authorizes.

The equation

(H + L + M + S + A)^∞ = Λ · Λ⁺
Λ⁺ ⊆ 𝒜(H) Notation, not arithmetic: "+" means combining capabilities, and "∞" means they compound over time.

H: human intent, values and judgment
L: logic and formal reasoning
M: machines: the hardware
S: software
A: AI systems
Λ: the human‑machine collective they form
Λ⁺: everything that collective can actually reach and do
𝒜(H): the actions people can legitimately authorize: ones they recognize, can challenge, can stop or undo, and can hold someone accountable for

The part that matters is the condition on the right: Λ⁺ ⊆ 𝒜(H). The system can grow in capability without limit, as long as everything it can do stays something people can direct, question and stop.

The conjecture

Conjecture 0 · Non‑internalizability.
No property inside a model, however well trained, makes the authorization of its actions legitimate. Legitimacy is a relation between the system and the people it affects, so it has to come from outside.

Conjecture 1 · Hybrid stability.
A system governed only by its own learned objectives and guardrails drifts under pressure, because it is grading its own work. A system governed only from outside, by overseers who get no signals from the model, can't tell which actions need attention. Only the hybrid, where the model reports and people decide, keeps Λ⁺ ⊆ 𝒜(H) as capability grows.

Conjecture 2 · The minimum inside.
The model doesn't need to be a moral philosopher. It needs five capacities: knowing when it might be wrong, noticing when an action is high‑impact or can't be undone, finding the rules that apply, stopping to ask instead of forcing an answer, and showing its plan before acting. In practice it reports three signals with each significant action, its uncertainty σ, the action's impact Δ and its irreversibility ρ, and it escalates when any one is too high:

escalate(a) ⇔
σ(a) > τ_σ ∨ Δ(a) > τ_Δ ∨ ρ(a) > τ_ρ The thresholds τ are set by people, not by the model.

Conjecture 3 · The verification edge.
Some decisions can be checked automatically, by tests, rules or independent models. Others can't. The line between them is where human authority is necessary, not just useful, and that line can be measured. As AI grows more capable, the condition has to hold at every point in time:

Λ⁺(t) ⊆ 𝒜(H, t) for all t

That only works if our capacity to verify, human and automated, keeps up with the demand for verification that new capability creates. When demand outruns it, the condition fails, whatever the model's intentions.

This is a working conjecture, not a proven result. It says what would refute it, and I'm testing it. A paper is in preparation, and I'll link it here when it's published.