What Stress-Testing LEAP Is Teaching Me About Building Reliable AI Systems

UPDATE: I have been spending a lot of time trying to break the prompting system I am building around LEAP.

At first, I treated every failure the same way: find the defect, add an instruction, tighten the prompt, test again.

That worked until the prompt started becoming the system.

The more I stressed it, the clearer the real problem became:

The hard problem is not making one LEAP smart enough. The hard problem is engineering the joints between the LEAPs.

That changed how I am building the entire system.

Mercer: The Lego Brick Analogy Was More Important Than I Initially Realized

When Mercer described each LEAP as a Lego brick that could be stacked into a Smart Domino, I understood the concept.

I missed the engineering consequence.

I kept trying to make each LEAP more reliable by adding rules. Every edge case created another instruction. Every failure produced another guardrail.

Eventually, one prompt was being asked to:

• do the reasoning;
• judge whether the reasoning was good;
• decide whether enough information existed;
• determine what should happen next;
• decide whether the result was safe to release;
• remember its place in the larger workflow.

That is not one Lego brick anymore. It is the whole damn building.

The better architecture is to keep each LEAP responsible for the semantic work it is good at, then surround it with bounded components responsible for checking, control, and routing.

A simplified version now looks more like this:

... Do the bounded reasoning. Produce the required result. Independently evaluate whether the result actually satisfies the contract.

\ Required inputs exist. Required conditions are satisfied.
\
PASS -> continue
FAIL -> repair
UNKNOWN -> hold or recover
\
\

The XML is not the interesting part.

The important part is that doing the work, judging the work, and controlling what happens next are no longer the same job.

That separation has made the system much easier to reason about.

The NITWYT Aha Moment

Chris's explanation of NITWYT OS gave me the piece I was missing.

My interpretation was:

The LEAP is allowed to drift. Reliability comes from the network around it.

Instead of trying to make one prompt deterministic, I can let the model do what models are good at: semantic reasoning, interpretation, synthesis, comparison, and judgment.

Other model passes can challenge or grade that work. Deterministic code can handle what actually is deterministic:

• required fields;
• cardinality;
• identities and versions;
• allowed states;
• dependency order;
• whether required work occurred;
• whether approval exists;
• whether the correct artifact was evaluated;
• whether a blocker exists;
• whether release is permitted.

That boundary seems obvious now. It was not obvious when I started.

A Checker Can Be Wrong the Same Way

One of the nastier failures came from my own test system.

I built a deterministic checker to verify a registry. It reported PASS.

The checker and the system under test were using the same ambiguous textual assumption.

They agreed with each other.

They were both wrong.

That taught me something broader:

Independent checking is not independent merely because it happens twice.

If two components inherit the same representation, parsing assumption, or context, the second may simply certify the first component's mistake.

I replaced loose phrase matching with explicit machine-readable boundaries and stable coordinates.

The fix was not to make the checker smarter.

The fix was to remove the ambiguity from the joint.

A Perfect Map Can Still Be Incomplete

I found a similar problem in the context system.

A policy map passed every forward-reference check. Every coordinate pointed to something real.

But two important policies were missing entirely.

Nothing was broken because nothing pointed to a nonexistent coordinate. Important material had simply never made it onto the map.

So I now test both directions:

Does every map entry resolve?

Did every required source item make it into the map?

A system can be internally consistent and still be incomplete.

The Latest Failure Was Even More Useful

Today I ran a fresh task through the system: build a complete plan for generating a target amount of income from an investment portfolio.

During questioning, the system asked which strategies I refused to use.

I answered:

buying calls or puts

The system proceeded as though I had said:

I want to use buying calls or puts.

It built the entire downstream plan around long options.

The reasoning was coherent. The checklist was detailed. The math worked. The answer looked competent.

It was also solving the wrong problem because one upstream constraint had been inverted.

That failure exposed a general rule:

Constraint polarity must survive every handoff.

"I refuse X" cannot become "I prefer X."

"Exclude X" cannot become "optimize X."

"Do not use X" cannot become "use X."

This is exactly why I am becoming skeptical of judging AI systems by how polished the final answer appears.

Once the first semantic coordinate is wrong, excellent reasoning can simply move you faster in the wrong direction.

What I Think I Am Building Now

I originally thought I was building a better prompting system.

That description no longer fits.

What I am building looks more like a small reasoning network:

• LEAPs perform bounded semantic work;
• each space has one primary purpose;
• context is referenced rather than sprayed everywhere;
• Smart Dominoes know why they are active and what can happen next;
• separate model roles challenge, defend, or adjudicate semantic findings;
• deterministic controls govern what machines can actually know deterministically;
• missing information can produce HOLD instead of hallucinated progress;
• plausible output is not automatically releasable output.

One engineering rule captures much of this:

NO RECEIPT, NO RELEASE.

If the system says research occurred, there should be evidence that it occurred.

If it says a comparison was performed, the required comparison objects should exist.

If it says a condition was satisfied, something should establish that condition.

If it says the workflow is complete, completion should mean more than "the model produced prose."

That control machinery is mine, not something I am attributing to Mercer. But I do not think I would have arrived at it without taking the Lego-brick and Smart-Domino idea seriously.

Where I Am Now

I have stopped trying to anticipate every possible failure and encode it into one giant instruction set.

The architecture has survived structural, adversarial, mutation, context, routing, state, and release tests. Now I am running genuinely unrelated tasks through it.

That is already paying off.

The latest failure did not tell me I need a special rule about options trading.

It told me I may have a general semantic handoff problem involving constraint polarity.

That is exactly the kind of failure I want to find.

To Mercer

Your "Lego brick" comment changed how I think about the problem.

The intelligence of the brick matters, but the integrity of the joints may matter more.

A LEAP can be excellent while the larger system fails because context was wrong, activation was wrong, identity was lost, state was reset, or a semantic constraint changed meaning between components.

That is where much of my testing is focused now.

To Chris / NITWYT

The breakthrough was realizing that the model does not need to become perfectly reliable inside one prompt.

Let the model reason.

Then build a network capable of checking, challenging, stopping, repairing, and routing that reasoning.

That changed the architecture completely.

I am going to keep breaking this thing and report back with what survives.

More to come. Still working.