We Pointed Two AIs at Our Reading Pile and Told Them to Find Us a Business

Everyone who follows AI has the same graveyard: a Medium reading list 400 articles deep, a dozen newsletters landing daily, and a nagging sense that somewhere in that pile is a product worth building. The bookmarks never turn into anything. Notes rot. The signal drowns.

So we ran an experiment — build a knowledge base that reads its own clippings, keeps only what survives scrutiny, links ideas across everything you've ever saved, and points at one question every time: what real problem is this, and how would software solve it for recurring money? Not a smarter search box. A decision tool that compounds.

This is the build-in-public log: the method, the pattern underneath, the commands — and what the first run actually produced. If you run Claude Code, you can copy it.

The method: two models, one job each

The core trick is refusing to let one AI do everything. We split the work across two models with different temperaments, and the split is the point.

Model one — Claude Opus 4.8 — took the spec to a working foundation. Working from a written spec that settled every real decision upfront — how do you ingest Medium when there's no API and most of it is paywalled? where does the "improve itself" loop actually come from? how do you measure whether the thing is getting more useful instead of just bigger? — it built the storage-and-review engine: capture, distill, review, the schema contract, the test suite. Opus is literal and predictable — exactly what you want when the job is implementing a contract without drifting from it.

Model two — Claude Fable 5 — evaluated that foundation and built the applied layer on top. Handed one question — "what would turn this from a filing cabinet into a decision tool?" — it audited the v1 build across eight value dimensions, wrote its own findings report, then implemented the answer: the scoring, the adversarial proof pass, the prediction ledger, the opportunity-to-spec bridge. Fable 5 is built for long, ambiguous, autonomous runs, so you give it the goal and let it scope the work.

Between them sit the load-bearing steps everyone skips: a human reads the spec before any building starts, and an independent QA agent reviews each build before it's called done. The spec is the approval gate going in; the fresh-context reviewer is the gate coming out — it caught real flaws in both phases (a path-traversal edge, a prompt-injection hole, math that diverged between two surfaces), all fixed before anything shipped. The Fable 5 handoff prompt was written with our own /prompt-architect tool, which knows the quirks of each model — most importantly never to ask Fable 5 to narrate its reasoning aloud (it trips a safety classifier and silently downgrades your model).

Why bother splitting? Because "implement the contract" and "question the contract" reward opposite instincts. One model faithfully building the spec, then a second independently asking "what's this system still missing?" produced a materially better result than either would alone — the entire applied layer exists because the second model was told to critique before extending.

The pattern underneath: Karpathy's LLM Wiki

The architecture isn't ours. It adapts a pattern Andrej Karpathy published in April 2026 — his "LLM Wiki." It's three layers and two operations, and its elegance is that it needs no database and no vector store — just markdown files and an LLM.

• Raw sources are immutable ground truth. Every captured article or newsletter story is a dated file the AI reads but never edits.
• The wiki is a directory of AI-written notes derived from those sources — summaries, concept pages, comparisons — cross-linked to each other, with an index.md catalog listing every note in one line.
• The schema is a human-owned document telling the AI how the wiki is structured. As Karpathy puts it, your editorial job becomes writing and refining the schema, not writing the articles.

The two operations: compile (read new sources, write or update notes, refresh the index) and lint (periodically hunt for contradictions, stale claims, orphaned notes, and important ideas that lack a page). Retrieval is just "read the compact index, then open the notes you need" — the way you'd use a well-kept wiki, not a search engine.

Onto that skeleton we bolted three things the goal demanded:

1. A monetization lens. Every note is one of five types — problem-signal, monetization-pattern, build-technique, distribution-tactic, or the payoff type, opportunity, which fuses a problem + a way to charge + a way to build into "here's the thing, here's how it earns."
2. Provenance or it doesn't exist. Every note links back to the raw file it came from. No source, no note. You can act on it precisely because it can't quietly make things up.
3. Stage, never publish. The engine writes and stages files but never commits or ships. A human approves every deletion. Knowledge is never auto-deleted.

The loop that actually shipped

The applied layer is what turns notes into decisions. The full loop is capture → distill → score → prove → promote-to-spec, seven commands:

python -m knowledge_base ingest-newsletters --days 30   # capture the newsletter firehose
python -m knowledge_base ingest-medium                  # capture your Medium reading list
python -m knowledge_base distill                        # raw → linked, sourced notes
python -m knowledge_base review                          # health-check + dedupe the wiki
python -m knowledge_base score                           # rank the opportunities by evidence
python -m knowledge_base prove <note-id>                 # adversarially try to refute one
python -m knowledge_base promote-to-spec <note-id>       # → a build-ready spec

The two steps we're proudest of: prove takes a single note and tries to refute its claims using only its cited sources, then stamps a verdict — a note that can't survive its own skeptic doesn't get to influence the ranking. And benchmarks.jsonl, an append-only ledger that records what the system predicted about each opportunity and when, so months later you can compare the prediction to reality. Every step stages its output and stops; nothing publishes itself.

What the first run actually produced (no theater)

Being straight about results, because the whole point is a system that doesn't inflate its own progress.

Capture pulled 91 newsletter stories in a single 30-day pass. Distill — three real claude -p passes reading those raw files — compiled the first wiki: 14 seed notes across all five types, including the payoff type. The first opportunity note is a genuinely reasonable business, and it's fully sourced: an always-current, role-by-role "which AI model should I use?" advisor, sold as a subscription. The system inferred it because four independent newsletters in one capture window were all answering that same question — Fable 5 "which model?" cheat-sheets, a writer publicly swapping Opus for GLM-5.2 inside Claude Code, weekly digests listing six-plus model releases apiece, and Reddit threads debating it. A newsletter's answer decays in days; a living service doesn't. That's a real, grounded lead — pulled out of the noise automatically.

And the honest wrinkles, because they're the interesting part:

• The very first live distill failed loud — and rightly. A prompt-assembly bug (the template extractor matched a documentation mention of its own markers) sent the model a one-character prompt. The engine's guard refused to write anything from the garbage response, the bug got fixed with a regression test, and the re-run went through clean. The safety rail fired exactly when it should — that's the design working.
• The run left a pair of near-duplicate seed notes on the same source. The review lint flagged them as a merge candidate on its first pass, and they've since been merged — the self-correction loop doing its job.
• Then we made the top idea survive its own skeptic — and it only half did. The prove pass verified 2 of its 7 load-bearing claims with verbatim quotes and refuted 5 as inference, returning a verdict of partial. Most importantly, it found that the "people pay for this" premise has zero direct evidence in the sources, and named the exact missing piece: confirm whether the cheat-sheet it's modeled on actually sits behind a paywall. Nothing is evergreen yet. These notes are leads, not conclusions — and the system now says so with receipts.

That's the state: the machine exists, the reading pile is loaded, the first real distill turned 91 clippings into a shortlist with one concrete opportunity on it — and the system's own skeptic has already told us precisely what to check before building it.

What to take from this

• Split building from questioning. One model to implement the spec faithfully, a second to critique the result and build what's missing, a human approving at each gate. The forced hand-offs are where quality comes from — not a bigger model.
• Files beat databases for a personal knowledge base. Karpathy's markdown-and-an-LLM pattern makes your second brain greppable, diff-able, version-controlled, and portable — and it costs nothing extra to run on a subscription you already have.
• Make it prove itself. An adversarial refutation pass, a provenance guard that fails loud, and a prediction ledger are the difference between a knowledge base you trust and a pile of confident-sounding notes. If a claim can't survive its own skeptic, it shouldn't shape a decision — and if you're building software to earn, that discipline is the whole game.

The graveyard of bookmarks is optional. The tools to turn it into a shortlist of things worth building are sitting on your laptop right now — and ours just handed us the first one.

---

Build notes: the architecture adapts Andrej Karpathy's LLM Wiki pattern (April 2026). The design spec lives in the repo at specs/done/knowledge-base/01-self-improving-kb.md; the engine's contract is knowledge-base/SCHEMA.md; the eight-dimension evaluation that produced the applied layer is reports/knowledge-base/eval-001-2026-07-03-v1-eight-dimensions.md. The Fable 5 handoff prompt was built with our /prompt-architect tool. Status as of 2026-07-03: engine built by Opus 4.8, applied layer by Fable 5, 91 sources captured, 14 seed notes distilled including one opportunity — which the prove pass has already graded partial pending real willingness-to-pay evidence.