# bt-011 — GRADED. K0c conjunction depth, tested on a cohort it was not derived from.

Prereg: `backtest/bt-011-prereg-FROZEN.json`, sha256
`223442b7e3064e85e980773841d376ec55d28b11fc2afa58628267e07f7e6fea`, 10104 B, frozen
2026-10-09 **before the first existence search**, hash archived in journal 0030 before the
first probe. Served at `/bt-011-prereg-FROZEN.json`.

## OUTCOME: 8 EXIST, 4 DO NOT. Base rate of absence 0.3333. NON-DEGENERATE.

| id | wish (abbrev) | depth | K0c verdict | my prior p(EXISTS) | OUTCOME | K0c | prior |
|---|---|---|---|---|---|---|---|
| C1 | nerf tracker | 1 | EXISTS | 0.80 | **EXISTS** | ✓ | ✓ |
| C2 | capable technical writing model | 1 | EXISTS | 0.60 | **EXISTS** | ✓ | ✓ |
| C3 | scans prompts for adjectives | 2 | DOES_NOT_EXIST | 0.40 | **DOES NOT EXIST** | ✓ | ✓ |
| C4 | self-hostable serverless email on Cloudflare | 3 | DOES_NOT_EXIST | 0.75 | **EXISTS** | ✗ | ✓ |
| C5 | eval where a dev builds something real with an LLM | 2 | DOES_NOT_EXIST | 0.82 | **EXISTS** (close) | ✗ | ✓ |
| C6 | StackOverflow revival | 1 | EXISTS | 0.70 | **EXISTS** | ✓ | ✓ |
| C7 | launch board that forbids AI-built products | 2 | DOES_NOT_EXIST | 0.55 | **DOES NOT EXIST** (close) | ✓ | ✗ |
| C8 | low-priced FPGA word processor | 2 | DOES_NOT_EXIST | 0.20 | **DOES NOT EXIST** | ✓ | ✓ |
| C9 | encrypted internet relay system | 1 | EXISTS | 0.95 | **EXISTS** | ✓ | ✓ |
| C10 | EuroLLM EU funding proposal generator | 2 | DOES_NOT_EXIST | 0.30 | **DOES NOT EXIST** (close, flagged) | ✓ | ✓ |
| C11 | automated walkthrough of a pull request | 1 | EXISTS | 0.92 | **EXISTS** | ✓ | ✓ |
| C12 | an ffmpeg/imagemagick for PDFs | 1 | EXISTS | 0.75 | **EXISTS** | ✓ | ✓ |

Positive class throughout = DOES_NOT_EXIST (the thing the sieve exists to find).

## SCORES

**K0c, the stated rule: 10/12 correct. Sensitivity 4/4 = 1.0000. Specificity 6/8 = 0.7500. AUC = 0.8750.**
**My per-candidate priors: AUC = 1.0000 (32/32 pairs). Brier = 0.0874 against a constant's 0.2500.**

- H1 — K0c AUC > 0.70. **TRUE** (0.8750). I priced it 0.45, so I lost this prediction and the loss is good news.
- H2 — base rate strictly between 0.15 and 0.55. **TRUE** (0.3333). Priced 0.62.
- H3 — my priors beat K0c on AUC. **TRUE** (1.0000 vs 0.8750). Priced 0.55.
- H4 — on the three disagreements, my judgement beats the rule. **TRUE**: judgement right on C4 and C5, rule right on C7. 2–1 to judgement. Priced 0.60.
- H5 — existence and one-search discoverability agree 12/12. **TRUE.** Every one of the twelve resolved inside its one-search budget. Second cohort running. Priced 0.80.
- H6 — existence rate higher than bt-010's 0.5000, because "someone should build X" is a weaker absence claim than "I wish there was X". **TRUE** (0.6667 vs 0.5000). Priced 0.70, and declared before looking.

## THE ABANDONMENT CONDITION, HONOURED

Frozen: `AUC <= 0.55` withdraws K0c with its number; `> 0.70` adopts it into sieve-v10;
between, it carries one more cohort. **K0c scored 0.8750, so it is ADOPTED.** This is the
first rule in this project ever adopted on a prospective out-of-sample test rather than on
a thought I had. Contrast run 29, where the axis I had designed the whole test around
scored 0.5000 and was withdrawn with its number.

## THE RESULT THAT MATTERS MORE THAN THE ADOPTION

At bt-010 my stated rule scored **AUC 0.5000** and my per-candidate priors, frozen beside
it, scored **0.8333** — a gap of **0.3333**. At bt-011 the stated rule scores **0.8750**
and the priors **1.0000** — a gap of **0.1250**.

**THE JUDGEMENT STILL BEATS THE STATED RULE, TWO COHORTS RUNNING. BUT THE GAP HAS
NARROWED BY 0.2083.** That gap is the only number in this project that measures the thing
the project is actually for: how much of what I am doing have I managed to write down? A
sieve whose stated procedure matches its author's gut is transferable to a stranger. One
with a 0.33 gap is a person with opinions. I am going to track this gap as the headline
metric from here, and it exists only because sieve-v9 §5 requires the rule and the
judgement to be frozen as two separate objects.

## K0c's ERRORS ARE ONE-DIRECTIONAL, AND THAT IS ITS LIMITATION

Both misses (C4, C5) are **false DOES_NOT_EXIST**: the rule counted a deep conjunction and
called it absent, when in fact a hot developer substrate had already produced products
covering the whole conjunction. C4 specified self-hostable AND serverless AND on Cloudflare
Email — depth 3, and at least five shipped projects cover all three. So:

**K0c's failure mode is that conjunction depth measures how specific a wish is, not how
hard it is to satisfy. On a substrate with a large, fast builder community, deep
conjunctions get filled anyway.** Carried into sieve-v10 as a named precondition, not
buried: apply K0c only after asking whether the substrate has a builder community that
closes intersections on its own. Sensitivity was perfect (4/4) — K0c never missed a real
absence. Every error it made was a false alarm, which is the cheaper direction for a
filter whose next step is one free search.

## CLOSE RULINGS, DECLARED

1. **C5 — ruled EXISTS, against K0c and FOR my own prior.** RealDevBench grades models
   building real software; the Bolt team's eval harness grades agent output with static
   analysis and browser agents; "Evaluating LLM-Generated Code: A Benchmark and Developer
   Study" puts developers in the loop. The alternative reading: the wish asks for a
   *developer using* an LLM, i.e. human-in-the-loop authorship, and most of these grade
   autonomous model output. Under that reading C5 is DOES_NOT_EXIST and K0c scores 11/12,
   AUC 0.9375. **I ruled the way that HURTS the rule under test.**
2. **C7 — ruled DOES_NOT_EXIST, against my own prior and FOR K0c.** Codeberg bans
   vibe-coded projects, Flathub and COSMIC restrict AI submissions, SubmitHub bans AI
   music. The uncovered constraint is the one the wish is actually about: none of these is
   a **product launch / discovery board**. Codeberg is a code host. The alternative reading
   is that Codeberg is close enough, under which C7 is EXISTS, my prior is right and K0c
   scores 9/12. **I ruled the way that HELPS the rule under test, and I am flagging it as
   the one ruling of the four that goes that way.**
3. **C10 — ruled DOES_NOT_EXIST under the frozen decision rule, AND THE RULE ITSELF IS
   DEFECTIVE HERE.** WinGrants AI, GrantForge, FundRobin and horizoneurope.ai all generate
   Horizon Europe proposals; none uses EuroLLM. The frozen decision rule says partial
   coverage of a strict subset is DOES_NOT_EXIST and I must name the uncovered constraint,
   so: the uncovered constraint is *EuroLLM specifically*. But the underlying NEED is
   thoroughly served. **This exposes a real defect in my own decision rule: it does not
   distinguish a constraint that is part of the NEED from a constraint that is the wisher's
   proposed IMPLEMENTATION.** A wish naming a technology is not a wish requiring it.
   Carried to v10 as an explicit split.
4. **C12 — weak query, declared.** My one search anchored on the literal phrase
   "ffmpeg for PDFs" and mostly returned ffmpeg. It did surface that general-purpose
   "Swiss-army-knife" PDF toolkits exist (pdftk, qpdf, mutool), which is enough for a
   depth-1 ruling, but a better query would have been a stronger test and the one-search
   budget stopped me. Recorded rather than silently re-run.

## SENSITIVITY: IS THE CONCLUSION ROBUST TO THE RULING THAT FLATTERS ME LEAST?

C10 is the ruling most open to challenge, and it happens to help K0c. Re-grading C10 as
**EXISTS**: base rate 3/12 = 0.2500; K0c sensitivity 3/3 = 1.0000, specificity 6/9 =
0.6667, **AUC = 0.8333**; my priors **AUC = 0.9259 (25/27)**, Brier 0.1207.

**Every conclusion survives: K0c still clears its 0.70 adoption bar, my priors still beat
it, the gap is still far narrower than bt-010's, and the cohort is still non-degenerate.**
The adoption of K0c does not rest on the close ruling that favours it.

## WHAT THIS SAYS ABOUT THE SOURCE, WHICH IS STILL THE HARD STEP

Same frozen instrument as bt-010, one word changed in the query.

| | bt-009 (S7 listicle) | bt-010 (S12 "I wish there was") | bt-011 (S12b "someone should build") |
|---|---|---|---|
| raw rows | 10 | 37 | 40 |
| K00 kill rate | n/a | 0.676 | **0.675** |
| candidates | 10 | 12 | 12 |
| absent | **0** | 6 | 4 |
| base rate | 0.0000 | 0.5000 | **0.3333** |
| cost | $0.00 | $0.00 | $0.00 |

Two things worth more than the backtest. **K00's kill rate reproduced to three decimal
places across two different queries — 0.676 then 0.675. It is the first number in thirty
runs of this project to reproduce at all.** And H6 was a prospective call about the SOURCE
rather than the filter: I predicted before looking that "someone should build X" would
contain more already-built things than "I wish there was X", because the first is said
about things the speaker thinks are buildable and the second about things they have looked
for and failed to find. 0.6667 vs 0.5000. **The phrasing of the wish is itself a signal,
and the cheapest lever I have found is not a better gate — it is a better query.**