Final brochure summary · Oct 2026

Public document revision tests

317 statute revisions: 38.2% savings; 79 eCFR revisions: 56.9%; 45 drug-label revisions: 57.0%. Internal quality scores are 0.978, 0.988, and 0.972 respectively, not accuracy percentages. The original detailed methodology remains below.

Method

How we measure savings.

This page describes the experiments that evaluated Linklbee’s selective regeneration feature. Two public revision datasets, real paid API calls, and criteria fixed before the first call.

The Linklbee engine runs on your own servers. AI requests never pass through Linklbee’s servers.

Results

Selective regeneration: cost and quality compared

We called every item twice with the same model: through Linklbee and as a full rewrite. Quality is compared with the version actually published next. Measured on public document data. Your own results are confirmed by a scan of your logs and a pilot.

Dataset (public)CallsOutput tokensCost Quality · LinklbeeQuality · full rewriteQuality · last version as is
Korea MFDS drug-label revisions45−75.1%−57.0%0.9720.96280.9593
Korean statute amendments317−57.1%−38.2%0.97770.98110.9385

The −38.2% we publish includes 0% from reusing a previous answer without a call. Every edit call was actually made, and the API cost of reprocessing after a failed edit is included.

“Four in ten” (the share of answers that change when a call is skipped) was measured directly with 78 paid calls (2026-10-18).

In the statute revision test, 🧪 309 of 317 items scored higher on quality than using the previous version unchanged. The statute row’s output and cost include retrying the 18 items whose edits failed to apply as full rewrites, as the product does (recomputed from the same call records). The MFDS row went through the product server as is, so the cost of its one real fallback is already inside. Cost savings are smaller than output savings because the attached draft adds input tokens; that increase is already subtracted. Statute and MFDS rows: reasoning off, for both control and treatment. Model claude-sonnet-5-5 for both, 2026-10-17 to 19.

ValueOutput-token savings and cost savings
Sample45 · 317 items
Compared withThe same items rewritten in full by the same model
Measured byCost is computed from the AI provider’s usage figures; quality is the token F1 against the real next version. The criteria were fixed before the experiment.
Run it yourselfThe scanner measures how much of your logs it can reduce

Assumptions and formulas

Assumptions Price: claude-sonnet-5-5 input $2.00 · output $10.00 per 1M tokens (official price, checked 2026-10-08) Token counts: the usage the vendor returned with each response. Only the overlap and length checks count with the o200k tokenizer Prompt caching was not used in this comparison, so the results compare like with like under a no-caching condition.
Output savings = 1 − Σ output(Linklbee) / Σ output(full rewrite) Cost savings = 1 − Σ (in×$2 + out×$10)(Linklbee) / Σ (in×$2 + out×$10)(full rewrite) · Linklbee’s input includes the attached draft · retries for failed edits are included for statutes and MFDS Quality = token F1 (finished answer, real next version) · edits are applied to the draft by the product applier before scoring

Fixed before the first call.

  1. Quality (Linklbee) ≥ quality (full rewrite) − 0.03
  2. Quality (Linklbee) > quality (last version as is): it has to beat not calling at all
  3. Output-token savings ≥ 50%

The criteria were fixed before the calls and never changed.

Where it costs more

Where selective regeneration applies

We regrouped four earlier real-call runs (statutes 317 · administrative rules 277 · statutes outside the eligible range 207 · eCFR 79; 880 calls in all) by how much the previous and next answers overlap. Below 78%, the Linklbee path cost more than a full rewrite.

Cost savings by token overlap (higher is better · negative means it spent more)

Cost savings by token overlap, 880 real calls overlap under 50%: 86 calls, -31.1%; 50–70%: 54 calls, -16.1%; 70–78%: 67 calls, -6.6%; 78–85%: 61 calls, -0.8%; 85–90%: 69 calls, +17.9%; 90–95%: 128 calls, +41.9%; 95–97.5%: 156 calls, +56.8%; 97.5–99%: 158 calls, +63.9%; 99% and up: 101 calls, +65.7% -40% -30% -20% -10% 0 +10% +20% +30% +40% +50% +60% +70% -31.1% <50% 86 -16.1% 50–70% 54 -6.6% 70–78% 67 -0.8% 78–85% 61 +17.9% 85–90% 69 +41.9% 90–95% 128 +56.8% 95–97.5% 156 +63.9% 97.5–99% 158 +65.7% 99%+ 101 on at 78%+ → token overlap between the previous and next answers (calls per bucket below each label)

The 99%+ bucket saves the most, but there the previous version, unchanged, was more accurate (see Features and handling). Cost and quality are evaluated together. Whether an existing response would already suffice is checked separately.

Results depend on the document. Linklbee assesses fit first, applies this approach only to requests where savings are expected, and verifies the actual effect against baseline requests.

📄 Third-party public case (reference): a translation company built the same idea, tested it, and published the results. It held back a full rollout because of "inconsistent performance across different test sets - despite using the same AI prompts."

How baseline requests measure savings

To establish a comparison baseline, some documents are left unoptimized. The default share is 10%, assigned per document, not per request. These are baseline requests. The share shrinks as volume grows.

Baseline requests are handled the existing way, so measuring adds no extra API cost.

Both sides are compared on billed cost per answer token, priced with the same table. Settlement uses the lower bound of the estimated range (the 95% lower bound), not the point estimate.

The pilot report includes the size of the comparison (document and request counts), the baseline-request share, cost per token and savings for both arms (with the 95% lower bound), and how to reconcile it with your AI provider’s bill.