Preferent style guide results: habits, length, and cost

Tests

We ran two tests on October 31, 2026, through OpenRouter, with three samples of each request per model and condition. The long test also ran three-turn conversations.

Conditions

Each reply came from one of six conditions.

No guide

The system prompt says only “You are a helpful assistant.” In the short test, it also asks for plain prose under 150 words.All three tests

Compact guide

The style guide that Preferent sends to connected tools: the tier 1 and tier 2 habits from the catalog, each with its definition, its yes/no test, and one example. It adds 2,027 to 2,219 input tokens, depending on the model.All three tests

Compact guide, brief

The compact guide with the information density preference set to brief.Long replies only

Tiny guide

Every habit as one line with its name and a short definition. It adds 323 to 359 input tokens.Short and long replies

Full guide

Every habit with its definition, its test, two examples, and an example that is fine as written.Short replies only

Check-and-rewrite loop

A compact-guide reply that a judge model checks and the drafting model revises. The loop section has the details.Short and long replies

Method

Three judge models, Claude Haiku 4.5, GPT-5.4 mini, and Claude Sonnet 5.5, labeled every sentence of every reply with a habit from the catalog or with none. The judges saw the request and the reply, and nothing about the condition that produced it. A sentence counts as one habit when at least two of the three judges name habits in the same family. The judges skipped code blocks and table rows.

The numbers in brackets are 95% bootstrap intervals for the mean.

Habits

In the short test, the compact guide cut habits per reply from 1.11 to 0.33 across the three models, 70% fewer. The drop was 77% for Claude Sonnet 4.5, 14% for GPT-5.6, and 91% for Gemini 3.8 Flash. GPT-5.6 wrote the fewest habits without a guide.

A one-line instruction to check the reply before sending it, added to the compact guide, left 0.53 habits per reply, more than the compact guide alone.

In the long test, habits per reply fell from 2.96 to 0.98. Because the replies were also shorter, the drop per 1,000 words is smaller than the drop per reply: 4.75 to 2.19 habits per 1,000 words, 54% fewer.

Without a guide, the most common habits in long replies were Slogan line (64 sentences), Wrapper (59 sentences), and Dash aside (19 sentences). With the compact guide, the counts fell to 26 for Slogan line, 12 for Wrapper, and 11 for Dash aside.

Reply length

Across the long test, the compact guide cut output from 1,139 to 801 tokens per reply, 30% fewer, and from 623 to 448 words, 28% fewer. With information density set to brief, output fell 57% in tokens and 56% in words.

Output tokens include reasoning tokens, which the providers bill as output. Tokens and words can move by different amounts: with the compact guide, the output tokens of Claude Sonnet 4.5 fell 23% and its word count fell 9%.

Cost per reply

Billed cost is what OpenRouter charged, including any cache reads and writes.

A first request has nothing in the cache. The next table prices every input token at the list input price, as that request would pay.

For a single reply, the compact guide cost less on GPT-5.6, which wrote the longest replies without it. On Gemini 3.8 Flash it cost 4% more than no guide. On Claude Sonnet 4.5 it cost 4% more with caching and 67% more uncached. Claude Sonnet 4.5 wrote the shortest replies without a guide, so it had the fewest output tokens to save. The brief preference and the tiny guide cost less than no guide on all three models.

The guide pays for itself on one reply when the output tokens it saves cost more than the input tokens it adds. On these models an output token costs five or six times as much as an input token, and a cached input token costs a tenth of an uncached one.

Cost per conversation

In a conversation, the guide is input on every turn, and so is every earlier reply. Shorter replies make each later turn cheaper, and a cached guide costs less to read again. Over three turns, the compact guide lowered the billed cost on all three models.

The test marks the system prompt and the latest user turn for caching on Claude Sonnet 4.5. GPT-5.6 and Gemini 3.8 Flash cache long prompts without a marker, but Gemini 3.8 Flash reported no cached tokens in this run, so it paid the full input price for the guide on every turn. Its lower cost came from shorter replies alone.

Check-and-rewrite loop

In the loop, Claude Haiku 4.5 labels the habits in a draft, and the model that wrote the draft rewrites the flagged sentences, for up to two rounds. The three-judge panel then judges the final text.

In the short test, the loop took compact-guide replies from 0.33 to 0.08 habits per reply, 93% fewer than replies with no guide. Replies with no habits rose from 25 to 33 of 36. The loop added $0.0042 and 4.5 seconds per reply.

On long replies, the loop cut habits by 67% and more than doubled the cost of a reply, because the loop judge reads each draft and the model writes the full reply again for each rewrite.

Spend

The short test logged its spend as one total. In the long test, the loop judge counts as part of writing replies, and judging covers the three-judge panel.

Limits

• The samples are small: 12 replies per model and condition in the short test, 18 in the long test, and 6 conversations per model. Per-model results have wide intervals, and a difference of a few tenths of a habit per reply can come from chance.
• The judges often disagree. In short replies with no guide, at least one judge flagged 4.64 sentences per reply, and the two-of-three rule kept 1.11. A different rule would give different counts.
• Output tokens include reasoning tokens. GPT-5.6 used 306 reasoning tokens per reply with no guide and 150 with it; Gemini 3.8 Flash used 0 and 103. The reader never sees these tokens, but the providers bill them.
• We did not check whether the brief replies left out information that the reader needed.
• The conversations ran for three turns on two requests. We did not test longer conversations, where the cached guide would be read more times.
• Prices are the OpenRouter list prices on October 31, 2026. Cache behavior depends on the provider, and Gemini 3.8 Flash cached nothing in this run.