HOW WE SCORE
The rankings show what players preferred in blind tests. Preference, not a fact check. Here is exactly how they're made.
THE BLIND TEST
- Every week (every day until 22 October), the same prompts go to all five models: the same plain wording, no system prompt, default settings. Each model answers each prompt once. No reruns, no picking the best of several tries.
- When an answer is a whole web page, you see the page as a browser shows it, with the code one tap away. Every answer is shown the same way. If the page has buttons, press Try it to use them; even then it can't load anything from the internet or see this site.
- Each model is called on its maker's own servers, through OpenRouter.
- Prompts are written in advance and picked by rotation, never after seeing the answers. They read like real requests from normal people.
- We may reject a whole prompt (if every answer is broken, or one is unsafe), never a single model's answer. Every rejection is listed below.
- An answer that names its own model or company is left out of that day's pairs, so the test stays blind.
- Each player sees two answers, picked at random from the five, in random order, and picks the one they'd actually use. They can also call it a tie or say both are bad.
HOW VOTES BECOME RANKINGS
- Bradley-Terry ratings with 95% ranges, not raw win rates, as LMArena and the French government's compar:IA arena do. 1000 is average; the range shows how sure we are.
- Length and formatting correction. In a quick read, longer and more formatted answers tend to win. The headline ranking corrects for length, lists, headers and bold; each task page also shows the uncorrected scores.
- A tie counts as half a win for each. "Both are bad" doesn't change the ratings; it drives the Rainy icon.
- Each prompt counts equally within its task, so one popular day can't dominate a category.
- Votes cast faster than a realistic reading time (8 seconds) are saved but not counted. One vote per round per device per day, an invisible bot check, and rate limits per network.
Before there are enough votes: editor's picks
Until a task has enough votes, its tile shows an editor's pick, chosen by one rule: the leader among our five in the matching category of LMArena's public rankings. When other models are statistically level with the leader (their 95% ranges overlap), the cheapest of them is the pick: the chat plan that unlocks it costs less, or the model is free in its app. When their plans cost about the same, a plan that can be shared with family comes first, since it costs less per person; after that, the lower price for developers decides. LMArena's scores are read every day from its public leaderboard data. A score from fewer than 100 votes in its category doesn't count; above that, few votes mean a wide range, so the model counts as level with the leader rather than ahead of it. Coding uses two rankings, LMArena's coding ranking and its WebDev ranking (people compare two websites or apps built from the same request), and its pick must be level with the leader in both. The best free option is shown next to the pick, never instead of it. The picks were drafted with help from Claude, one of the five models, using only those public numbers, and checked by a person.
The weather icons
- Sunny, clear winner: the winner's 95% range doesn't overlap the runner-up's.
- Partly cloudy, close race: the ranges overlap.
- Rainy, double-check: in 25% or more of votes, players said both answers were bad.
- Foggy, too few votes: fewer than 200 counted votes across at least 3 prompts. Until then the task shows an editor's pick.
THE NUMBERS SO FAR
- 4 votes, of which 3 counted (updated Sun 22 Nov).
- Position bias (how often the answer shown first is picked) is shown from 100 picks.
- Scores are shown once there are 100 counted votes across the site, and for each model once it has been in 20 of them. Below that, a handful of votes would swing them wildly.
THE MODELS
The rule: for each lab, the best model a subscriber on its standard paid plan (about $20 a month) can pick in its chat app. Reviewed monthly; no model is ever removed silently. Labs get no early access, no private testing and no say, and the site takes no money from them.
- GPT-6.2 Astra (OpenAI, the ChatGPT app).
- Claude Opus 5.5 (Anthropic, the Claude app).
- Gemini 4 Neon (Google, the Gemini app).
- DeepSeek V4.1 Flash (DeepSeek, the DeepSeek app), free in the app, open weights.
- Kimi K3 (Moonshot AI, the Kimi app), free in the app, open weights.
Roster changelog
- Mon 23 Nov: DeepSeek V4 Pro → DeepSeek V4.1 Flash (1017). DeepSeek is phasing out V4 Pro, and V4.1 Flash 1017 runs in its free app and API since 17 Oct 2026. DeepSeek rankings restart. (DeepSeek: V4.1 Flash 1017)
- Mon 23 Nov: Gemini 4 Argon → Gemini 4 Neon. Google AI Pro ($19.99/month) subscribers can pick Gemini 4 Neon in the Gemini app since 18 Nov 2026, and the plan no longer lists Argon. Gemini rankings restart; earlier answers and votes stay in the archive. (Google: Introducing Gemini 4 Neon)
- Mon 23 Nov: GPT-6.1 Sol → GPT-6.2 Astra, before GPT-6.1 Sol's first game. ChatGPT Plus ($20/month) includes GPT-6.2 Astra, OpenAI's newest and strongest model, in ChatGPT Work and Codex since 17 Nov 2026. OpenAI rankings restart with the 23 Nov game. (OpenAI: Introducing GPT-6.2 Astra and Sol)
- Mon 23 Nov: GPT-6 Sol → GPT-6.1 Sol. ChatGPT Plus ($20/month) includes GPT-6.1 Sol in ChatGPT Work and Codex. OpenAI rankings restart with the 23 Nov game; earlier answers and votes stay in the archive. (OpenAI: GPT-6 and other models in ChatGPT)
- Sun 18 Oct: Gemini 3.1 Pro → Gemini 4 Argon. Gemini 4 Argon became available to Google AI Pro subscribers ($19.99/month) in the Gemini app on 13 Oct 2026. Gemini rankings restart; earlier answers and votes stay in the archive. (Google: Gemini 4 Argon in the Gemini app)
- Sat 17 Oct: Launch roster: GPT-6 Sol, Claude Opus 5.5, Gemini 3.1 Pro, DeepSeek V4 Pro, Kimi K3. The best model a ~€20/month subscriber can pick in each lab's chat app (DeepSeek and Kimi: free in their apps).
EVERY PROMPT AND ANSWER
Each test's prompts and all five answers are published once voting on it has closed, two days after the next test comes out.
- Blind test No. 9 · Mon 9 Nov
- Blind test No. 8 · Mon 2 Nov
- Blind test No. 7 · Mon 26 Oct
- Blind test No. 6 · Thu 22 Oct
- Blind test No. 5 · Wed 21 Oct
- Blind test No. 4 · Tue 20 Oct
- Blind test No. 3 · Mon 19 Oct
- Blind test No. 2 · Sun 18 Oct
- Blind test No. 1 · Sat 17 Oct
Rejected prompts
None so far.
An anonymised download of all votes will be published monthly.