Two models rewrite the same unhelpful error message. One sounds patient and reassuring. The other tells the reader exactly where to find the missing setting. Someone has to decide which answer helped more. Arena turns choices like that into public evidence about what people prefer. A product team can learn a great deal from those comparisons if it also checks whether the setting exists.
- 01The comparison People submit a prompt, inspect competing answers and vote. Arena combines paired preferences across many encounters.
- 02The data boundary Prompts submitted to the public service can reach providers and public research datasets. Use material cleared for that audience.
- 03The buying decision Reading a leaderboard, commissioning feedback on a model and paying for API inference purchase three different things.
01 / ProductFrom a research experiment to a model comparison service
The project began with researchers from UC Berkeley. It developed from Chatbot Arena into LMArena and then Arena. The company behind it is Arena Intelligence Inc. Its subject has expanded beyond written conversations to work such as generating images and building websites. That history matters when following older links or reading an evaluation announcement under the LMArena name. There is one continuing platform here. Arena company history From LMArena to Arena
In a public battle, a visitor supplies the task and sees two responses before learning which systems produced them. Hiding the names helps someone concentrate on what is in front of them. A brand reputation need not decide which paragraph reads better. Aggregating the votes through a Bradley–Terry statistical model gives Arena a way to estimate relative preferences across a field where every possible pair need not face the same visitor. How Arena battles work Arena voting and data FAQ
The result answers a particular question. It says something about the tendency of participants to choose one response over another under the conditions of that arena. Imagine two answers that both give the wrong return deadline. Voters may consistently prefer the clearer one. That preference is real evidence about presentation. A retailer still has to check the deadline before either version goes into a customer email.
02 / AudienceWho can use a preference signal
A designer choosing a writing assistant can start here because the experience of reading the output is part of the product. A fluent answer that makes the reader hunt for the first step can be a poor fit. Looking through example comparisons can help the designer name qualities to evaluate at home, such as whether a response leads with the next action or gives an explanation before the reader has enough context.
A model developer has a different question. A change to a post-training recipe might improve instruction following on a fixed test while making replies unnecessarily formal. Feedback from people bringing their own prompts can expose cases the developer never wrote down. A lab then needs examples and enough detail about the comparison to investigate changes. A public ranking by itself cannot identify which training change caused a preference to move.
A regulated service or a team handling confidential material should plan the exercise around what it can disclose. Arena explains that providers can receive conversations and that some conversations may be released as research data after de-identification. An unpublished customer complaint may identify someone through its details even after removing a name. A fictional, publishable scenario can preserve the writing problem while keeping private records out of a public battle. Arena voting and data FAQ
03 / WorkflowA proposed comparison for a help centre rewrite
Start with a section of an already public help centre that explains an easily misunderstood policy. Write a prompt asking for a plain-language rewrite for a first-time customer. Include a real constraint from the published material, such as an exception that applies only after the first renewal. Save the original source and the exact input so colleagues can inspect what the models were given.
Before viewing an answer, ask the policy owner what a reader must be able to do afterward. Perhaps they must distinguish exchanging an item from cancelling a recurring order. Sketch a separate preference question for the editor, such as which version a new reader could navigate more easily on a phone. A model can win the editorial comparison and still fail the policy check. Recording those judgments separately makes a disagreement interpretable.
Use the public comparison only for the cleared example. Examine both responses for missing exceptions, invented deadlines and a clear first action. After choosing a preference, record what influenced it before looking at the model labels. To choose two named models, use Arena’s Side by Side mode. The matchup is visible, and these votes are excluded from the public leaderboard. Use direct endpoints when the trial also needs controlled settings or repeated samples. Arena voting and data FAQ How Arena battles work
Bring the responses to a colleague who has never seen the policy. Ask them to resolve a concrete situation using each version. A preference accompanied by the wrong application of a rule should send the team back to its editorial brief. Conversely, two factually correct answers that lead to different levels of reader effort reveal something a fact-check alone would miss. Treat this as a proposed product exercise, with no claim that one model won it.
04 / PricingWhat the public site, a paid evaluation and the API provide
Arena describes the public leaderboard as freely shared. Its commercial AI Evaluations service provides analysis based on community feedback, representative samples for inspection and delivery timelines under a service agreement. The announcement mentions flexible pricing for open-weight models and nonprofit institutions. It gives no universal evaluation price. Ask which tasks, modalities, sample details and delivery obligations a quote covers. Commercial AI Evaluations
| Route | Commercial basis | What it delivers |
|---|---|---|
| Public leaderboard | Freely accessible rankings | A view of observed community preferences |
| Commissioned evaluations | Scope and pricing by arrangement | Community feedback, representative samples and a delivery agreement |
| Arena API direct models | Eligible paid or all-model promotional balance | Inference requests using an explicitly selected model ID |
Commercial routes, consulted 21 November 2026. Commercial AI Evaluations Arena API model selection
The Arena API is another product. Its documentation distinguishes direct model calls from a coding router in preview. Direct model calls require paid funds or promotional funds eligible for all models. With the preview router, the response identifies the router and does not disclose the underlying model that served a particular request. A team reproducing a comparison between named models should choose direct model IDs and retain the settings. A router trial answers a different deployment question. Arena API model selection
05 / DistinctionsWhere a public leaderboard fits in an evaluation stack
The Braintrust Blueprint addresses teams assembling their own examples and evaluating application changes. A private regression set can repeatedly ask the exact renewal question after an edit to a system prompt. Arena brings a broader population of tasks and voters. A useful division of work is to use public results for a first shortlist, then ask whether each shortlisted model survives the same product-specific gates.
The OpenRouter Blueprint is relevant when the next decision is how an application reaches different models. A model identity on a chart does not establish that an integration supports the required tools or behaves the same way under load. Keep a record of the endpoint actually tested. Otherwise a change to the delivery route can look like a change in what users prefer about the model.
Arena also separates evaluating a model from listing a public score. Its policy specifies routes to eligibility, including public weights, an accessible public API or a generally available public service. Anonymous pre-release tests and preliminary scores are subject to separate rules. A research team tracking releases should preserve the date and status of the score it cites, especially while a new version is collecting votes. Arena leaderboard policy
06 / QuestionsWhich aspects of a result travel to another audience
A popular answer format can attract votes on its own. Arena's FAQ describes style controls in the rating system. An editor should still look at what happened in the application: did a bulleted response make the next action clearer, or simply look more organised in a side-by-side comparison? Preserve a few examples of both outcomes so a future prompt change can be discussed in terms of readers' experience. Arena voting and data FAQ
The same caution applies to the mix of tasks. A broad text leaderboard can include activities far removed from reading a contract exception or following a troubleshooting procedure. A narrow application may depend on situations that are rare among public prompts. Ask the person proposing a switch to show one real requirement for which the public signal is relevant and one that needs separate evidence. Both questions belong in the decision record.
07 / DecisionThree reasons to open Arena
Shortlisting a writing model
Choose the leaderboard closest to the intended task, keep the observation date, then run the candidates on publishable examples with an editor and a policy owner.
Investigating a model release
Define the behavior that changed, ask what representative feedback a commissioned evaluation can return and read the public listing policy before announcing a score.
Selecting an API route
Decide whether each response must be attributable to a named model. Check credit eligibility and compare the direct and preview-router documentation for the tool that will send requests.
A business worth understanding.
Suggest your business or one you find interesting. Tell us what you want to understand about its product, positioning, design or workflows.
Suggestions are free. Selection and publication stay with the desk.
- Arena company historyConsulted
- From LMArena to ArenaConsulted
- How Arena battles workConsulted
- Arena voting and data FAQConsulted
- Commercial AI EvaluationsConsulted
- Arena leaderboard policyConsulted
- Arena API model selectionConsulted

