---
name: model-picker
description: Picks the right LLM for a task by weighing capability, context window, latency and price, and backs the choice with a quick side-by-side test on real inputs. Use when the user asks which model to use, wants a cheaper model, or a model fails at a task.
---

# Model Picker

Choosing a model is an engineering decision with a measurable outcome. Do not answer from memory
or brand loyalty: model lineups and prices change every few months.

## 1. Pin down the task

Ask, or infer from the code, before naming any model:

- **Task type**: agentic coding, code review, chat, extraction to JSON, classification,
  summarization, translation, vision, long-document Q&A.
- **Quality bar**: what does a wrong answer cost? (A typo in a tweet vs. a wrong SQL migration.)
- **Input size**: typical and worst-case tokens per request, including system prompt and tools.
- **Volume**: requests per day and whether they run in a user-facing path (latency matters) or a
  batch job (throughput and price matter).
- **Hard constraints**: data residency, tool calling, JSON mode, image input, streaming.

## 2. Build the shortlist

Get the current list instead of relying on memory. With AnyModel one key reaches every model:

```bash
curl -s https://anymodel.org/v1/models -H "Authorization: Bearer $ANYMODEL_API_KEY" | jq -r '.data[].id'
```

Prices and context windows per model are on https://anymodel.org/en/models.

Pick three candidates across tiers:

| Tier | Typical fit |
|---|---|
| Flagship | hard reasoning, multi-file agentic coding, ambiguous specs |
| Mid | everyday coding, review, most chat and RAG answers |
| Small / flash | classification, extraction, routing, high-volume batch work |

Drop any candidate whose context window is below the worst-case input, or that lacks a required
feature (tool calls, vision, JSON).

## 3. Test on real inputs

1. Collect 10–30 real examples, including the awkward ones (long input, messy data, edge cases).
2. Write down what a correct answer looks like for each before running anything.
3. Run every candidate with the same prompt, temperature and max tokens. Record per call:
   correctness (pass/fail against step 2), latency, input and output tokens.
4. For structured output, validate it programmatically (JSON schema, tests) instead of eyeballing.

A minimal harness is a loop over examples and model ids that writes a CSV; keep it out of the
production code path.

## 4. Decide

- Choose the **cheapest model that passes the quality bar**, not the best model overall.
- If the small model passes 90% and fails on a recognizable subset (long inputs, a language),
  route that subset to a bigger model instead of upgrading everything.
- Compute monthly cost: `(avg input tokens × input price + avg output tokens × output price) ×
  monthly requests`. Show the number, not just "cheaper".
- Keep the model id in configuration (env var or config file), never scattered through code, so
  switching later is a one-line change.

## Report back

```
Task: <one line>
Candidates: <model A>, <model B>, <model C>
Results (n=<examples>): A 28/30, 3.1s, $X/month · B 27/30, 1.2s, $Y/month · C 19/30, 0.6s, $Z/month
Pick: <model> — <why, in one sentence>
Fallback: <model> for <subset> (if any)
Where it is configured: <file / env var>
```

## Pitfalls

- Comparing models on one cherry-picked example.
- Changing the prompt between candidates, so the comparison measures the prompt, not the model.
- Ignoring output tokens: verbose models can cost more despite a lower input price.
- Hardcoding a model id that is later retired; always keep it configurable.

Source: [AnyModel skills catalog](https://anymodel.org/en/skills) · License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
