Model picker

Picks the right LLM for a task by weighing capability, context window, latency and price, and backs the choice with a quick side-by-side test on real inputs. Use when the user asks which model to use, wants a cheaper model, or a model fails at a task.

AI models & APIsPack: AnyModel83 linesmodel-picker

Install

Run one command — it saves SKILL.md into your agent's skills folder. Restart the agent and the skill is picked up automatically.

Claude Code — macOS, Linux
mkdir -p ~/.claude/skills/model-picker && curl -fsSL https://anymodel.org/en/skills/anymodel/model-picker/SKILL.md -o ~/.claude/skills/model-picker/SKILL.md
Codex, Cursor — macOS, Linux
mkdir -p ~/.agents/skills/model-picker && curl -fsSL https://anymodel.org/en/skills/anymodel/model-picker/SKILL.md -o ~/.agents/skills/model-picker/SKILL.md
Claude Code — Windows PowerShell
New-Item -ItemType Directory -Force "$HOME\.claude\skills\model-picker" | Out-Null; Invoke-WebRequest https://anymodel.org/en/skills/anymodel/model-picker/SKILL.md -UseBasicParsing -OutFile "$HOME\.claude\skills\model-picker\SKILL.md"
Codex, Cursor — Windows PowerShell
New-Item -ItemType Directory -Force "$HOME\.agents\skills\model-picker" | Out-Null; Invoke-WebRequest https://anymodel.org/en/skills/anymodel/model-picker/SKILL.md -UseBasicParsing -OutFile "$HOME\.agents\skills\model-picker\SKILL.md"

Any other agent: save the file into its skills folder, or give it the link to the file: https://anymodel.org/en/skills/anymodel/model-picker/SKILL.md

SKILL.md

Raw

Model Picker

Choosing a model is an engineering decision with a measurable outcome. Do not answer from memory or brand loyalty: model lineups and prices change every few months.

1. Pin down the task

Ask, or infer from the code, before naming any model:

  • Task type: agentic coding, code review, chat, extraction to JSON, classification, summarization, translation, vision, long-document Q&A.
  • Quality bar: what does a wrong answer cost? (A typo in a tweet vs. a wrong SQL migration.)
  • Input size: typical and worst-case tokens per request, including system prompt and tools.
  • Volume: requests per day and whether they run in a user-facing path (latency matters) or a batch job (throughput and price matter).
  • Hard constraints: data residency, tool calling, JSON mode, image input, streaming.

2. Build the shortlist

Get the current list instead of relying on memory. With AnyModel one key reaches every model:

curl -s https://anymodel.org/v1/models -H "Authorization: Bearer $ANYMODEL_API_KEY" | jq -r '.data[].id'

Prices and context windows per model are on https://anymodel.org/en/models.

Pick three candidates across tiers:

Tier Typical fit
Flagship hard reasoning, multi-file agentic coding, ambiguous specs
Mid everyday coding, review, most chat and RAG answers
Small / flash classification, extraction, routing, high-volume batch work

Drop any candidate whose context window is below the worst-case input, or that lacks a required feature (tool calls, vision, JSON).

3. Test on real inputs

  1. Collect 10–30 real examples, including the awkward ones (long input, messy data, edge cases).
  2. Write down what a correct answer looks like for each before running anything.
  3. Run every candidate with the same prompt, temperature and max tokens. Record per call: correctness (pass/fail against step 2), latency, input and output tokens.
  4. For structured output, validate it programmatically (JSON schema, tests) instead of eyeballing.

A minimal harness is a loop over examples and model ids that writes a CSV; keep it out of the production code path.

4. Decide

  • Choose the cheapest model that passes the quality bar, not the best model overall.
  • If the small model passes 90% and fails on a recognizable subset (long inputs, a language), route that subset to a bigger model instead of upgrading everything.
  • Compute monthly cost: (avg input tokens × input price + avg output tokens × output price) × monthly requests. Show the number, not just "cheaper".
  • Keep the model id in configuration (env var or config file), never scattered through code, so switching later is a one-line change.

Report back

Task: <one line>
Candidates: <model A>, <model B>, <model C>
Results (n=<examples>): A 28/30, 3.1s, $X/month · B 27/30, 1.2s, $Y/month · C 19/30, 0.6s, $Z/month
Pick: <model> — <why, in one sentence>
Fallback: <model> for <subset> (if any)
Where it is configured: <file / env var>

Pitfalls

  • Comparing models on one cherry-picked example.
  • Changing the prompt between candidates, so the comparison measures the prompt, not the model.
  • Ignoring output tokens: verbose models can cost more despite a lower input price.
  • Hardcoding a model id that is later retired; always keep it configurable.

Source: AnyModel skills catalog · License: CC BY 4.0

Connect through AnyModel API

This skill runs on whatever model your agent uses. Point Claude Code, Codex or Cursor at AnyModel to run it on Claude, GPT, Gemini and more with one key — pay per token, no subscription.

More from the pack AnyModel