Benchmarking Open-Weight Models on Real Tasks

Sample content. This is a placeholder post for the first version of the site.

Why this matters

We tested open-weight models across coding, reasoning and tool-use tasks. Results, cost and what the numbers mean. This section is filler text that shows how long-form reading looks on Andraid, with comfortable line length and spacing.

A quick example

const agent = createAgent({ model: 'frontier-model', tools: [search, shell] });
for await (const step of agent.run('Fix the failing test')) {
  console.log(step.type, step.summary);
}

Comparison

Model	Context	Tool use	Price / 1M tokens
Model A	200K	Excellent	$3.00
Model B	1M	Good	$1.25
Open-weight C	128K	Fair	Self-hosted

Takeaways

• Measure on your own tasks, not just leaderboards.
• Use prompt caching and small models where possible.
• Keep the harness simple until it hurts.