Choosing AI Models

לקריאה בעברית

At current API rates, GPT-6 Astra costs a hundred times as much per token as GPT-6 Luna. Before paying that premium, I want to know what it actually buys.

The price of a token is only part of the cost of a task. Providers charge separately for the text a model receives and the text it generates, measured in units called tokens. The prompt and final answer can be the visible ends of a much longer process. A reasoning model generates intermediate tokens billed as output. An agent may write commands, read documents and tool results, and go through the same material again as it works.

A model that reaches a good result in fewer attempts can therefore be cheaper to use even at a higher rate. For a calculation, a short calculator command may save a long stretch of generated arithmetic, although the command and returned result still consume tokens and the tool may have its own charge. More reasoning can also make the result worse: in an Anthropic study using constructed tasks, extending the reasoning sometimes reduced accuracy.

The quality of the result changes the calculation too. In one evaluation run of a Bureau of Labor Statistics plugin on 108 held-out questions, Astra achieved a 94% pass rate and Luna 88%. Six percentage points can look like a modest difference. But on the task I need done, the difference may be between code that works and code with a critical bug. If I have to find and fix the bug, that work may cost far more than the tokens. A mistake that goes unnoticed can cost more still.

I would choose a model much as I would choose someone for a job. Benchmarks and model recommendations help me decide whom to consider; I still want to see the work. For a task I do often, it is worth trying several models and a few reasoning settings. If the cheaper model does the job well, there is no reason to pay more. If I keep having to correct it, I may be saving on tokens while paying for the difference with my own time.