开放模型推理市场有多大？
论证要点
- 作者以 Baseten、Fireworks、Together AI 三家开放模型推理创业公司作为风向标，并参考 OpenRouter 报告的开放模型 token 占比，因为它们的命运与开放模型推理这一品类直接相关。
- 公开披露的 token 量呈近指数级增长，且这一增长发生在 GLM 5.2 带来开放 agent 阶跃变化所引发的上扬之前。
- Fireworks 在首次公司大会上称，每天为客户定制模型提供超过 40T token，这只是其推理业务的一个切片，该垂直业务自今夏以来增长超过 2 倍。
- 作者预计 Fireworks 秋季总 token 量将远超 100T，但强调这是基于单一业务切片的推断，而非完整披露。
- 作者用 Google 财报中披露的全平台与 API token 量做对照，试图把开放模型推理规模锚定在闭源或超大规模厂商的 token 量上。
- 作者承认现有测量极不精确：平台指标有噪声、创业公司数据带营销色彩、公开评测不完美，需要内行判断趋势。
对实践的启示
对做 AI 应用的人来说，开放模型推理是一个真实且快速增长、但难以精确量化的市场；选型时可把 Fireworks、Together 等作为成本与供给的参照，但不要轻信单一 token 数字，需结合自身场景验证。
作者承认的反方与局限
作者承认所有开放模型采用数据都极不精确，超大规模厂商（如 Amazon Bedrock、Azure model garden）的开放模型用量未按模型类别拆分，因此无法完整衡量该市场规模。
原文摘引
Still, all the measurements we have on open model adoption are incredibly imprecise.
This shows near-exponential growth, before the uptick that would follow GLM 5.2’s step change in open agents.
Is it a rounding error relative to OpenAI/Anthropic?
▸ 展开原文全文（7 段，抓取自 Interconnects）
My longest running agent testbed has been building the Interconnects Dashboard — an aggregator of all of the best data on open model adoption and performance from across the web. This dashboard is freely available and becoming quite useful. Still, all the measurements we have on open model adoption are incredibly imprecise. We have noisy platform metrics, marketing-related metrics from startups, and imperfect public evaluations. It takes an informed eye to figure out where things are going. These research notes are my current best approximations of where we stand.
As with most things on Interconnects, our goal is to build well more value than we capture, so the data on the dashboard will remain free for the foreseeable future. These notes will be more raw than many of my articles, a place for sharing what I’m mulling over but don’t have complete evidence for (yet). They will be primarily paid, enabling me to speak more freely. If you work in AI policy or policy-facing research, please reach out to us and we can likely get discounted or free access.
The primary question I wanted to address today is: What is the market for open model inference? Is it growing? Is it a rounding error relative to OpenAI/Anthropic?
I’ve been building the best measurements I can here, primarily using the open model inference startups Baseten, Fireworks, and Together AI as the bellwethers here (along with the open model share of OpenRouter’s token volume, which they thankfully report in their API). The hyperscalers also likely have a major footprint here, e.g. Amazon Bedrock and Azure’s model garden, but I haven’t seen them break down usage by model category. The startups’ fates are much more directly correlated with the success of open model inference as a category.
For the latest data, we can start with a slide from my talk on the open model economy (recording here). These are the public disclosures from the open model startups on their token volume, over time, log-scale.
This shows near-exponential growth, before the uptick that would follow GLM 5.2’s step change in open agents. For example, Fireworks announced at their first company conference this week that they’re serving more than 40T tokens per day to customers’ customized models. This is just one slice of their inference, a vertical that they said grew over 2x from this summer, so I would expect their total number this fall to be well above 100T tokens when we get the next update.
My first instinct with these numbers has been to ground them versus any closed model or hyperscale company we can, at least in terms of token volume. Thankfully, Google releases token volumes for the entire platform and API usage in their earnings reports. We can overlay those two numbers on top of the open model inference platforms, with the resulting plot below: