Open-weight AI models give enterprises three deployment choices: access the model through a managed API, operate it on rented Amazon Web Services infrastructure, or download the weights and run them on owned servers.
The optimal choice depends on more than the advertised token price. Enterprises must consider internal engineering usage, consumer-facing traffic, peak concurrency, data policies, availability and infrastructure staffing.
A representative enterprise workload
Consider an enterprise with 100 internal engineers. Each engineer processes an average of 300,000 tokens per working day across coding, debugging and agentic tasks. Over 22 working days, that produces approximately 660 million tokens per month.
Assume the workload consists of 90% input and 10% output, with 70% of input served from cache. At Kimi K3’s rates of $3 per million uncached input tokens, $0.30 for cached input and $15 for output, the internal engineering workload costs approximately $1,649 per month through the API.
Now add a consumer-facing AI feature.
Assume 10% of monthly active users are active daily, each active consumer makes five interactions per day, and every interaction uses 2,000 input tokens and 500 output tokens. With 30% of consumer input cached, each interaction costs approximately 1.19 cents.
At 10,000 monthly active consumers, combined consumer and engineering API expenditure is approximately $3,431 per month. At 50,000 consumers, it becomes $10,559. At 100,000 consumers, the enterprise processes 1.5 million monthly interactions and spends approximately $19,469.
At 250,000 monthly active consumers, combined expenditure reaches $46,199. At 500,000 consumers, it becomes $90,749. At one million monthly active consumers, producing 15 million interactions, the combined API bill reaches approximately $179,849 per month.
These estimates cover model inference. They exclude embeddings, databases, moderation, networking, observability and application development.
Option 1: Managed API
A managed API offers the fastest route to production. The provider operates the GPUs, deploys the model, manages capacity and handles hardware maintenance. The enterprise pays according to actual token consumption.
The advantages are low initial investment, rapid integration and variable expenditure. When demand falls, the bill falls. The enterprise can test a model without making a long-term infrastructure commitment.
The disadvantages include provider dependency, external processing of company data, rate limits and exposure to pricing or model changes. At sufficiently high and sustained usage, per-token billing can exceed the cost of owned infrastructure.
For most enterprises, an API is the logical starting point because it provides real usage data before a major infrastructure decision is made.
Option 2: Amazon Cloud
An enterprise can rent an AWS instance containing eight NVIDIA B300 GPUs and operate the model itself. This provides greater control over model versions, private networking, security policies and data location without purchasing physical hardware.
AWS is useful for temporary dedicated deployments, controlled experiments and workloads with large but irregular capacity requirements. Infrastructure can be provisioned without waiting for hardware procurement.
However, renting GPUs is not the same as consuming a managed model API. The enterprise must install the inference engine, load the weights, configure distributed execution, monitor performance and manage failures.
An eight-B300 instance costs approximately $142 per hour. Running continuously costs roughly $104,000 per month before storage, networking, support and engineering. The instance continues charging while it is running but idle.
Option 3: Owned self-hosted infrastructure
Self-hosting means buying GPU servers and operating the model in an enterprise data centre or colocation facility.
Its main advantages are control, data isolation and model permanence. The organization can freeze a model version, modify the deployment and continue using it even if the original provider changes its API. This is relevant to regulated, sovereign and air-gapped environments.
At high and consistent utilization, owned infrastructure can also be less expensive than continuously rented cloud GPUs.
The disadvantages are capital investment and operational responsibility. An eight-B300 server may cost approximately $786,000. The enterprise must also manage power, cooling, networking, hardware failures, inference software and security. Production availability may require a second server, substantially increasing the investment.
Monthly and annual cost comparison
Using the representative workload of 100 engineers and 100,000 monthly active consumers, the managed K3 API costs approximately $19,469 per month, or $233,628 per year.
One continuously running eight-B300 AWS instance costs approximately $103,966 per month, or $1,247,592 per year, excluding several supporting cloud and operational expenses.
One owned eight-B300 server has an estimated five-year average cost of approximately $25,095 per month, or $301,136 per year, including modeled hardware, maintenance, electricity, cooling and colocation. It also requires approximately $786,000 upfront.
A production deployment with two owned servers for failover averages approximately $50,190 per month, or $602,272 per year, before internal staffing.
What is optimal?
Most enterprises should start with a managed API and measure token volume, cache performance, concurrency and latency. Amazon Cloud becomes appropriate when private cloud deployment or temporary dedicated capacity is required. Owned infrastructure becomes relevant when usage is large and predictable or when data cannot leave the organization.
The mature architecture may be hybrid: owned servers handle predictable baseline demand, APIs absorb traffic peaks, and smaller models process routine tasks while larger models handle complex work.
Open weights provide control. APIs provide flexibility. Cloud provides infrastructure without ownership. The optimal enterprise choice is the one aligned with actual utilization, security requirements and operational capability.
