An open-weight model can give an organization control of the weights while leaving it with a difficult operating question. Who makes sure the next thousand requests finish on time when some users paste a paragraph and others submit a book? Inferact, founded by creators of the open-source vLLM serving engine, sells help with that second problem. Its commercial offering spans customer-operated infrastructure, provider-supplied GPUs and model-specific serverless rates.
- 01What it sells A serving platform with workload tuning, authenticated endpoints, observability and rollout controls.
- 02Who should look Teams that have picked a model and need to reconcile its behavior, performance and operating cost.
- 03What to bring A representative traffic sample, a quality baseline and a realistic account of how often prompts actually repeat.
01 / ProductFrom a model checkpoint to an endpoint people can depend on
The platform overview describes two infrastructure arrangements. Inferact Platform runs in a customer’s cloud or data center. Inferact Cloud uses GPUs the provider provisions. Both address the work of turning model weights into an available service. The person building an application still chooses what to ask the model, which answers are acceptable and what should happen when an endpoint runs slowly. Those decisions provide the operating targets that a serving team can tune against.
The account of the vLLM relationship explains why an organization might pay even though the engine is open. New checkpoints, hardware and traffic patterns can each change the best serving configuration. The underlying project supplies widely usable engine capabilities. Inferact describes the commercial system as a way to optimize a particular combination and operate it over time. Its account also sketches a longer-term ambition for autonomous tuning. Treat illustrative future operations in that essay as a research direction when specifying what a contract must deliver today.
A concrete example comes from Tencent’s Hy4 collaboration with Inferact. The announcement assigns the checkpoint and its required behavior to Tencent. Inferact operates the serving path and is responsible for areas including routing and capacity. This is evidence of the company’s role in a particular deployment. An ordinary customer should establish which duties transfer in its own engagement, especially who validates a model revision before live traffic reaches it.
02 / AudienceA stronger fit once a team knows which model it wants
Consider a software publisher building a reading assistant for technical manuals. Its editors already know which mistakes would make an answer unusable, such as citing a different product revision. If they can pick an acceptable checkpoint and assemble a representative set of questions, engineers can investigate serving cost and latency without repeatedly reopening the model selection exercise. A managed endpoint gives the publisher a way to separate the two investigations.
A model developer has a different reason to engage. A new architecture might require engineering work before a standard runtime can serve it efficiently. The commercial question is then about responsibility for integration, verification and ongoing operation. A buyer that simply wants access to a popular model for an occasional prompt can start with the published serverless offer. A buyer whose model is absent from the rate card needs a discussion about support before doing financial arithmetic.
03 / WorkflowReplay a day in the life of a manual-reading assistant
In a proposed evaluation, select three sorts of conversations from the publisher’s own permitted material. Include short lookups that identify one part, long questions that supply an entire manual, and follow-up turns that reuse a previously supplied section. Keep a reference answer with its source passage for each case. Freeze the prompt template and model version so a promising latency measurement cannot quietly come from shortening the task or letting the model skip a citation.
First replay the requests individually. Record the time until the first useful text appears and the time until the answer is complete. Then replay the same distribution in short bursts, including moments when one long manual arrives beside several small questions. A steady stream of equally sized requests can conceal delays that a real reader experiences. Record failures and retries as requests that consumed time, even where a final successful response looks normal to the user.
Inferact’s engineering study on long-input traffic offers a reason to run both experiments. Processing an input and generating its answer place different demands on the serving system. The article examines specific arrangements for sharing prefill work across accelerators. Its findings depend on the tested workload and configuration. Ask which part of your measured waiting time is caused by the input stage before assuming an improvement in that stage will fix a slow conversation.
Have an editor reread the returned answers against the same excerpts, including answers from the busiest windows. Faster serving matters little to the publisher if a new configuration omits a required reference or truncates an explanation. A clean handoff from this pilot would include the request mix, the observed delay distribution, the rate of acceptable answers and the name of the person authorized to roll back a new deployment.
04 / PricingA serverless bill depends on three separate token totals
| Model | Input | Cached input | Output |
|---|---|---|---|
| MiniMax M3 | $0.300 | $0.060 | $1.200 |
| MiniMax M3.1 Flash | $0.150 | $0.015 | $0.600 |
| Kimi K3.1 | $2.000 | $0.200 | $10.000 |
Selected US-dollar rates per million tokens, consulted 19 November 2026. Inferact serverless pricing.
Inferact says charges draw on account credits. Calculate uncached input, eligible input actually served from cache, and generated output separately. For example, at the MiniMax M3 rates above, two million ordinary input tokens, one million cached input tokens and half a million output tokens would cost $1.26. This is our arithmetic for the stated counts. Confirm the recorded breakdown from actual usage before assuming a repetitive application will earn the cached rate.
The serverless page directs its getting-started and higher-volume inquiries to the team. Check onboarding, available capacity and any account terms directly. The listed token figures support an estimate for the named serverless models. A dedicated deployment on your own GPUs changes the cost boundary. It adds customer-side infrastructure and a separately agreed commercial relationship. Obtain that arrangement in writing before comparing it with a month of metered requests.
05 / DistinctionsA runtime, a serving operator and a GPU supplier solve different problems
CoreWeave’s GPU infrastructure is a useful comparison for a team confident in its own model-serving engineers. Access to compute gives those engineers somewhere to deploy a runtime. It still leaves the work of tuning the runtime, checking model fidelity and responding to traffic changes. Put a realistic value on that engineering time when deciding how much responsibility to retain, especially if capacity is already under contract elsewhere.
Baseten’s deployment approach provides a closer comparison for teams buying operated inference. Give both prospective providers the same model and workload description. Ask for the deployment location, the division of responsibility and the evidence they would provide after a failed rollout. This exercise can reveal a meaningful fit difference even when two demonstrations return similar answers. For a model developer, support for the exact checkpoint and its output conventions may settle the choice.
06 / QuestionsWhat happens after the traffic or the checkpoint changes?
Cache eligibility is a good place to question an optimistic forecast. A documentation publisher may reuse an introduction in every request, then change that introduction when a manual revision ships. Record how frequently that happens and watch the effect on both the token breakdown and reader latency. Ask how to distinguish a temporary cold period from a persistent change in how the application constructs prompts.
A second question concerns portability. Document which decisions a customer can reproduce in its own vLLM environment and which depend on the operated platform. Before a seasonal spike, agree how much extra demand the operator will accept, which events cause throttling and how users will see a failure. The answer should refer to your endpoint and agreement. A chart for a different model cannot establish the behavior of this particular service.
07 / DecisionChoose who owns the next serving problem
A product with irregular usage
Begin with an offered serverless model and a short run of genuine requests. Measure the three token buckets, response quality and user-visible waiting time.
An organization with an existing cluster
Bring the exact model, deployment constraints and current latency trace to a platform discussion. Identify who can adjust or roll back each serving component.
A lab releasing a new architecture
Treat optimization and model fidelity as joint acceptance criteria. Name who owns reference outputs and which behaviors must survive each serving change.
A business worth understanding.
Suggest your business or one you find interesting. Tell us what you want to understand about its product, positioning, design or workflows.
Suggestions are free. Selection and publication stay with the desk.
- Inferact Platform and CloudConsulted
- Serverless model rates and credit meteringConsulted
- How Inferact relates to open-source vLLMConsulted
- Pipeline parallelism for long-input workloadsConsulted
- Tencent and Inferact on operating Hy4Consulted


