Ollama connects local open models with cloud inference

Ollama makes open AI models available through a local runtime and a hosted cloud service. A developer can run a model on a workstation, connect an existing coding tool, or call cloud inference from an application. The useful decision is where each workload should run and which capabilities that route actually supports. Sharing a familiar interface does not make local hardware and hosted models interchangeable in every respect.

01 / ProductA runtime and cloud service around open models

The documentation overview positions Ollama across desktop applications, coding agents and developer APIs. Its library includes models for different jobs, rather than one assistant with a fixed personality. A downloaded model can run on your own machine; a cloud model runs on infrastructure operated for Ollama. That placement changes capacity, privacy review and recurring cost even when the application experience looks similar.

The cloud guide documents direct authenticated API access without installing the desktop application. It also explains that cloud model names used by the API differ from the app or CLI convention. Developers should use the names returned by the service rather than guess a suffix. OpenAI and Anthropic compatibility covers subsets of those interfaces, so existing client code still needs integration checks.

Ollama does not remove the distinction between a model and the application around it. A model can suggest a tool call, but software must supply the tool, execute it and interpret the result. The company therefore sits alongside agent frameworks and application infrastructure. Its appeal is a convenient model execution layer, while the business workflow remains the responsibility of the team building on top.

02 / AudienceTeams deciding where their inference should live

A software team experimenting with an internal categorisation feature can start locally, inspect the request and output, then decide whether its own machine has sufficient capacity. A developer using a coding agent may instead value a cloud route to larger open models without owning the necessary GPU. These are different adoption paths: the first centres on an embedded feature, the second on a tool used by a person.

Local execution is attractive when the information and processing should remain within a controlled environment. That does not establish that the entire application is offline: connected tools may still contact external services. Conversely, cloud execution is useful when device capacity is the constraint, but the data now crosses a service boundary. Decide on that boundary before choosing a model merely because it appears in a menu.

For a managed endpoint with a broader deployment and serving service, compare Together AI. For teams selecting and distributing models across a development ecosystem, Hugging Face provides a different starting point. Ollama is most relevant when the runtime and familiar integration surface themselves simplify the job. It is less compelling if a team already has a satisfactory serving stack and no local-use requirement.

03 / WorkflowProposed workflow: extract a structured local record

Consider a proposed internal document triage tool. Start with a small set of representative, non-sensitive messages and define fields for subject, requested action and evidence text. Select a local model and send the expected JSON schema with each request. The structured-output guide describes schema-constrained responses and application-side validation. This is a design proposal, not a report of a completed implementation.

Keep the extracted evidence alongside each suggested field. A syntactically valid record can still misread the document, omit an exception or confuse a requested date with a confirmed date. Compare the output with human-labelled examples, including incomplete messages and contradictory instructions. Route missing or uncertain fields for review rather than filling them with plausible values. This evaluates the usefulness of the entire extraction feature, not only JSON validity.

The important deployment constraint is explicit: the same structured-output documentation says Ollama’s cloud currently does not support structured outputs. Do not move this workflow to a cloud model on the assumption that the local schema feature comes with it. Either keep the qualifying local route, redesign the validation strategy, or choose a hosted service that documents the required behaviour. That capability difference can outweigh apparent savings in machine administration.

If the application later needs a lookup, the tool-calling guide shows a model requesting a function and receiving its result in a follow-up message. Begin with a read-only lookup. Check arguments in ordinary application code and return a bounded result. A model-proposed function call is a request for an action; it should not silently become permission to alter a business record.

04 / PricingSeparate local hardware from cloud usage

The live pricing page offers pay-as-you-go cloud usage alongside subscriptions with included credits. Pro is displayed at US$20 monthly or US$200 billed annually; Max is US$100 monthly. Local model execution has no Ollama usage charge, but the computer, memory, electricity and maintenance remain part of the operating cost. A free runtime is not a zero-cost production environment.

Cloud prices depend on the selected model’s token rates. Included subscription credits are not an unlimited capacity promise, and model calls can contain more context than the visible user question. Estimate input, output and repeated conversation context separately. A short answer after a long document may consume materially more input than output. Compare subscriptions against the actual workload instead of assuming that the highest tier is automatically the economical choice.

Team is labelled early access, and some advertised collaboration features are marked coming soon. These distinctions matter when choosing a shared operating arrangement. Confirm the live account controls and current usage balance before expanding a pilot to colleagues. No subscription was purchased for this article, and no measured token bill or throughput result is being asserted.

05 / DistinctionsOne familiar surface across different model placements

Ollama’s practical distinction is continuity between local experimentation, existing tools and cloud model access. The cloud documentation includes model retirement notices for recently used hosted models and states that downloaded local models are unaffected by those retirements. That creates an operational choice: local files give a team a stable artifact to retain, while a hosted catalogue is a service whose lifecycle must be monitored.

The FAQ describes how to inspect whether a running model occupies GPU memory, system memory or a mixture. This is more informative than judging a deployment solely by its model name. Two machines running the same model may behave differently because of memory placement and context settings. Record the model artifact, runtime version and configuration when comparing results, so a later change can be explained.

These controls are useful for investigation, but they do not establish that open models are equivalent to a particular proprietary service. Model selection still depends on the task, licence, input type and required reliability. A useful evaluation includes cases where the model should ask for clarification or decline to produce an answer. Convenience in loading a model should shorten setup, not replace an acceptance test.

06 / QuestionsQuestions to settle before committing the application

First, determine the maximum realistic input size. A small demonstration message says little about a long document with embedded tables or contradictory notes. The runtime allows context configuration, but increasing context can alter memory pressure and speed. Measure the chosen configuration on the actual device rather than promising colleagues a response time derived from a vendor demonstration or another model’s benchmark.

Second, make the network boundary explicit. Local-only operation requires local models and a surrounding workflow that respects the same boundary. Cloud prompts are processed remotely; the official cloud guide states that they are not used for model training. That statement is a provider policy, not independent verification of every connected tool, application log or downstream storage system. Review the complete route for the information involved.

Finally, track model-specific capability and licence conditions. A model that supports text generation may not support the tools or input format needed by the application. Keep a fallback that reports failure clearly when a model is retired or a response cannot be validated. Retrying a request without a limit can increase cost while leaving the original cause unresolved.

07 / DecisionChoose the execution route before scaling the feature

Ollama deserves attention when local control and easy access to open models are central to the work. Build one bounded feature, preserve the evidence that makes its output checkable, and compare local and cloud routes only where their documented capabilities match. The strongest pilot is one that explains both a successful result and a rejected result without relying on the model’s fluency as proof.