DigitalOcean gives application teams several ways to consume AI within its cloud: direct model APIs, managed dedicated GPU endpoints and agents with attached knowledge bases. The current documentation calls the service Inference; older Gradient AI Platform documentation redirects there. The best starting point is choosing which operating responsibility the application needs to retain.
- 01The offer A common model catalog with serverless and dedicated inference plus managed agent features.
- 02The fit Developers adding AI to an application, especially when its compute and data already live on DigitalOcean.
- 03The boundary This review uses current public documentation and a proposed workflow; no models or deployments were benchmarked.
01 / ProductOne catalog exposes several different operating models
The Inference overview groups model discovery, a playground, direct serverless requests, dedicated deployments, routing, batch processing and managed agents. DigitalOcean-hosted models and third-party commercial models appear within this platform, but their execution and data-handling arrangements differ. A shared catalog should not be mistaken for one identical hosting or retention policy.
Serverless inference lets developers call supported models without first building an agent or provisioning an endpoint. Dedicated Inference instead supplies managed GPU capacity for an endpoint, with configuration choices around the model and hardware. Managed agents add application-level features such as instructions, knowledge bases and guardrails above those model interactions.
DigitalOcean remains the company identity for this coverage. The current Inference documentation is the basis for the workflow and billing discussion, rather than older Gradient or Paperspace subscription terminology. A familiar brand name can survive in commands or historical guides even when the current product structure has changed.
This separation is useful because the same product team may need different routes over time. An irregular feature can begin with serverless requests. A stable, heavily used custom model may justify dedicated capacity. Those are decisions about workload and responsibility, not automatic steps in a required upgrade ladder.
02 / AudienceDevelopers should choose the smallest service that owns the right job
A software company building a documentation assistant can use a managed agent when its retrieval and conversation needs match the platform’s features. Another team that already has its own retrieval, interface and evaluation system may prefer direct model requests. Buying both application layers without a clear division can make incidents harder to diagnose.
Dedicated inference is relevant when a team needs to host a supported custom model or optimize an endpoint for steady traffic. The dedicated guide explicitly contrasts that route with serverless access for unpredictable or spiky demand. The comparison should include idle capacity, model startup and operating control as well as the rate for a busy hour.
The Vercel blueprint provides a comparison for application deployment and model access close to a web-development workflow. The Hugging Face blueprint helps assess model discovery and deployment choices when a particular set of weights is central. Compare model eligibility and the application work still required on each route.
A nontechnical buyer expecting a complete support operation needs more than the ability to create an agent. Escalation, account permissions, source maintenance and human review belong in the surrounding service. DigitalOcean’s platform can supply components, but a team still needs to own what users receive and what happens when answering fails.
03 / WorkflowA proposed documentation assistant begins with retrieval and a bounded source set
Consider a proposed assistant for a software product’s currently supported documentation. Select a small corpus with stable source URLs, product-version metadata and a known owner. Prepare questions whose answers require conditions or exceptions, as well as questions the documentation does not answer. This avoids judging the assistant only on easy definitions.
The agent creation guide requires a model, instructions, project and region, and recommends grounding with a knowledge base. It also says first use of a provider’s model requires accepting the provider’s terms in the control panel. Resolve that step before expecting an automated deployment to work unattended.
Give the agent narrow instructions: answer from the approved documentation, retain version qualifications and provide a useful fallback when evidence is insufficient. Treat these instructions as one layer of behavior, not as access control. An internal version of the assistant must restrict what the retrieval system can supply to the requesting employee.
Build a thin interface that preserves source links and separates a suggested answer from an authoritative configuration instruction. Test whether the source actually supports the displayed answer. A citation beside a sentence is only useful when a user can follow it to the relevant passage rather than to a loosely related documentation home page.
Use the serverless guide to distinguish direct API access from agent calls. If the application already assembles context itself, send that request directly and retain the prompt configuration with its evaluation cases. Do not add a managed agent merely because the model catalog offers one.
If the workload later needs dedicated serving, confirm its region and model compatibility first. The dedicated guide lists ATL1, NYC2 and TOR1, and describes an option to disable the public endpoint in favor of VPC access. A publicly reachable test endpoint should not become the default private-document route through accidental configuration reuse.
Exercise empty prepaid balance, rate limits and unavailable models as ordinary product states. Return a clear status and a documentation link instead of repeatedly retrying a request that cannot succeed. Keep request identifiers and coarse usage metrics sufficient to investigate failures without unnecessarily retaining full user questions.
Evaluate changes to the model, corpus and guardrails separately. A retrieval improvement should be visible in whether the correct passage is supplied; a model improvement should be visible in how faithfully it uses that passage. These proposed checks make the next engineering action more precise than simply declaring the assistant good or bad.
04 / PricingThe bill follows model usage, capacity and added agent features
| Component | Published basis | Planning implication |
|---|---|---|
| Serverless inference | Model-specific consumption; prepaid balance required | Zero balance suspends access |
| Dedicated NVIDIA H100 | $4.41 per GPU-hour | Capacity-based endpoint cost |
| Managed agent creation | No creation charge | Model and additional feature usage still billed |
| Content moderation guardrail | $0.20 per million processed tokens | Inspection is an additional usage line |
Commercial basis from DigitalOcean Inference pricing, consulted 24 September 2026. USD; selected examples, not a complete model tariff.
The pricing documentation says serverless inference requires a positive prepaid balance and suspends access when it reaches zero. Model rates vary, including input, output and other model-specific units. A prepaid balance is an access condition, not an included usage allowance that makes all listed models equally priced.
Dedicated inference is billed by GPU hours. Managed agent creation itself is free, but model usage, knowledge-base processing, storage and guardrails can add charges. The table deliberately separates those units so a team does not budget only the final answer tokens while overlooking the work that prepares and inspects each request.
The same pricing page identifies Inference Router and the Agent Development Kit as public-preview features with specific current charging treatment. Treat that treatment as preview pricing, and confirm changes before relying on it for a long-term operating budget. The platform’s breadth makes careful service selection more valuable than assuming every visible feature is included.
05 / DistinctionsThe platform makes the transition between API and managed capacity explicit
DigitalOcean’s useful distinction is the ability to compare model access and managed capacity within one cloud environment. A developer can begin with direct requests, attach a managed knowledge base where appropriate and later evaluate a dedicated deployment without pretending those routes have identical economics or control.
The model catalog and playground also make selection part of application development. They provide a place to inspect options, but the meaningful comparison still comes from the product’s own questions and constraints. A model that produces an attractive demo answer may mishandle version-specific documentation or generate unnecessarily expensive long responses.
The limits documentation exposes important compatibility boundaries. Imported models are restricted by supported formats, architectures and licenses, and imported BYOM models cannot currently be used for agents. That prevents a common planning error: assuming that importing weights into the catalog automatically makes them available in every higher-level feature.
06 / QuestionsPrivacy depends on the selected model and the surrounding application
The AI data privacy page distinguishes DigitalOcean-hosted models from external providers. It describes provider-specific arrangements and exceptions, including mandatory retention for named Claude Fable models. A general claim that DigitalOcean does not store model inputs or outputs should not be expanded into a guarantee that every upstream provider retains nothing.
A custom application also chooses its own storage behavior. Conversation history, error logs and evaluation datasets can retain user content even when the inference provider does not. Map those stores before using confidential documents, and make the permitted retention part of the product’s operating design.
Check quota and model support against the actual account. The limits page identifies account-scoped quotas, import constraints and feature-specific limitations. If a supported-model change affects a production agent, rerun the relevant evaluations before routing live requests to a replacement. API compatibility establishes a request format, not equivalent behavior.
07 / DecisionChoose a route before choosing the rest of the platform
DigitalOcean is a practical candidate when developers want model access and cloud application services in one place. Begin with either a direct model request or a managed agent for a clearly defined job. Move to dedicated capacity when workload evidence or model requirements justify it, and keep pricing, eligibility and privacy tied to that chosen route.
You already own the application logic
Use direct model requests and keep retrieval and evaluation under your existing design.
You need a managed knowledge assistant
Pilot an agent with a small owned corpus and inspect both citations and complete usage costs.
You need a custom dedicated model
Verify architecture, license, region and feature compatibility before importing weights.
A business worth understanding.
Suggest your business or one you find interesting. Tell us what you want to understand about its product, positioning, design or workflows.
Suggestions are free. Selection and publication stay with the desk.
- DigitalOcean Inference overviewConsulted
- Inference pricingConsulted
- Inference limitsConsulted
- AI data privacyConsulted
- Dedicated inference guideConsulted
- Create agentsConsulted
- Serverless inference guideConsulted
