Cloudflare brings model inference, retrieval and application code onto one platform

Cloudflare lets developers combine model inference with the application code, storage and request handling around it. Workers AI supplies the model call, while services such as Vectorize and AI Gateway handle different parts of a complete application. The value is in how those pieces fit a real request. A simple inference endpoint is only the beginning of a reliable customer-facing feature.

01 / ProductWorkers AI executes models while the application owns the task

The Workers AI overview describes serverless model execution on Cloudflare’s network. A developer can invoke supported models from Workers or through the REST API. Cloudflare operates the inference infrastructure, while the application chooses the model, supplies input and handles the response. This removes one infrastructure responsibility without removing product engineering.

Vectorize stores and searches embeddings, which represent content in a form suitable for similarity queries. It can return references to documents or other application objects. The vector result is a candidate source; the application still needs to retrieve the actual content, apply access rules and decide whether it answers the user’s question.

AI Gateway provides visibility and controls around model requests, including logging, caching, rate limiting and fallback. It also works with external providers. A Gateway request therefore does not necessarily mean Workers AI executed the model. Keep the serving provider and the gateway separate in architectural diagrams and cost reports.

02 / AudienceThe strongest fit is an application with a clear request path

A public documentation assistant, product-search feature or internal knowledge endpoint can be a sensible candidate. Each has a recognisable input and a source set the application already owns. Developers who use Workers can add inference near their existing request handling, rather than introducing a separate long-running service just to forward prompts.

This is less suitable for a buyer who needs a ready-made support desk with conversation routing and agent supervision. Cloudflare supplies development primitives. Someone must build the interface, decide what counts as a grounded answer and maintain the content pipeline. The absence of GPU administration does not make the product an automatically maintained business assistant.

The Vercel blueprint offers another application-oriented comparison, particularly when framework deployment and model routing are central. The Pinecone blueprint helps examine a dedicated retrieval layer. These comparisons are most useful when made against the same source corpus and user request, so differences in application design do not masquerade as provider differences.

03 / WorkflowA proposed documentation assistant separates retrieval from generation

Start with a proposed assistant for a software product’s public documentation. Restrict the initial scope to the currently supported version and a small collection of common setup questions. Store the canonical source URL, product version and last revision with every document chunk. This metadata is necessary because a fluent answer from an obsolete setup guide can be more damaging than an explicit failure to answer.

Create embeddings with one selected model and keep its version with the index definition. Add the document identifiers and relevant metadata to Vectorize. When a user asks a question, embed the question with the matching model, retrieve candidate chunks and filter to the supported product version. Avoid mixing embeddings from unrelated models simply because they have the same number of dimensions.

Build the request handler using the Workers binding guide. The documented approach connects a Worker to Workers AI through an AI binding and invokes the chosen model from application code. Keep the model identifier in configuration so a later change can be reviewed with the prompt and evaluation cases that depend on it.

Supply the retrieved passages as evidence and ask for a concise answer with source references. Use a separate route for cases where the retrieved material does not support an answer. For example, a question about an unreleased API should produce a link to the supported reference or a clear limitation, rather than an invented configuration example assembled from a neighbouring feature.

Return the original documentation URLs beside the answer. The application should generate those links from stored metadata, not trust a model to reconstruct them. That distinction makes citations useful: the user can inspect the precise source, while the system avoids turning a plausible-looking URL into a broken or misleading destination.

Add Gateway observability with deliberate logging choices. For a public documentation question, retaining the request may be acceptable; an internal variant could contain credentials or customer details. Configure what the application records before a production rollout. A debugging log should not accidentally become a second, poorly governed copy of the knowledge base or user conversation.

Evaluate retrieval and answering separately. First inspect whether the correct passage appears among candidates. Then judge whether the final answer preserves its conditions and exceptions. Include questions with similar vocabulary but different versions, and questions the documents cannot answer. This proposed evaluation can identify whether a failure belongs to indexing, filtering or generation.

Exercise quota exhaustion, provider errors and stale-index behavior as ordinary product states. A static link to the relevant documentation may be a better fallback than a second model call. If a fallback model is used, rerun the same answer checks: a successful HTTP response establishes availability, not equivalent reasoning or instruction-following.

04 / PricingNeuron billing is only one component of the application bill

The Workers AI pricing page displays model-specific units while billing in neurons. Its daily free allocation is 10,000 neurons, with paid usage above that allocation charged at US $0.011 per 1,000 neurons on Workers Paid. The page also identifies models requiring a paid billing method or prepaid AI Gateway credits; free allocation does not make the entire catalog freely accessible.

Treat neurons as a billing measure rather than a universal request count. A longer prompt, larger answer or different model can change consumption. The documentation assistant also consumes application execution, vector queries and storage. Build an estimate from representative complete requests, including failed attempts and index updates, instead of multiplying an optimistic single prompt by a traffic forecast.

Local development is another easy source of confusion. The binding guide states that Workers AI calls still use the Cloudflare account and can incur usage during local development. Developers should use controlled fixtures and understand which parts of their test run call the live model. A localhost browser address does not imply that inference is happening locally.

05 / DistinctionsA connected platform reduces plumbing but preserves distinct boundaries

Cloudflare’s combination is attractive because the request handler, retrieval references and inference call can sit within one developer platform. The Workers AI product page presents a range of text, image, audio and embedding use cases. That breadth can support several features without requiring the team to operate a separate serving stack for each one.

The practical benefit is easier coordination between application components, not automatic correctness. Version metadata still has to survive ingestion; a revoked document still has to leave retrieval results; a request still needs authentication when the corpus is private. These are visible application responsibilities that should remain understandable even when the infrastructure is managed.

Gateway caching deserves particular care. Reusing a response can reduce repeated inference, but the cache key and invalidation policy must match the source version and permission scope. In the public documentation example, a changed installation command should invalidate a previously correct answer. Otherwise, a cache that improves response time can make a documentation correction appear ineffective.

06 / QuestionsModel access, location and beta features require exact checks

Cloudflare marks particular AI capabilities as beta in the documentation, while Workers AI itself is generally available. Do not transfer the platform’s status to every optional feature. The proposed assistant needs basic inference and retrieval; it does not depend on an experimental function-calling path or an unverified promise about custom model hosting.

The global-network description also does not establish a specific data-residency guarantee for every request and service. A regulated workload should obtain the applicable processing and storage commitments for its selected configuration. The source corpus, prompts, gateway logs and stored embeddings may have different retention and location considerations.

Finally, catalog availability and model behavior can change. Maintain a small set of version-specific questions that must continue to work before changing providers. A routing fallback should improve continuity without silently changing the product’s evidence standard. If the answer cannot be supported, the user should see that limitation rather than a more confident substitute.

07 / DecisionChoose Cloudflare around a complete feature, then measure its parts

Cloudflare is a credible choice for teams that want managed inference as part of an application platform. Begin with one request path and make retrieval, generation, logging and billing visible. The resulting design should be understandable as an ordinary web feature with an additional probabilistic component.

The first decision is whether the complete assistant improves access to the product documentation. Measure useful supported answers, retrieval misses, user corrections and total consumption. A low inference price is useful only when the surrounding application gives users dependable access to the right information.