Fiddler AI observes agents and predictive models with runtime guardrails

Fiddler AI combines monitoring, evaluation and runtime guardrails for AI systems. Its scope spans predictive models, language-model applications and agents, making it relevant to enterprises that operate more than one kind of AI. The central distinction is between observing a problem and intervening in a live request. Both can be useful, but they require different evidence and different decisions about what happens when a check fails.

01 / ProductAn operational view across several kinds of AI

The Fiddler platform site presents an AI control plane covering monitoring, evaluations, enforcement and governance. These are related functions, not synonyms. Monitoring identifies patterns, evaluation tests behaviour against criteria, and enforcement can alter or stop a live interaction. A buyer should specify which part of that chain is missing from its current operations.

The ML observability guide describes performance tracking, drift detection, data integrity and segment analysis. Those functions fit predictive systems whose inputs and outcomes can change over time. Drift is an indication that a distribution changed; it is not automatically proof that a model is wrong. Performance assessment still depends on suitable outcomes or other validated measures.

For agents, the agentic observability guide describes hierarchical traces, model interactions, tools and errors. Its OpenTelemetry basis supports integration with application instrumentation. The benefit is being able to inspect the sequence behind a failure. It does not reveal a business rule or execution step that the application never records.

02 / AudienceA fit for enterprises with both builders and operational owners

Fiddler makes particular sense when a platform team supports established predictive models while product teams add generative features. A risk-scoring service, a retrieval assistant and an agent may share deployment owners but need different metrics. A common operational platform can help coordinate responsibility without pretending all three systems should be judged by the same score.

Security and governance teams have a reason to participate when live controls affect requests. They need to know what is checked, where payloads go and who changes a policy. Engineers need enough detail to distinguish a guardrail rejection from an application defect. These roles should agree on intervention behaviour before the first blocked request reaches a user.

The Arize AI blueprint offers a comparison for tracing and evaluation workflows. The Databricks blueprint covers a broader environment for data and AI development. Use those comparisons to locate the required operational boundary: a specialist measurement layer can complement a development platform, but integrating another system is worthwhile only if it creates evidence the existing process lacks.

03 / WorkflowA proposed investigation of a retrieval-assisted service

Consider an internal service assistant that retrieves procedure documents and drafts a response for staff. It sometimes answers from an obsolete procedure and sometimes exposes information from the wrong department. This proposed workflow separates retrieval quality from live disclosure controls. It is an analytical example, not a Fiddler deployment tested by Sequenced.

Begin by defining two different failures. For stale procedures, the question is whether retrieval and generation use the current document. For restricted information, the question is whether the user may receive the material at all. A language-model guardrail should not become a substitute for application authorization. Permission filtering belongs in the document-access path before sensitive content becomes model context.

Record the input, selected document identifiers, retrieval filters, model call and returned answer. Associate them with the application version and the intended user group without logging unnecessary personal information. Compare several incorrect answers with correct answers from the same procedure family. This helps establish whether the bad result originated in source selection, context assembly or final generation.

Create a reference set that contains current and superseded procedure versions. Ask reviewers to identify the required source and any critical exceptions. Then compare a retrieval-filter change on the same inputs. If stale citations disappear while useful answers also disappear, the change has traded one failure for another. Review individual cases as well as aggregate metrics.

The experiments documentation provides the product route for testing and comparing outputs. In the proposed design, keep a baseline execution and the changed execution, with the same evaluation criteria. A deterministic check can validate an allowed document identifier; a domain rubric can assess whether the response preserved a procedural exception. Those checks answer different questions and should remain separately inspectable.

Next evaluate the Guardrails API. The documentation describes safety, faithfulness and PII/PHI detection using task-specific Centor Models in the customer's Fiddler environment. For the assistant, a faithfulness check could flag an unsupported claim, while a sensitive-data check could trigger redaction or a review path. Test the exact configuration before deciding whether an intervention should block or only alert.

Include normal requests likely to resemble prohibited ones. Staff may legitimately discuss an identifier format, quote a suspicious message or ask why a procedure was rejected. A filter that blocks these requests can create an operational burden even when its detection rate looks impressive. Review both missed problems and unnecessary interventions, assigning each a consequence relevant to the service.

Plan failure handling explicitly. If a guardrail service times out, decide whether the assistant should stop, return a limited response or route the request to a person. That decision belongs to the application owner and depends on the data involved. Test the chosen behaviour in a controlled environment; the existence of an endpoint does not establish the application's behaviour when that endpoint is unavailable.

Once a change is released, compare recurring errors and intervention rates with the reference cases. Keep a record of policy and evaluator changes so that a sudden shift in dashboard scores can be traced to a real application change or a measurement change. A lower alert count can mean an improvement, a disabled check or missing telemetry. An operational review must distinguish them.

04 / PricingThe public tariff charges Developer usage by trace

The pricing page, consulted 23 September 2026, lists Free, Developer and Enterprise. Developer displays $0.002 per trace and includes unified observability, custom evaluators, role-based access and SSO in SaaS. Enterprise adds broader guardrail and deployment terms. The Free route is presented through a request-demo action; the page does not establish an unrestricted self-service allowance.

As illustrative arithmetic, 100,000 billable traces at the displayed Developer rate would cost $200 before taxes or additional contracted services. This is not a complete quote. Confirm what constitutes a billable trace, how retries and retention are treated, and whether the proposed evaluator route has additional costs. A conversation, a tool call and a trace should not be assumed to be interchangeable billing units.

The plan table also needs to match deployment expectations. Developer lists SaaS, whereas Enterprise lists SaaS, VPC or on-premises. The Guardrails documentation discusses operating in a Fiddler environment and private deployment. Confirm which combination is available on the intended contract instead of transferring the strongest enterprise description to every plan. This matters both for budget and for how request data travels.

05 / DistinctionsPredictive monitoring and agent controls share an operational context

Fiddler's breadth is a meaningful distinction for an organisation with a mixed AI estate. An established classifier may need outcome-based performance monitoring, while a new assistant needs trace inspection and faithfulness evaluation. Bringing the operational evidence together can help the same team manage incidents without forcing the systems into an identical testing method.

Its Centor approach also gives buyers a specific evaluator architecture to examine: purpose-built models rather than an assumption that every check must call an external general-purpose language model. That is a deployment choice to assess, not proof of a universal cost or accuracy advantage. The relevant comparison uses the checks, volumes and hosting requirements that the application will actually run.

06 / QuestionsIntervention quality matters as much as detection quality

A runtime control's value depends on what it prevents, what it incorrectly interrupts and how the application responds. Measure those outcomes against your own cases. Marketing latency or precision figures do not establish performance for a different payload size, deployment or policy set. This review has not benchmarked Centor Models or verified Fiddler's private-network behaviour.

For predictive models, clarify when usable outcomes become available. A system can observe input drift before the correct label arrives, but the two signals mean different things. For generative systems, clarify which evaluators use external providers and which operate within the chosen environment. Keep these details in the runbook so incident responders understand both the evidence and its limits.

07 / DecisionChoose the operational loop you can support

Fiddler AI deserves a shortlist place when an enterprise needs a coordinated view of predictive and generative systems, especially where selected runtime controls are required. Start with one observable failure and one clearly defined intervention. Compare the resulting evidence with the existing incident process, then choose a deployment and plan that supports that work. The objective is a system whose errors can be explained and handled consistently.