Laminar makes agent traces searchable for recurring failures

Laminar helps developers inspect agent executions and identify recurring patterns across them. Traces provide the underlying record; evaluations compare known cases; Signals use a model to look for defined behaviours in recorded runs. The opportunity is to turn a difficult collection of agent conversations into evidence a team can act on. The decision also involves a consequential data choice: Laminar’s documented privacy-mode defaults differ by plan, particularly for the material used in Signals.

01 / ProductExecution records support several different questions

The current product site presents tracing, evaluation and Signals as related parts of agent development. These serve different purposes. A trace explains a particular run; an evaluation compares behaviour against a known requirement; a Signal searches for a defined pattern in traces. Keeping those purposes distinct helps prevent an automatically generated observation from becoming an unexamined quality verdict.

The trace-viewing documentation describes transcript, tree and timeline views. A transcript helps follow the conversation, a tree exposes nested execution, and a timeline makes the order and duration of operations visible. The right view depends on the question. A slow response and an unsupported claim may arise in the same run but require different evidence to diagnose.

Signals combine trace data with instructions and an output schema to identify specified behaviours. They can attach structured events and supporting evidence, and related events can be grouped into clusters. This can help a team find a pattern that is hard to spot manually. It remains model-based analysis, so the team should examine the cited trace evidence before treating an event as an established defect.

02 / AudienceUseful when agent failures are repetitive but hard to find

Laminar is a plausible fit for a developer team with multi-step agents and enough execution history to make manual inspection difficult. A research assistant may repeatedly cite a source that does not support its conclusion; a coding agent may repeatedly abandon a tool failure. These are proposed failure categories. The platform’s value depends on whether the team can define them clearly enough to recognise a correct detection and a false alarm.

Compare Braintrust when evaluation workflows are the centre of the requirement, and LangWatch for another view of agent observability and evaluation. Consider how each product connects a finding to the original execution and to a maintained regression case. Finding more anomalies is only useful if the team can decide what to fix and verify the result.

A team with very little representative traffic may get more value from a carefully designed evaluation set before investing in broad pattern analysis. Conversely, a team with abundant traces but unclear product requirements can spend time clustering behaviour without deciding whether it is wrong. Begin with one failure type and a few examples the team already understands.

03 / WorkflowProposed workflow: find unsupported claims in a research agent

This proposed workflow starts with a research agent that retrieves documents and produces a cited brief. Select a small authorised sample of runs and confirm that the traces include the relevant retrieval outputs, tool errors and final answer. If the record omits the source material, a later analysis may be unable to distinguish unsupported reasoning from evidence that simply was not captured.

Inspect known cases manually in the transcript and tree views. Define the failure narrowly: for example, a final claim presented as established fact when the retrieved passage expresses uncertainty. Include a counterexample where the agent accurately reports that uncertainty. This gives reviewers a concrete standard for judging the Signal rather than allowing its interpretation to define the requirement after the fact.

Configure a Signal with a specific instruction and structured fields for the claim, supporting trace reference and reason for concern. Use filters and sampling appropriate to the selected workload. Review both flagged and unflagged cases, because a seemingly precise list of findings can still miss important failures. Treat the Signal as a triage aid whose output needs calibration, especially when the underlying agent or document collection changes.

Move confirmed examples into a repeatable evaluation. The evaluation documentation separates the input set, executor and evaluators, supporting comparisons across changes. Score factual support separately from writing style and task completion. A shorter answer that avoids every substantive claim should not automatically count as a better research result merely because it contains fewer unsupported statements.

After changing retrieval or instructions, rerun the same examples and inspect any regression. Then observe a bounded new sample of live behaviour to see whether the failure category has changed. This is an evaluation design, not a measured outcome from this article. Its purpose is to connect an observed problem to a reproducible test and a specific change the team can explain.

04 / PricingIngestion volume and Signal analysis are separate meters

The pricing page measures hosted trace ingestion in gigabytes and lists separate Signal analysis charges. The monthly figures below are in US dollars. Included data access windows also differ by plan, which affects how long a developer can inspect an older run through the service.

The Signals documentation says new workspaces receive a small trial credit, while the public Free plan table says Signals are not included. Do not treat trial-credit wording as a recurring free-plan entitlement. Confirm the account’s actual access before designing a process around it. Starter explicitly lists a one-time Signal credit, which likewise should not be mistaken for an ongoing monthly allowance.

The usage-limit guide describes warning and hard limits for ingestion and Signal spending on eligible paid plans. A hard cap can stop new data or analysis. Set limits with an operational response: someone should know whether a missing finding means the agent behaved correctly or the relevant analysis stopped because the budget was exhausted.

05 / DistinctionsSignals can make a trace collection easier to investigate

The useful distinction is the connection between structured pattern detection and the execution evidence behind it. A Signal can organise a recurring behaviour into fields that a team can filter and review. That is more actionable than a generic summary of an agent’s conversation, provided the fields correspond to a real engineering question and the cited evidence survives manual inspection.

Clustering can help group related findings, but a cluster is not automatically a single root cause. Similar final answers may result from different retrieval failures, tool restrictions or instructions. Use the grouping to prioritise investigation, then return to individual traces before choosing a fix. Otherwise an appealing high-level explanation can hide several distinct problems that need different changes.

The open-source route gives teams another deployment option, while the hosted service offers managed operation. The data-use policy says self-hosted instances do not send data to Laminar. That statement should be assessed alongside the chosen configuration and any external model providers the team connects. Operating the software yourself also means owning upgrades, storage and the reliability of the tracing pipeline.

06 / QuestionsPrivacy defaults materially change the evaluation

The data-use policy states that privacy mode is off by default for Free and Starter and on by default for Pro and above. When it is off, Signal runs may be used to train Laminar’s models after redaction; the policy distinguishes these runs from other traces, datasets and recordings. This is a material setting to review before submitting work, not an obscure preference to revisit after a pilot has accumulated sensitive examples.

The same policy describes the effect of switching privacy mode on and the limits of reversing prior training. Review the current setting and applicable agreement before using Signals. An organisation should make an explicit decision about the data included in the analysis and the permitted use of that data, rather than assuming every hosted observability plan has identical defaults.

The PII-redaction guide documents Pro and Enterprise modes, including replacement before storage and a dual mode with restricted access to originals. Its scope is span inputs and outputs; metadata, Signal events, datasets and recordings are excluded. Changes apply prospectively. Therefore, enabling the feature does not establish that every stored field is redacted or that historical data has been retroactively cleaned.

This research did not test redaction accuracy, access enforcement or Signal detection quality. Use representative authorised examples to verify the specific controls and fields that matter to the team. The privacy choice, redaction scope and model-based analysis quality should be assessed together, because each affects what data can be collected and how confidently the resulting findings can be used.

07 / DecisionChoose it with a defined failure and a deliberate data policy

Laminar deserves a trial when detailed agent traces exist but recurring failure patterns remain difficult to find and turn into tests. Start with one known behaviour, check the evidence behind automated findings and preserve confirmed cases in an evaluation. Resolve privacy mode and redaction scope before uploading material, then price both ingestion and analysis around the actual workload.