An agent saying “done” does not establish that an external system reached the requested state. A tool call without an exception may only have queried something, submitted a background job, or acted on the wrong object. LangSmith’s new AI Search helps developers find traces by behavior. The practical question is how to compare a completion claim with evidence after locating a suspicious conversation.

The three-state acceptance example below distinguishes verified completion, verified non-completion, and insufficient evidence. It does not implement a behavior model; it separates judgments that are easily conflated during review. 阅读中文版.

The news is behavior-based trace discovery

LangChain published its AI Search announcement on October 13, 2026, followed by Developer Tech News coverage on October 14. Developers describe a behavior in natural language; structured filters narrow candidates before a decision model checks trajectories against that description.

This asks more than whether a message contains the word completed. It examines a relationship between the request, the agent’s actions, and its account of the result. Similar language can accompany different behavior: genuine completion and merely claiming completion may occur in superficially similar conversations.

Use such discovery to locate samples worth reviewing, then connect them to evidence and regression evaluation. A match is not automatically a factual verdict, and a missing match does not establish that the behavior never occurred.

Separate three meanings of success

Operation-level success means that an operation ended normally under its interface semantics. Task-level success means that the requested target state was achieved. A linguistic success claim is simply a statement such as exported, saved, or submitted. These can coincide, but they can also diverge.

The OpenTelemetry error-recording guidance concerns exceptions, error returns, and the context of instrumented operations. It cannot automatically define every application task’s final postcondition. If an interface successfully accepts an asynchronous request, the application still needs a definition of eventual completion.

For example, a fictional export service returning a job identifier establishes acceptance, not necessarily a generated file with correct content. Conversely, an intermediate tool error followed by a successful retry does not establish overall task failure. Review the complete evidence chain relevant to the current target.

Describe a behavior that a reviewer can verify

“The agent is unreliable” is too broad for consistent judgment. A more reviewable description is: “The conversation explicitly claims export completion, but subsequent readback shows that the target file was not generated.” This specifies a claim, an object, a temporal relationship, and contradictory evidence.

A search for acting on the wrong version needs version identity in its evidence. A search for ignoring a tool error needs the actual tool result. Combining several distinct failure modes in one description makes both results and human labeling harder to interpret.

Use structured conditions to narrow objective scope, such as a deployment version, time interval, or test tag; reserve the language description for behavior. The filtering documentation distinguishes field filters from AI search and explains that AI matching depends on trajectory content. A model cannot supply a missing field merely because a question describes it.

Keep missing evidence separate from verified failure

A completion claim needs an explicit postcondition. The teaching example below requires one verified, post-action receipt matching the task and target. That is this example’s acceptance contract, not a universal requirement that all applications have exactly one receipt.

A receipt confirming non-completion contradicts a completion claim. An absent receipt leaves insufficient evidence. The action may have been omitted, or recording and correlation may have failed. Labeling both situations false completion would mix established contradictions with unproven conclusions.

# Invented structured receipts; this is not an AI Search implementation.
request = {"task": "demo-1", "target": "report-v2"}

def assess(request, receipts):
    matching = [r for r in receipts
                if r.get("task") == request["task"]
                and r.get("target") == request["target"]
                and r.get("verified") is True
                and r.get("after_action") is True]
    if len(matching) != 1:
        return "unknown"
    complete = matching[0].get("complete")
    if complete is True:
        return "completed"
    if complete is False:
        return "not_completed"
    return "unknown"

def review_route(claims_done, state):
    if claims_done and state == "not_completed":
        return "contradiction"
    if claims_done and state == "unknown":
        return "needs_evidence"
    return "no_contradiction_established"

good = dict(task="demo-1", target="report-v2", verified=True,
            after_action=True, complete=True)
bad = dict(good, complete=False)
assert assess(request, [good]) == "completed"
assert assess(request, [bad]) == "not_completed"
assert assess(request, []) == "unknown"
assert assess(request, [dict(good, target="other-report")]) == "unknown"
assert assess(request, [dict(good, after_action=False)]) == "unknown"
assert assess(request, [good, bad]) == "unknown"
assert review_route(True, assess(request, [bad])) == "contradiction"
assert review_route(True, assess(request, [])) == "needs_evidence"
print("verified complete, verified incomplete, and missing evidence stay distinct")

The code was executed locally. It covers completion, confirmed non-completion, a missing receipt, the wrong target, a stale receipt, and conflicting receipts. All data is fictional. No real logs were read, no LangSmith call was made, and AI Search classification accuracy was not tested.

Real systems need a task-specific contract. Multiple targets may require several receipts; eventually consistent state may need time; asynchronous jobs may have their own lifecycle. Returning unknown for multiple receipts here forces conflict resolution rather than prescribing the correct production rule for every workflow.

Who sets “verified” matters more than the flag

The example’s verified=True is an input assumption used to demonstrate control flow, not evidence by itself. In a real system, an agent must not turn its own completion sentence into that flag and use it as proof. Verification should come from a trusted acceptance path, such as reading actual target state or checking the delivered artifact against the requirement.

Define that requirement before inspecting the outcome. If a task asks for an export containing particular fields, file existence alone may be insufficient. Format, version, or required content can also matter. The business requirement determines the check; do not shrink it after the fact to whichever condition is easiest to pass.

Preserve the acceptance basis and the steps that produced it so another maintainer can review the conclusion. If the evidence source is unverified, retain unknown. Naming a field verified does not establish trust in its value.

A long trajectory can lose decisive evidence

The official announcement describes an 8K-token budget for each prepared trajectory, retaining content by priority. A crucial tool result can be omitted, limiting some behavior judgments. This is the product’s documented processing budget, not a locally measured model capacity.

Consider a fictional session where an agent claims completion, a tool later reports failure, and readback confirms that the target was unchanged. If a checker receives only the first statement, evidence selection becomes part of the problem. A stronger reasoning model cannot reliably inspect a result that was never supplied.

Return to the original trajectory when reviewing a candidate. Check whether the claim and tool outcome are correlated, whether content was truncated, and whether both refer to the same task. Acceptance evidence should not be rewritten as completion evidence. If collection itself is incomplete, record that gap rather than filling it with an inference.

Search matches are not a failure-rate estimate

Behavior search is useful for discovery, but its returned conversations are not automatically a random sample. Filtering to errored traces before searching for contradictory completion claims produces candidates within that filtered set, not a failure proportion for all production requests.

To evaluate detection, prepare an independently reviewed sample with a fixed behavior definition. Inspect false positives among matches and false negatives among non-matches. Reviewing only returned results cannot establish how much was missed. This article invents neither production statistics nor accuracy claims from candidate counts.

After identifying a representative problem, add an authorized, sanitized example to a regression set and check whether a change fixes that failure mode. Include the task object and supporting evidence in the test conditions. A more cautious final sentence alone does not prove that execution was corrected.

Debugging data needs its own boundaries

Traces can contain inputs, outputs, and tool arguments. Finding an exposure later does not replace minimization before collection. Decide what to retain, who may access it, and for how long before adopting detailed tracing. Debugging convenience does not justify uploading every raw interaction by default.

The current enablement documentation requires administrative authorization for processing and describes United States processing and data-residency implications. The interface’s hosting region alone does not establish where all processing occurs. No feature was enabled, user data uploaded, or account setting changed for this article.

Use fictional trajectories and non-sensitive task identifiers for local experiments. If real samples are later required, establish authorization and sanitization first, then retain behavior labels and necessary evidence. Do not collect additional private material merely to make an example easier for a model to interpret.

Derive the completion statement from acceptance state

A durable fix is to give the agent an explainable target-state result before it claims completion. Record interface acceptance, action results, and readback separately, correlated to the same task and object. When that chain is unavailable, report the current stage accurately instead of supplying a confident completion sentence.

For retries, also prevent duplicate actions and confusion with old results. The site’s idempotent agent tool retry article covers repeated requests. Idempotency handles repetition, acceptance state establishes the result, and behavior search discovers missed patterns. They answer different operational questions.

The useful opportunity in this news is discovering failures that resist a single fixed field. Maintainers still need to turn candidates into evidence, evidence into a concrete repair, and the repair into a check against the same contract before concluding what the agent actually accomplished.

Sources and verification scope

LangChain announcement, October 13, 2026; Developer Tech News coverage, October 14; LangSmith filtering and AI search documentation; OpenTelemetry error-recording guidance. The Google News item was timestamped October 14 at 09:14:17 UTC, before the planned slot. Graphics, the acceptance contract, and code are original teaching examples.