An empirical study · ReInBench + ReCoF

When to Intervene?

An Empirical Study of Errors and
Self-Recovery in LLM Agents

Anonymous Authors

An error occurs. Should we intervene, or let the agent continue?
We study how agents self-recover and use these findings to inform that decision.

1,259Agent traces

1,403Error events

725Self-recoveries

9.02%Relative Macro-F1 gain

The question behind the study

Understanding self-recovery
before intervening

An agent can correct a local error through its own subsequent reasoning and actions.

Continued execution can restore progress.

LLM agents interact with websites and desktop applications to carry out users’ tasks. When an error occurs, external assistance can redirect execution. Agents can also self-recover without additional guidance.

Intervening immediately may interrupt a self-recovery process that would otherwise succeed. We examine this process to understand when waiting is reasonable and when assistance may be useful.

Self-recovery concerns the resolution of a local error. It does not require an observable recognition point, and it is distinct from completing the whole task.

**Figure 1. Agent self-recovery and error detection.** The agent interacts with its environment while external monitoring can choose between waiting and intervention.

RQ1 · Recovery behavior

How often and how quickly do agents self-recover?

Measure recovery frequency, duration, and variation across Execution LLMs.

↗RQ2 · Subsequent execution

Does self-recovery help later recovery and task completion?

Examine subsequent same-type and different-type errors, and overall task outcomes.

↗RQ3 · Intervention decisions

When an error occurs, should we intervene or wait?

Predict self-recovery using the execution context available at error occurrence.

↗

The benchmark

ReInBench

Error and self-recovery events grounded in agent execution traces.

ReInBench brings together computer-use and Web task executions, spanning Web browsing, document and spreadsheet editing, and workflows across applications. Each error event links its occurrence, observable recognition, and recovery outcome to supporting execution evidence.

Table 1. ReInBench overviewComplete benchmark

Execution LLM	Traces	Errors	Recovered
OSWorld tasks · 358 source tasks
Claude 3.7 Sonnet	304	588	268
Claude Sonnet 4.5	339	343	226
Kimi K2.5	343	271	165
Web tasks · 273 source tasks
GPT-5 / SeeAct	273	201	66
Total	1,259	1,403	725

Task sources: OSWorld-Verified and Online-Mind2Web. ReInBench provides the error-event annotations and recovery boundaries. Self-recovery is measured within each recorded trace.

Error occurrence→Recognition, when observable→Recovery outcome

A local error, followed by self-recovery

During a flight query, the agent types “SEA” into the browser’s address bar rather than the departure field. It then recognizes the problem and reopens the website to resume the query.

Step 3 Error onset→Step 5 Recognition→Step 8 Self-recovery

**Figure 2. A Web self-recovery example.** The flight-query trace shows error onset, continued error, recognition, and local self-recovery. Screenshots show the state after each action.

Empirical findings

How agents self-recover

Recovery behavior provides the starting point for deciding when to intervene.

RQ1

Self-recovery is common, fast, and model-dependent.

Agents self-recover from 54.8% of errors in ReInBench’s OSWorld tasks and 32.8% in its Web tasks. Recovery frequency and the tasks on which errors recover vary across Execution LLMs.

**Figure 4. Self-recovery across Execution LLMs.** Recovery proportions differ across Execution LLMs. The task-overlap regions sum to 100% over shared tasks on which at least one LLM recovers from all its annotated errors.

2 stepsMedian CUA recovery
3 stepsMedian Web recovery

**Figure 5. Self-recovery timing.** End-to-end recovery concentrates in the first few steps. Recognition and correction durations are shown separately within each task group.

Why waiting can matter. Many errors self-recover within a few steps. An error may therefore be resolved before external intervention takes effect, although some recovery durations have long tails.

RQ2 · Beyond the current error

Self-recovery supports
subsequent execution

We examine 613 within-trace error-event pairs and relate local recovery to final task completion.

Experience helps with later errors of the same type.

Later errors self-recover in 76.0% of same-type pairs, compared with 56.8% across all pairs. For same-type pairs with comparable durations, 79.4% of later recoveries take no more steps than the earlier recovery.

**Figure 6. Subsequent error recovery.** Later recovery outcomes and durations are pooled across tasks and Execution LLMs. The two rings compare faster-or-equal and slower later recoveries.

Reusable recovery experience

An earlier self-recovery can provide feedback and corrective actions that are useful when a similar error arises in the same task.

Error type matters

Evidence of help for different-type errors is limited. Different errors may require different information or actions and have different intrinsic recovery durations.

Local self-recovery can support overall task completion.

Successful traces have a higher mean proportion of recovered errors than unsuccessful traces, at 71.9% versus 37.8%. Among error-bearing traces, task completion reaches 60.0% when all annotated errors recover.

**Figure 7. Task completion and self-recovery.** Task completion by trace-level recovery status, and the mean within-trace share of recovered versus not recovered errors.

Local recovery and the whole task. Complete recovery accompanies higher task completion. Partial recovery has a completion rate similar to no recovery, as unresolved errors can still obstruct the remaining execution.

RQ3 · From findings to a decision

Introducing ReCoF

Predicting whether an error will self-recover to inform whether additional intervention is needed.

RQ1 reveals differences in self-recovery capability across Execution LLMs. RQ2 shows that self-recovery experience contains useful patterns. ReCoF combines these patterns with an assessment of the current task and execution context.

Component 01

Recovery Pattern Experts

Learn from historical execution contexts and their recovery outcomes. Text, visual, and joint image–text representations capture patterns in how agents understand and handle errors.

Component 02

Conditional Decision Fusion

Combine expert predictions with the Assessment LLM’s contextual verdict and the Execution LLM’s identity. Conditional interactions adapt the contribution of expert predictions to the current error context.

r = p ⊙ (2a − 1)

Context at error occurrence→Predicted self-recovery · allow continuationPredicted non-recovery · consider assistance

Prediction and ablation results

Can we predict self-recovery?

ReCoF improves self-recovery prediction across three Execution LLMs on ReInBench’s CUA tasks.

We compare ReCoF with five methods that can be used to predict self-recovery. Core ablations test Recovery Pattern Experts and Conditional Interaction.

+9.02%Average relative improvement in Macro-F1
over the strongest baseline for each Execution LLM.

Table 2. Self-recovery prediction and core ablationsValues in %

Self-recovery prediction results
Method / variant	Macro-F1	F1	Precision	Recall	Accuracy
Direct LLM	68.75	62.50	69.77	56.60	70.00
AgentForesight	61.49	47.37	78.26	33.96	66.67
Sentry	61.10	48.78	68.97	37.74	65.00
PrefixGuard	58.59± 3.19	59.40± 1.77	52.96± 3.40	68.30± 6.86	58.83± 3.15
Wink	66.26	59.79	65.91	54.72	67.50
w/o Recovery Pattern Experts	68.75	62.50	69.77	56.60	70.00
w/o Conditional Interaction	71.14± 2.34	68.01± 1.95	69.32± 5.98	68.30± 10.87	71.83± 2.53
ReCoF	73.53± 2.33	71.61± 0.31	69.56± 5.62	74.72± 7.01	73.83± 2.54

ReCoF · 73.53% Macro-F1 · Claude 3.7 Sonnet

Bold red marks the highest mean in each model–metric group. Precision, recall, and F1 treat non-recovery as positive. Macro-F1 averages both class F1 scores. Smaller values show sample standard deviation over five classifier seeds for PrefixGuard and five fusion seeds for ReCoF and its ablations. ReCoF’s experts and Assessment LLM outputs are held fixed. API methods use cached predictions. SD is omitted where predictions are identical across seeds.

Recovery patterns contribute useful evidence.

Removing Recovery Pattern Experts reduces Macro-F1 to 68.75%, 72.88%, and 61.69%. These predictions match the direct contextual verdict.

How the evidence is combined matters.

Removing Conditional Interaction reduces Macro-F1 to 71.14%, 61.50%, and 69.74%. Conditioning expert contributions improves the precision–recall balance.

Stable around the selected configuration

Small threshold offsets of ±0.01 change mean Macro-F1 by at most about 1.43 percentage points. We also examine regularization strength and task-fold count.

**Figure 8. Parameter sensitivity.** Sensitivity to regularization C, number of task folds K, and threshold offset Δ. The dashed lines mark the selected settings.

Self-recovery belongs in the intervention decision.

When an error occurs, predicting whether it will self-recover provides a basis for allowing continued execution or considering external assistance.

Anonymous Code ReInBench