Loading the changelog…
Loading the changelog…
What changed in Larkspur's agents, what it did for customers and revenue, and what's waiting on you.
Each change is judged twice: Evals (did it pass its tests?) and Customers (did it help them and revenue?).
Waiting on you
7
decisions to make
Revenue at risk
$495,000
on launches with a problem left
Won from agent fixes
$310,000
from Email claims intake · $790,000 across all launches
Held before release
2
failing their evals
Decide before it ships.
The change: Read UK broker dates day first
Evals
FixedCustomers
Not live yetRecommended
Approve it, then merge it
Approve and merge it. It passes 98.8% of test cases, including all 24 UK date cases. It's the fix Kestrel Ridge Mutual asked for before their December renewal ($275,000 at risk).
The change descriptions, authors, who proposed each change, statuses, results and next steps are example content. Your decisions are saved for this sandbox session only; Reset sandbox clears them. Changes still in evals are judged by the eval suite; live changes by whether a known problem remains, because the example logs hold too few runs to compare rates. “Outcome” is the verdict on the launch the change belongs to. Version dates are when each version first appears in the agent logs. Post-launch evaluation measures how each launch moved revenue, usage and customer satisfaction.
The change: Don't approve when the deductible is missing
Evals
RegressionCustomers
Not live yetRecommended
Don't ship it
Don't ship. The auto-fix raised the pass rate but now denies valid claims with a $0 deductible. Have Decisioning rewrite the rule by hand.
Can't approve yet: It now fails a test case that used to pass.
The change: Read approval limits written with a comma
Evals
Below thresholdCustomers
Not live yetRecommended
Send it back to Decisioning
Read limits written like "$5k", then run the tests again. It needs 97% to ship.
Can't approve yet: It passes 94.8% of test cases and needs 97%.
Another agent fix won't solve it. Add the change to the roadmap.
The change: Explain claim statuses in plain English
Revenue since it went live
No revenue tied to it yet. Customers are still reporting the problem.
Evals
Partly fixedCustomers
Outcome partly metRecommended
Add “Proactive claim status updates” to the roadmap
Another wording change won't solve this. Customers still call because nobody tells them what happens next. Build proactive status updates.
The change: Hand off after three failed uploads instead of asking again
Revenue since it went live
No revenue tied to it yet. Customers are still reporting the problem.
Evals
Partly fixedCustomers
Outcome partly metRecommended
Add “Voice and chat claim intake” to the roadmap
Another agent fix won't solve this. The person taking over needs a summary of the chat, which is a change to the handoff screen.
Ask the team that owns the agent to fix what's left.
The change: Send failed pushes to a person instead of marking them done
Revenue since it went live
Evals
Partly fixedCustomers
Outcome partly metRecommended
Ask Claims intake to fix what's left
Teach it UK date formats before Kestrel Ridge Mutual renews in December ($275,000 at risk).
The change: New layout model for faxes and low-resolution scans
Revenue since it went live
Evals
Partly fixedCustomers
Outcome partly metRecommended
Ask Document AI to fix what's left
Detect sideways pages and read the letter O as zero. Fernhollow won't move its commercial book ($220,000) until faxes read cleanly.
Written by Dana Okafor, Claims intakewaiting for your sign-off
Tool
Dates from brokers with a UK address are now read day first, so "03/10/2026" becomes 3 October, not 10 March. When a date could be either and the sender's country is unknown, the run asks a person instead of guessing.
Customer outcome Not live yet
No customer has seen it. It ships only once it passes its tests and is signed off.
Evals and logs Fixed
Passes 98.8% of test cases.
Auto-fixProposed by an observability tool, reviewed by Marcus Lindqvist, Decisioningwaiting for your sign-off
Auto-fixProposed by an observability tool, reviewed by Marcus Lindqvist, Decisioningwaiting for your sign-off
Auto-fixProposed by an observability tool, reviewed by Sofia Ruiz, Conversational channels
Auto-fixProposed by an observability tool, reviewed by Aiko Brennan, Document AI
Approve and merge it. It passes 98.8% of test cases, including all 24 UK date cases. It's the fix Kestrel Ridge Mutual asked for before their December renewal ($275,000 at risk).
claims-intake-email@2.10.1-rc1 · Dana Okafor, Claims intake · Claims intake
- const dateOfLoss = parseDate(raw.dateOfLoss, "MM/DD/YYYY");+ const order = sender.country === "GB" ? "DD/MM/YYYY" : "MM/DD/YYYY";+ const dateOfLoss = parseDate(raw.dateOfLoss, order, { ambiguous: "handoff" });
After · 2.10.1-rc1
Test suite of 320 cases
Runs before
TRC-006 · 2.10.0 · Kestrel Ridge Mutual · 3 Oct 2026 · failed
run_e5512ab8 agent=claims-intake-email@2.10.0 account=kestrel-ridge env=prod
11:08:15.004 step extract_fnol -> ok dateOfLoss_raw="03/10/2026"
11:08:15.210 normalize date -> "2026-03-10" (assumed MM/DD; email from UK-based broker)
11:08:18.771 tool_call claimscore.create_claim -> 422 {"detail":"dateOfLoss 2026-03-10 is before policy effective date 2026-09-01"}
11:08:18.772 ERROR push_failed; no retry for 4xx
11:08:19.001 handoff -> queue=manual_entry reason=push_failedRuns after
TRC-031 · 2.10.1-rc1 · Larkspur internal · 15 Oct 2026 · eval passed
EVAL nightly claims-intake-email@2.10.1-rc1 suite=intake_regression_v3 n=320 02:40:00 passed=316 failed=4 (98.8%) threshold=97% -> PASS new cases: uk_broker_dates 24/24 passed (e.g. "03/10/2026" from a UK broker -> 2026-10-03) failures by rule: attachment_over_25mb ............ 4 (unchanged, known limit) release gate passed: 2.10.1-rc1 ready for sign-off
Prompt
The decision prompt now tells the model to send a claim to a person when the policy's deductible is blank, instead of paying the full invoice.
Customer outcome Not live yet
No customer has seen it. It ships only once it passes its tests and is signed off.
Evals and logs Regression
Passes 95.8% of test cases (was 94.8%), but now fails a case that used to pass.
Still wrong
It now also denies a valid claim whose deductible is set to 0 (case_090).
Don't ship. The auto-fix raised the pass rate but now denies valid claims with a $0 deductible. Have Decisioning rewrite the rule by hand.
Can't approve yet: It now fails a test case that used to pass.
auto-decision@0.9.1-rc5 · changed from 0.9.1-rc4 · Marcus Lindqvist, Decisioning · Claims decisioning
Approve only when the estimate is within the account's auto-approve limit.+ If the policy's deductible is missing or blank, do not approve.+ Return decision "refer" with reason "missing_deductible".
Before · 0.9.1-rc4
Test suite of 400 cases
After · 0.9.1-rc5
Test suite of 400 cases
| Failed test cases, by check | 0.9.1-rc4 | 0.9.1-rc5 |
|---|---|---|
| over limit approved | 2 | 2 |
| missing deductible paid full | 11 | 9 |
| excluded peril approved | 6 | 6 |
| wrong line of business | 2 | 0 |
A version needs 97% to ship.
Newly failing: case_090 now denies a valid claim (deductible=0 explicitly set, treated as missing)
Runs before
TRC-010 · 0.9.1-rc4 · Larkspur internal · 4 Oct 2026 · eval failed
EVAL nightly auto-decision@0.9.1-rc4 suite=simple_claims_v4 n=400 02:10:00 passed=379 failed=21 (94.8%) threshold=97% -> FAIL failures by rule: over_limit_approved ............ 2 (limit parsing fixed for "5,000" but not "$5k") missing_deductible_paid_full ... 11 (unchanged) excluded_peril_approved ........ 6 wrong_line_of_business .......... 2 note: llm still returns approve when deductible missing; guard is advisory not blocking ERROR release gate blocked
Runs after
TRC-028 · 0.9.1-rc5 · Larkspur internal · 11 Oct 2026 · eval failed
EVAL nightly auto-decision@0.9.1-rc5 suite=simple_claims_v4 n=400 02:10:00 passed=383 failed=17 (95.8%) threshold=97% -> FAIL failures by rule: missing_deductible_paid_full ... 9 excluded_peril_approved ........ 6 over_limit_approved ............ 2 (case_311 limit="$5k") regression vs rc4: case_090 now denies a valid claim (deductible=0 explicitly set, treated as missing) ERROR release gate blocked; 3rd consecutive nightly fail
Tool
The limit check now reads "5,000" as 5000. Before, it read it as 5, so every claim looked over the limit and the check was ignored.
Customer outcome Not live yet
No customer has seen it. It ships only once it passes its tests and is signed off.
Evals and logs Below threshold
Passes 94.8% of test cases (was 90.3%). It needs 97% to ship.
Still wrong
Limits written as "$5k" are still misread.
Read limits written like "$5k", then run the tests again. It needs 97% to ship.
Can't approve yet: It passes 94.8% of test cases and needs 97%.
auto-decision@0.9.1-rc4 · changed from 0.9.1-rc2 · Marcus Lindqvist, Decisioning · Claims decisioning
- const limit = parseFloat(rules.auto_approve_limit);+ const limit = Number(rules.auto_approve_limit.replace(/,/g, ""));
Before · 0.9.1-rc2
Test suite of 400 cases
After · 0.9.1-rc4
Test suite of 400 cases
| Failed test cases, by check | 0.9.1-rc2 | 0.9.1-rc4 |
|---|---|---|
| over limit approved | 14 | 2 |
| missing deductible paid full | 11 | 11 |
| excluded peril approved | 8 | 6 |
| wrong line of business | 6 | 2 |
A version needs 97% to ship.
Runs before
TRC-009 · 0.9.1-rc2 · Larkspur internal · 27 Sept 2026 · eval failed
EVAL nightly auto-decision@0.9.1-rc2 suite=simple_claims_v4 n=400 02:10:00 passed=361 failed=39 (90.3%) threshold=97% -> FAIL failures by rule: over_limit_approved ............ 14 e.g. case_118 limit="2,500" est=3110 -> approve missing_deductible_paid_full ... 11 e.g. case_207 deductible=null -> payout=full excluded_peril_approved ........ 8 e.g. case_033 cause=flood policy excl flood -> approve wrong_line_of_business .......... 6 ERROR release gate blocked: 0.9.1-rc2 not promoted
Runs after
TRC-010 · 0.9.1-rc4 · Larkspur internal · 4 Oct 2026 · eval failed
EVAL nightly auto-decision@0.9.1-rc4 suite=simple_claims_v4 n=400 02:10:00 passed=379 failed=21 (94.8%) threshold=97% -> FAIL failures by rule: over_limit_approved ............ 2 (limit parsing fixed for "5,000" but not "$5k") missing_deductible_paid_full ... 11 (unchanged) excluded_peril_approved ........ 6 wrong_line_of_business .......... 2 note: llm still returns approve when deductible missing; guard is advisory not blocking ERROR release gate blocked
System prompt
The status assistant's instructions now tell it to describe a status in everyday words instead of showing internal queue names such as PENDING_REVIEW_L2.
Customer outcome Outcome partly met
2 measures improved; 2 complaints or failed runs since launch.
Revenue, usage and satisfaction →Evals and logs Partly fixed
Live, and the main problem is fixed. One part still goes wrong.
Still wrong
"In progress" doesn't say what happens next or who it's waiting on, and the status was served from a cache 19 days old (TRC-029).
Another wording change won't solve this. Customers still call because nobody tells them what happens next. Build proactive status updates.
claim-status@1.3.2 · changed from 1.3.0 · Sofia Ruiz, Conversational channels · Claims servicing
- Show the claim's status field.+ Describe the status in plain English. Never show internal queue names+ such as PENDING_REVIEW_L2.
Before · 1.3.0
Live 1 Sept 2026 to 14 Oct 2026
Running for Tallwater Home & Auto
After · 1.3.2
Live 14 Oct 2026 to 12 Nov 2026
Running for Saltmarsh Casualty
Too few runs in the example logs to compare rates. Read the runs below.
Runs before
TRC-018 · 1.3.0 · Tallwater Home & Auto · 1 Sept 2026 · handoff
run_8c0f4471 agent=claim-status@1.3.0 account=tallwater env=prod
10:15:02.110 tool_call get_claim_status(claim="CLM-TW-61822") -> {status:"PENDING_REVIEW_L2", updated:"2026-08-15", next_action_owner:"CLAIMANT", missing:["repair_estimate"]}
10:15:04.902 render status_card -> "Status: PENDING_REVIEW_L2" (fields next_action_owner, missing not rendered)
10:15:20.441 user: "what does that mean. do i need to do something"
10:15:27.003 llm.turn -> intent=talk_to_agent
10:15:27.010 handoff -> queue=contact_centre reason=user_confusedRuns after
TRC-029 · 1.3.2 · Saltmarsh Casualty · 14 Oct 2026 · handoff
run_2af5067d agent=claim-status@1.3.2 account=saltmarsh env=prod 16:30:11.003 tool_call get_claim_status(claim="CLM-SM-78004") -> served from cache age=19d 16:30:11.004 WARN cache ttl misconfigured (expected 1h) 16:30:12.550 render status_card -> "In progress" 16:30:40.118 user: "its been 3 weeks, is anyone looking at this" 16:30:52.002 handoff -> queue=contact_centre reason=user_requested
Written by Sofia Ruiz, Conversational channels
System prompt
The chat agent's instructions now stop it asking for photos after three failed uploads and pass the customer to a person instead of looping until they leave.
Customer outcome Outcome partly met
2 measures improved; chat CSAT got worse; 3 complaints or failed runs since launch.
Revenue, usage and satisfaction →Evals and logs Partly fixed
Live, and the main problem is fixed. One part still goes wrong.
Still wrong
The person receives the customer with no summary of the conversation (TRC-017). Large photos are still rejected.
Another agent fix won't solve this. The person taking over needs a summary of the chat, which is a change to the handoff screen.
claims-intake-chat@2.2.1 · changed from 2.2.0 · Sofia Ruiz, Conversational channels · Claims intake
- If an upload fails, apologise and ask the customer to try again.+ If an upload fails three times, stop asking. Tell the customer a person+ will help, and hand off to the claims_live queue.
Before · 2.2.0
Live 23 Aug 2026 to 16 Sept 2026
Running for Tallwater Home & Auto
After · 2.2.1
Live 16 Sept 2026 to 12 Nov 2026
Running for Kestrel Ridge Mutual, Tallwater Home & Auto
Written by Dana Okafor, Claims intake
Tool
When the push to ClaimsCore fails, the run now goes to the manual-entry queue instead of being marked complete. Runs also save a checkpoint after each step, so a deploy no longer kills them midway.
Customer outcome Outcome partly met
$310,000 won or kept; 3 measures improved; 1 complaint or failed run since launch; $275,000 still at risk.
Revenue, usage and satisfaction →Evals and logs Partly fixed
Live, and the main problem is fixed. One part still goes wrong.
Still wrong
A UK broker's date "03/10/2026" was read as 10 March, not 3 October, so ClaimsCore rejected the claim (TRC-006).
Teach it UK date formats before Kestrel Ridge Mutual renews in December ($275,000 at risk).
claims-intake-email@2.10.0 · changed from 2.9.2 · Dana Okafor, Claims intake · Claims intake
- if (push.failed) { log.warn("push_failed"); return run.complete(); }+ if (push.failed) return handoff({ queue: "manual_entry", reason: "push_failed" });+ checkpoint: "after_each_step"
Before · 2.9.2
Live 5 Sept 2026 to 3 Oct 2026
Running for Saltmarsh Casualty
After · 2.10.0
Live 3 Oct 2026 to 12 Nov 2026
Running for Kestrel Ridge Mutual, Tallwater Home & Auto
Written by Aiko Brennan, Document AI
AI model
Document extraction moved from layout model v4 to v5, which is tuned on faxed and scanned pages. This replaced the enhancement retries from 4.2.1.
Customer outcome Outcome partly met
3 measures improved; 3 complaints or failed runs since launch; $220,000 still at risk.
Revenue, usage and satisfaction →Evals and logs Partly fixed
Live, and the main problem is fixed. One part still goes wrong.
Still wrong
Sideways pages aren't detected, and faxed totals with the letter O for zero still can't be read.
Detect sideways pages and read the letter O as zero. Fernhollow won't move its commercial book ($220,000) until faxes read cleanly.
doc-extract@4.3.0 · changed from 4.2.1 · Aiko Brennan, Document AI · Document extraction
- layout_model: "v4"+ layout_model: "v5"- on_low_conf: [retry({ enhance: true }), retry({ enhance: true, upscale: 2 })]
Before · 4.2.1
Live 3 Sept 2026 to 1 Oct 2026
Running for Saltmarsh Casualty
After · 4.3.0
Live 1 Oct 2026 to 12 Nov 2026
Running for Fernhollow Insurance Group, Kestrel Ridge Mutual
Written by Dana Okafor, Claims intake
Tool
The ClaimsCore mapping tool now leaves empty fields out of the payload instead of sending them as null, which ClaimsCore rejected. Push retries were cut from 3 to 2.
Customer outcome Outcome partly met
$310,000 won or kept; 3 measures improved; 1 complaint or failed run since launch; $275,000 still at risk. Judged together with the later change, “Send failed pushes to a person instead of marking them done”.
Revenue, usage and satisfaction →Evals and logs Partly fixed
Live, and the main problem is fixed. One part still goes wrong.
Still wrong
Mapping now passes, but the push itself times out at busy times and the run is still marked complete (TRC-004).
Nothing more to do here: 2.10.0 sends pushes that time out to a person.
claims-intake-email@2.9.2 · changed from 2.9.0 · Dana Okafor, Claims intake · Claims intake
- payload[field] = extracted[field] ?? null;+ if (extracted[field] != null) payload[field] = extracted[field];- retry: { attempts: 3 }+ retry: { attempts: 2 }
Before · 2.9.0
Live 5 Aug 2026 to 17 Aug 2026
Running for Larchmere Auto Mutual, Tallwater Home & Auto
After · 2.9.2
Tool
When OCR confidence is low, the extraction tool now retries the page with enhancement, then again with 2x upscaling.
Customer outcome Not measured
Reverted before customers could judge it. The launch is judged on “New layout model for faxes and low-resolution scans”.
Revenue, usage and satisfaction →Evals and logs Reverted
Taken back out in 4.3.0.
Why it was undone
The retries pushed a 14-page scan past the 1,200-second limit, and nothing was saved (TRC-013).
Nothing more to do here: the new layout model in 4.3.0 replaced it.
doc-extract@4.2.1 · changed from 4.2.0 · Aiko Brennan, Document AI · Document extraction · rolled back in 4.3.0
+ if (page.meanConf < 0.6) retry({ enhance: true });+ if (page.meanConf < 0.6) retry({ enhance: true, upscale: 2 });
Before · 4.2.0
Live 18 Aug 2026 to 3 Sept 2026
Running for Fernhollow Insurance Group
After · 4.2.1
Live 3 Sept 2026 to 1 Oct 2026
Running for Saltmarsh Casualty
Too few runs in the example logs to compare rates. Read the runs below.
Runs before
TRC-011 · 4.2.0 · Fernhollow Insurance Group · 18 Aug 2026 · completed low confidence
run_1a6b8c77 agent=doc-extract@4.2.0 account=fernhollow env=prod doc=police_report_fax.tif pages=3 12:58:10.400 preprocess: dpi=98 skew=6.8deg (deskew threshold 10deg -> skipped) 12:58:16.902 ocr page1 mean_conf=0.61 12:58:21.115 field damage_estimate raw="$7,200" conf=0.48 (source region shows "$1,200") 12:58:21.116 field incident_date raw="06/14/2062" conf=0.52 12:58:22.001 WARN 2 fields below 0.6 conf; no human-review route configured -> accepted 12:58:22.010 run status=completed
Runs after
TRC-013 · 4.2.1 · Saltmarsh Casualty · 14 Sept 2026 · failed
run_33d1f620 agent=doc-extract@4.2.1 account=saltmarsh env=prod doc=estimate_scan_multi.pdf pages=14 09:27:02.001 preprocess -> 14 pages, 9 flagged low-quality 09:27:04.330 ocr pass 1 ... 09:33:10.118 ocr low conf -> retry with enhance=true 09:39:51.774 retry 2 with enhance=true, upscale=2x 09:47:05.002 ERROR job timeout after 1200s (limit 1200s) 09:47:05.003 run status=failed; no partial results saved
Too few runs in the example logs to compare rates. Read the runs below.
Runs before
TRC-016 · 2.2.0 · Tallwater Home & Auto · 23 Aug 2026 · abandoned
run_6a91d3e0 agent=claims-intake-chat@2.2.0 account=tallwater env=prod device=mobile_ios 19:28:01.330 step 1/6 collect_policy -> ok 19:29:40.118 step 2/6 collect_incident -> ok 19:30:02.004 step 3/6 request_photos 19:31:15.551 tool_call upload_media(file=IMG_4410.HEIC) -> 415 Unsupported Media Type 19:31:15.600 bot: "Sorry, that upload failed. Please try again." 19:32:48.209 tool_call upload_media(file=IMG_4410.HEIC) -> 415 19:32:48.250 bot: "Sorry, that upload failed. Please try again." 19:34:12.000 session idle 90s -> session.end status=abandoned last_step=request_photos
Runs after
TRC-017 · 2.2.1 · Kestrel Ridge Mutual · 16 Sept 2026 · handoff
run_7bd25e18 agent=claims-intake-chat@2.2.1 account=kestrel-ridge env=prod device=desktop 16:38:10.002 step 3/6 request_photos 16:39:01.443 upload_media -> 413 Payload Too Large (14.2MB, limit 10MB) 16:40:20.871 upload_media -> 413 16:41:30.006 upload_media -> 413 16:41:30.010 max upload attempts reached -> handoff 16:42:55.300 tool_call create_handoff(queue="claims_live", context=null) 16:42:55.301 WARN handoff context empty: summary builder not invoked for handoff_reason=upload_failed
Too few runs in the example logs to compare rates. Read the runs below.
Runs before
TRC-004 · 2.9.2 · Saltmarsh Casualty · 5 Sept 2026 · failed
run_c88d2e70 agent=claims-intake-email@2.9.2 account=saltmarsh env=prod 16:41:28.009 step extract_fnol -> ok 16:41:29.551 step map_to_claimscore -> ok 16:41:29.560 tool_call claimscore.create_claim -> ... 16:41:59.561 ERROR timeout after 30000ms (no response) 16:42:00.100 retry 1/2 -> timeout after 30000ms 16:42:30.700 retry 2/2 -> timeout after 30000ms 16:42:30.701 ERROR push_failed reason=timeout; queued_for_retry=false 16:42:30.702 run status=completed (push failure logged as warning only)
Runs after
TRC-006 · 2.10.0 · Kestrel Ridge Mutual · 3 Oct 2026 · failed
run_e5512ab8 agent=claims-intake-email@2.10.0 account=kestrel-ridge env=prod
11:08:15.004 step extract_fnol -> ok dateOfLoss_raw="03/10/2026"
11:08:15.210 normalize date -> "2026-03-10" (assumed MM/DD; email from UK-based broker)
11:08:18.771 tool_call claimscore.create_claim -> 422 {"detail":"dateOfLoss 2026-03-10 is before policy effective date 2026-09-01"}
11:08:18.772 ERROR push_failed; no retry for 4xx
11:08:19.001 handoff -> queue=manual_entry reason=push_failedTRC-025 · 2.10.0 · Tallwater Home & Auto · 8 Oct 2026 · completed
run_f7c2d34a agent=claims-intake-email@2.10.0 account=tallwater env=prod 22:00:58.004 worker received SIGTERM (scheduled deploy) 22:00:58.010 checkpoint saved at step extract_fnol 22:01:09.880 resumed on new worker from checkpoint 22:01:15.002 run status=completed (resumed=1)
Too few runs in the example logs to compare rates. Read the runs below.
Runs before
TRC-011 · 4.2.0 · Fernhollow Insurance Group · 18 Aug 2026 · completed low confidence
run_1a6b8c77 agent=doc-extract@4.2.0 account=fernhollow env=prod doc=police_report_fax.tif pages=3 12:58:10.400 preprocess: dpi=98 skew=6.8deg (deskew threshold 10deg -> skipped) 12:58:16.902 ocr page1 mean_conf=0.61 12:58:21.115 field damage_estimate raw="$7,200" conf=0.48 (source region shows "$1,200") 12:58:21.116 field incident_date raw="06/14/2062" conf=0.52 12:58:22.001 WARN 2 fields below 0.6 conf; no human-review route configured -> accepted 12:58:22.010 run status=completed
TRC-013 · 4.2.1 · Saltmarsh Casualty · 14 Sept 2026 · failed
run_33d1f620 agent=doc-extract@4.2.1 account=saltmarsh env=prod doc=estimate_scan_multi.pdf pages=14 09:27:02.001 preprocess -> 14 pages, 9 flagged low-quality 09:27:04.330 ocr pass 1 ... 09:33:10.118 ocr low conf -> retry with enhance=true 09:39:51.774 retry 2 with enhance=true, upscale=2x 09:47:05.002 ERROR job timeout after 1200s (limit 1200s) 09:47:05.003 run status=failed; no partial results saved
Runs after
TRC-014 · 4.3.0 · Kestrel Ridge Mutual · 1 Oct 2026 · completed low confidence
run_4e870aa2 agent=doc-extract@4.3.0 account=kestrel-ridge env=prod doc=proof_of_loss_scan.pdf 13:12:30.200 preprocess: page 2 rotation=90deg detected=false 13:12:34.552 ocr page2 mean_conf=0.22 (text sideways) 13:12:36.004 fields from page2: claimed_amount=null signature_date=null 13:12:36.005 WARN required fields missing; marked complete anyway (validation=lenient)
TRC-015 · 4.3.0 · Fernhollow Insurance Group · 9 Oct 2026 · completed low confidence
run_5f0a2c39 agent=doc-extract@4.3.0 account=fernhollow env=prod doc=medical_bill_fax_p1-4.tif 10:03:51.101 preprocess: dpi=96 noise=high (fax header lines) 10:03:57.440 table extraction: 11 line items found, expected 14 (3 rows merged) 10:03:57.998 field total_billed="$4,18O.OO" -> parse error (letter O vs zero) -> null 10:03:58.001 ERROR total_billed unparseable; sum(line_items)=3,412.00 mismatch 10:03:58.010 run status=completed_low_confidence
Live 5 Sept 2026 to 3 Oct 2026
Running for Saltmarsh Casualty
Too few runs in the example logs to compare rates. Read the runs below.
Runs before
TRC-002 · 2.9.0 · Tallwater Home & Auto · 12 Aug 2026 · failed
run_a03f9b12 agent=claims-intake-email@2.9.0 account=tallwater env=prod
14:03:44.218 step extract_fnol -> ok fields=31 empty=[lossLocation.postalCode]
14:03:50.977 step map_to_claimscore schema=claimscore-v7
14:03:51.402 tool_call claimscore.create_claim(payload=…) -> 400 Bad Request
{"error":"validation","detail":"field lossLocation.postalCode: expected string, got null"}
14:03:51.403 retry 1/3 (same payload) -> 400
14:03:52.611 retry 2/3 (same payload) -> 400
14:03:54.020 retry 3/3 (same payload) -> 400
14:03:54.021 ERROR push_failed after 3 retries; claim not created in ClaimsCore
14:03:54.030 step mark_complete -> run status set to 'completed' (BUG? push_failed not propagated)Runs after
TRC-004 · 2.9.2 · Saltmarsh Casualty · 5 Sept 2026 · failed
run_c88d2e70 agent=claims-intake-email@2.9.2 account=saltmarsh env=prod 16:41:28.009 step extract_fnol -> ok 16:41:29.551 step map_to_claimscore -> ok 16:41:29.560 tool_call claimscore.create_claim -> ... 16:41:59.561 ERROR timeout after 30000ms (no response) 16:42:00.100 retry 1/2 -> timeout after 30000ms 16:42:30.700 retry 2/2 -> timeout after 30000ms 16:42:30.701 ERROR push_failed reason=timeout; queued_for_retry=false 16:42:30.702 run status=completed (push failure logged as warning only)