Test AI on a closed QME case before you trust it

By Ahmed Jemaa, Co-Founder and CEO of Medrecords AI. Updated 2026-11-12.

Limits: We read each product's own pages and the DWC sources on 2026-11-12 and did not upload records to any product. Prices, plans and rules change, and nothing here is legal advice; ask your own counsel about Labor Code 4628.

Why a closed QME case is the right test file

Vendor demos run on files the vendor chose. Your closed case has 3 things a demo file lacks: you already know every fact that mattered, your served report is the answer key, and no deadline rides on the result. It also tests the 2 duties that make AI risky for a QME: Labor Code 4628(c), which has you review any excerpts made by someone else, and the DWC checklist's warning not to rely on another source's summary without checking it against the records.

Choose a case with no supplemental report or deposition pending, a file of 500 to 2,000 pages from both parties, at least 1 prior injury, and some duplicate pages. Those are the conditions where tools differ.

Before you upload anything

1. Sign the BAA. A closed file is still PHI. If a vendor will not sign a BAA for a trial, test it on a synthetic file instead.
2. Ask what happens to the file after the trial, and get the deletion answer in writing.
3. Upload only the records that came with a declaration. 8 CCR 9793(n) says the rest were never received, so they should not be in the test either.
4. Use the same upload for both tools: same files, same order.

Build the answer key from your served report

Copy this template and fill it from your report and your notes before you run any tool. 20 facts is enough to separate tools; more is better.

CLOSED-CASE ANSWER KEY (fill before running any tool)
Case label (no names): ____   Specialty: ____   Pages per declarations: ____
Documents listed in my report: ____   Exact duplicate pages I know of: ____

KEY FACTS (fact, date, page in the upload)
1. Date of injury / CT period ............ ____  p. ____
2. Mechanism as first recorded ........... ____  p. ____
3. First treatment visit ................. ____  p. ____
4. Each imaging study and its finding .... ____  p. ____
5. Surgeries or procedures ............... ____  p. ____
6. Work status changes ................... ____  p. ____
7. PTP's P&S date and ratings ........... ____  p. ____
8. Prior QME or AME findings ............. ____  p. ____
9-20. Other facts my opinions rested on .. ____  p. ____

PRIORS (for apportionment)
- Prior injuries, prior awards, pre-existing findings, each with page: ____

RECORDS I NOTED AS MISSING: ____

Score 2 tools on the same file

Enter what you counted for each tool. Facts found means the tool surfaced the fact with the right date. For page cites, check at least 30 cited lines at random and count the ones that point to the wrong page. An invented fact is any statement the record does not support.

Worked example: what the default numbers show

Synthetic example, no real patient, and no real product: the defaults in the sheet are an illustration. An orthopedic panel QME closed a cumulative trauma claim for the right shoulder and neck. The file ran 1,180 pages from both parties with 46 documents and 3 priors: a 2017 shoulder strain, a 2019 award for the left wrist, and degenerative findings on a 2020 cervical X-ray.

• Tool A found 18 of 20 facts and all 3 priors, missed 2 documents in its list, and had 2 wrong page cites in 30. It scores 84.3.
• Tool B found 19 of 20 facts and listed every document with no wrong cites, but missed the 2019 award and invented a "prior lumbar MRI" the file never held. It scores 71.3 and fails on the invented fact.

Tool B looked better on a quick read. For apportionment, the missed award and the invented MRI are the 2 errors that would have reached your opinion, which is why the sheet weights priors and invented facts above speed.

What to do with the result

• Any invented fact: ask the vendor how it happened, rerun, and drop the tool if it repeats.
• Missed priors: check whether the pages were handwritten or faxed. If yes, every future file needs a manual pass on those page types.
• Wrong page cites above 1 in 30: you will spend the time you saved on checking. Compare against the second tool.
• Write down what you checked. If a 4628 question ever comes up, a dated note of how you tested the tool and how you review its output is the record you want.

Questions about testing AI on a QME file