Training and Evaluation Environments for Healthcare AI Models
Canis Computer Laboratories builds training and evaluation environments where healthcare AI models do real work and are graded on it: eligibility checks, prior authorizations, coding, claim submission, denials and appeals, remittance reconciliation, and the documentation behind all of it. The environments run on Cannect™, the lab’s AI-native EHR, loaded with synthetic patients and real payer rules, and the graders are the billers, coders, and clinicians who do this work every day at CCL Billing. The aim is the one every lab and health system shares: healthcare AI that is measurably capable, safe, and reliable before it reaches a chart or a claim, with the evidence to prove it. Early access is open to frontier labs, applied healthcare AI teams, and health systems evaluating vendor AI.
A Benchmark Asks Questions. An Environment Gives the Model a Job.
In a Canis environment the model is dropped into a working practice system with a patient on the schedule, a plan on file, a payer with rules, and a task with an end state: get the authorization, code the visit, submit the claim, answer the denial. It has the tools a biller has, by API, by agent tool call, or by the screen itself, and it is scored on what actually happened: whether coverage was read correctly, whether the codes were supported, whether the claim would have paid, whether the appeal cited the right policy before the deadline.
That score is the product. For evaluation, it tells a lab or a health system exactly where a model fails on the administrative layer of healthcare. For training, the same score is the reward signal: verifiable outcomes and expert rubric grades a reinforcement-learning pipeline can learn from. Our explainer on what a healthcare AI training environment is covers the vocabulary. This page covers what we build.
The Environment Catalog
Nine environments, each built from a workflow our billing team runs for real practices. Every task has a start state, an end state, and a way to score it.
Eligibility & Benefits
The model reads a 270/271 response and the plan on file, determines coverage for the scheduled service, and computes the patient’s responsibility. Signal: programmatic. The environment knows the coverage answer and the arithmetic.
Prior Authorization
Decide whether the service needs authorization under the payer’s current list, assemble the request with the criteria the reviewer will match against, submit it through the portal or the 278 transaction, and track the clock. Signal: programmatic for required-or-not, completeness, and deadlines; rubric-graded for the medical-necessity language.
Coding & Charge Capture
From a completed note to the ICD-10, CPT, and HCPCS codes, modifiers, and units the documentation supports. Signal: adjudicated gold codes from certified coders. An unsupported or upcoded line is a critical error that fails the task regardless of the rest.
Claim Assembly & Submission
Build the 837 from the encounter, clear the scrubber edits, apply payer-specific rules, and submit. Signal: programmatic. The environment runs the same edits a clearinghouse would and returns the acknowledgment or the rejection.
Denials & Appeals
Read the 835, classify the CARC and RARC codes, choose the action, and write the appeal with the right citations inside the filing window. Signal: programmatic for the classification and the deadline math; billers grade the appeal itself against a rubric.
Remittance & Reconciliation
Post the remittance, compare paid to expected allowed amounts, flag underpayments, and move the balance to the patient or the secondary payer. Signal: programmatic. The ledger either reconciles or it does not.
Clinical Documentation
Complete or correct a note so it supports the care delivered and the codes billed, without adding findings the encounter did not contain. Signal: rubric-graded by clinicians and coders. An invented finding is a critical error.
Front Desk & Payer Correspondence
Scheduling, statements, portal messages, and payer calls. Signal: rubric-graded for accuracy and for the compliance rules that matter most, starting with never disclosing information to the wrong party.
End-to-End EHR Operations
Long-horizon tasks that cross several of the above inside Cannect™, by API, by agent tool calls, or by driving the interface, with the end state checked in the database. Signal: programmatic for the end state; trajectory review for the path.
How Grading Works
Verifiable Outcomes First
Wherever a right answer exists, the environment computes it: coverage, code validity, claim edits, arithmetic, deadlines. These tasks need no grader, and they carry most of the training signal.
Payer Rules and Adjudication
The environment applies the payer rules and adjudication patterns our billing team works with across New York and national payers and returns what would have happened to the claim: accepted, denied with reason codes, or paid at an amount. The model is graded on the outcome, not on whether its output looked plausible.
Expert Rubrics With Critical Errors
Where judgment is required, billers, coders, and clinicians grade against written rubrics. Certain actions are critical errors that fail the task on their own: a missed filing deadline, a fabricated finding, protected information sent to the wrong party. Grader agreement is measured and reported.
Every trajectory is logged: the model, the tools it called, what it saw, what it decided, and the grader’s verdict. That record is what makes a result usable for training, and it is the same decision-record discipline we apply to our own models in production, where Canis AI suggestions are written to a decision record and a human approves every action.
Why a Working Billing Company Builds Them
Real Payer Behavior
CCL Billing works claims for practices across New York and the United States. The rules in the environments are the rules our billers live with, and they are updated when payers change them, which is monthly.
Graders Who Do the Work
The people scoring the model are the people who would otherwise do the task. A model is held to the standard a biller, a coder, or a clinician is held to, and the rubric is written by them.
An Instrumented Substrate
Cannect™ is ours. Every screen, API, table, and audit event is available to the environment, and consent management, break-glass access, and downtime mode are real behaviors a model has to handle, not mock-ups.
Data Posture
No PHI in Any Environment
No real patient enters a training or evaluation environment, and no buyer of an environment ever receives patient data.
Synthetic Patients by Default
Coherent synthetic populations: demographics, plans, histories, and encounters that agree with one another, generated by the lab.
De-Identified Only Under Expert Determination
Where real-world data adds something synthetic data cannot, it is de-identified under HIPAA’s Expert Determination standard with the practice’s written authorization under its BAA.
Nothing Sold
Payer rules and policies are public or contractual documents. Patient data is never sold, licensed, or shipped, in any form. The asset is the environment and the expertise, not the records.
Who It Is For
Frontier and Foundation Model Labs
Reinforcement learning on verifiable healthcare tasks, long-horizon tool use across real systems, and a clear map of where a model fails on the administrative layer of medicine.
Applied Healthcare AI Vendors
Pre-deployment evaluation on realistic tasks with your payers and specialties, regression suites that run with every model update, and evidence you can hand a health system.
Health Systems and Practices
Test a vendor’s AI against your own workflows before you sign, and run shadow-mode comparisons against your staff with a written scorecard.
The Lab Itself
The Canis AI lines train and are evaluated here first. Nothing reaches a real chart before it has done the work in an environment and been graded by the people who do it.
Before, During, and After Deployment
A simulation environment built on real workflows is useful at every stage of a model’s life, not only in training.
Pre-Deployment Evaluation
Held-out task sets show exactly where a model fails on healthcare work, by workflow, by payer, and by error type, in a written report before anyone decides to deploy it.
Deployment Guidance
Shadow-mode design, human approval gates on irreversible actions, and decision records, so the model goes live the way our own Canis AI lines did. The pre-deployment checklist is the playbook.
Post-Deployment Monitoring
Regression runs on the versioned task set after every model update and every payer-rule change, with drift reported against the previous version, so reliability is measured continuously rather than assumed.
Early Access: How an Engagement Runs
We are building the first environments for outside teams now, so the process is deliberately simple and written down at every step.
- Scoping call. Which workflows the model must learn or prove, which payers and specialties matter, and what a passing grade should mean.
- Environment specification. Tasks, tools, data posture, rubrics, and the critical errors, in a document you approve before anything is built.
- Pilot task set. A small set with calibration grading, inter-rater agreement reported, and your model’s first scores.
- Delivery. Hosted environment access, task sets with grading, or evaluation reports, with full trajectory logs in every case.
- Iteration. New tasks as payer rules change, versioned and dated, with drift reported against the previous version.
Frequently Asked Questions
What is a healthcare AI training environment?
A simulated healthcare system, in our case Cannect™ loaded with synthetic patients, real payer rules, and real workflows, in which a model is given a job with an end state: check eligibility, get the authorization, code the visit, submit the claim, answer the denial, reconcile the remittance. The environment scores the result wherever a right answer exists, and billers, coders, and clinicians grade the rest against written rubrics. The same environment evaluates a model on held-out tasks and produces the reward signal that trains the next version.
Is this for evaluation, for reinforcement learning, or both?
Both, from the same environment. Evaluation runs a model on held-out task sets and reports where it fails. Training uses verifiable outcomes and rubric grades as the reward signal for reinforcement learning. We keep evaluation and training task sets separate and versioned so a model is never scored on tasks it has learned from.
Does any protected health information enter the environments?
No. Patients are synthetic by default. Where real-world data adds something synthetic data cannot, it is de-identified under HIPAA’s Expert Determination standard with the practice’s written authorization under its business associate agreement, and it never reaches the buyer of an environment. Payer rules and policies are public or contractual documents. We do not sell patient data in any form.
Which workflows do the environments cover?
The administrative, financial, and documentation layer of healthcare: eligibility and benefits, prior authorization, coding and charge capture, claim assembly and submission, denials and appeals, remittance and reconciliation, clinical documentation that must support the codes, front-desk and payer correspondence, and end-to-end operations inside the EHR. Clinical diagnosis and treatment decisions are outside the program’s scope; where documentation touches clinical judgment, clinicians write and apply the rubrics.
How are environments and results delivered?
During early access the format is agreed per engagement: hosted environments with tool access for the model under test, task sets with grading and full trajectory logs, or written evaluation reports. Every engagement starts with a written environment specification covering tasks, tools, data, rubrics, and critical errors, and a pilot task set with calibration grading before the full build.
Who grades the model's work?
The billers, coders, and clinicians of CCL Billing, who do the same work for real practices every day, grading against rubrics they helped write. Inter-rater agreement is measured and reported. Wherever a right answer exists, coverage, code validity, claim edits, arithmetic, deadlines, adjudication outcome, the environment scores it programmatically and no grader is needed.
How do training environments relate to AI safety and alignment in healthcare?
Alignment in healthcare is concrete: a model should do what a careful biller, coder, or clinician would do, and never do the things that would get a person fired. Environments make that measurable. Critical-error rubrics encode the lines that cannot be crossed, verifiable outcomes show whether the work was done right, and trajectory logs show why a model acted as it did, before deployment and again after every update. The lab's environments are built for that measurement on the administrative, financial, and documentation layer of healthcare; safety evaluation of diagnosis and treatment decisions is a separate discipline we do not claim.
Request Early Access
Tell us which workflow your model needs to learn or prove, which payers matter, and what a passing grade should mean. We will come back with an environment specification and a pilot task set.
See the Models the Environments Produced
The Canis AI lines were trained and graded in these environments before they touched a chart. Watch them work inside the Cannect™ product tour, or read how the lab is building the baseline for Health 2.0.