Model releases
How Vardera versions, tests and releases its models, and how enterprise customers control when changes reach their production traffic.
We improve our models continuously and ship each improvement to the API once it has passed our tests. This page explains what a release covers, how releases are tested, how you hear about them, and how enterprise customers can stay on a fixed version.
How versions work
Each category's identification, grading and other models are released together as one version of that category. We version them as a set because they often depend on each other. A version covers the models and the thresholds and settings they run with.
Every API response includes model_version, the version that produced it, such as cards-3f9a1c2. Store it alongside the result, and you can always trace an answer back to the release that gave it.
Reference data updates between versions. New items, like a newly released card set, are added to the catalog continuously, so you can identify them as soon as we have them, and prices update as new sales come in. Significant changes to existing reference data, such as large corrections, ship as data releases.
Release types
| Type | What changes |
|---|---|
| Model release | A retrained or new model, or a change to how our models work together |
| Data release | Significant changes to existing reference data, such as large corrections or cleanups |
| Urgent fix | A correction to an error affecting production results |
How you hear about changes
Releases are listed on the Release Notes page at app.vardera.com, with their date, type and what changed. Release notes say which areas are affected. Routine catalog additions and price updates don't get release notes.
Version availability on the shared API
Requests to the shared API always get the current version of each category. Earlier versions aren't kept available on the shared API. If you need a fixed version in production, see Dedicated enterprise deployments.
How releases are tested
Every release, including data releases, runs against our internal regression sets before it ships. These cover each category's common cases, known hard cases, and errors customers have reported to us.
We compare each candidate against the current version, overall and for each type of case. A gain in the overall score doesn't excuse a drop in an important area. A release that regresses beyond our thresholds doesn't ship until it's fixed and retested.
Enterprise customers can also have a dedicated evaluation set built from their own data, described in the next section.
Dedicated enterprise deployments
Enterprise customers can run on a dedicated deployment. We keep the version you're running in production, and run our latest version alongside it. Your production traffic stays on your version until you choose to switch, apart from routine catalog additions.
We can also build an evaluation set with you from your own images and the cases that matter most to your business. Before you upgrade, we run the new version against it and send you a report. It shows results for each type of case, examples of answers that changed, and any known limitations. You can also test the latest version yourself at any time before switching.
When a new version ships, you have 90 days to switch to it. You don't need to take every release: if another version ships in the meantime, you can go straight to the newest one. After you switch, we keep your previous version available for a short period so we can roll back if something comes up. Specific terms are set in your agreement.