
A note before the argument: the cloud-native annotation platforms (Labelbox, Scale, V7, Sama, and friends) are good products. The engineering teams behind them ship faster than we do. The interfaces are polished, the integrations are real, and for many ML teams they're the obviously-right choice.

This post is about the labs they don't fit. There are more of them than the SaaS market has noticed.

## The shape of "doesn't fit"

The labs we work with share four constraints. Any one of them complicates a cloud-only labeling tool. Together, they make it a non-starter.

**Data cannot leave the environment.** Patient samples, proprietary imaging assays, IP-sensitive datasets: depending on which one applies, the operative regulation is HIPAA, CLIA, GxP, FDA QSR, or a pharma NDA. The detail varies; the conclusion is the same: the data and its derivatives stay inside.

**Models cannot leave the environment.** Cytogenetics labs that have invested in proprietary scoring models, pharma R&D groups with imaging models trained on internal data, specialty diagnostics shops with hard-won classifiers: none of them want their model weights sitting on a vendor's GPU. The model is the asset; the cloud tool wants to host it.

**The workflow has structure the vendor doesn't model.** Assay-specific fields, range checks, reviewer sign-off chains, cytogenetic LUTs, lot tracking. Generic labeling tools render this as "custom metadata," which works until QC asks you to reproduce the exact reviewer state for a case from nine months ago.

**The buyer is not the engineer.** In the labs we serve, the person making the decision is a Lab Director, Director of Cytogenetics, or VP of Clinical Informatics. They aren't evaluating annotation tools on developer experience. They're evaluating on whether their compliance team will sign off and whether their SMEs will tolerate the change.

A SaaS tool optimized for a Series-B AI startup is solving a different problem.

## What the alternative needs to look like

If you accept the four constraints above, the requirements on the alternative tool are concrete:

- Deploys inside the customer's environment. Not "VPC-tenanted on the vendor cloud", actually inside.
- Supports air-gapped installs, because some labs need that.
- Treats the customer's ML as the primary model, not a fallback to a vendor model.
- Treats the workflow's structure (schemas, sign-offs, lookup tables) as first-class, not as a metadata blob.
- Doesn't ship telemetry, license check-outs, or support backdoors. The audit you sign off on once is the audit that holds.

That set of requirements is approximately what DataTorch is. We didn't invent these criteria; we listened to the labs we wanted to work with and built around what they couldn't have.

## A pragmatic note

If your data and models can sit in someone else's cloud, the cloud-native tools are great and you should use them. We don't compete for that buyer.

If they can't, the on-prem option needs to be just as serious as the cloud option. That's the bar we've been trying to clear.

[Book a call if you want to compare against a specific cloud tool you've already evaluated.](/contact)


Why a generation of polished SaaS labeling platforms doesn't fit regulated life-sciences, and what the on-prem alternative needs to look like.

Cloud annotation tools and the labs that can't use them


In most cytogenetics labs we've talked to, the probe Lookup Table (the mapping a SME uses to interpret what a stain or signal pattern actually means) lives in a spreadsheet. Sometimes a wiki page. Occasionally a printed binder near the microscope.

This isn't a knock on the teams running it that way. The LUT is one of the most carefully-curated assets in the lab. Each row in it represents years of decisions. Of course it lives somewhere safe and edit-controlled.

But it doesn't live where the work happens. And that has a cost.

## The cost of context-switching

Watch a cytogeneticist score for an hour. The pattern is roughly:

1. Pull up the image
2. Recognize the relevant signal
3. Tab over to the LUT spreadsheet
4. Find the matching row
5. Apply the interpretation
6. Record the call in whatever system is logging it
7. Repeat

Steps 3 and 4 are the cost. Per case, it's small. Across a shift, it isn't. And the interpretation step (5) gets noisier the more the SME has to jump out of context to look something up.

Worse: the LUT in the spreadsheet and the LUT the SME is using in their head can drift. A row added last week may or may not be in active use by every reviewer. If two reviewers are scoring the same assay using slightly different LUTs, your QC team finds out about it the hard way.

## What we do differently

DataTorch supports embedding the cytogenetic LUT directly into the scoring workflow. It works roughly like this:

- The LUT lives as a managed asset inside your DataTorch environment. Version-controlled, edit-permissioned, auditable.
- During scoring, the relevant LUT entries appear next to the image the SME is reviewing, not in a separate tab.
- The interpretation choices the SME makes are recorded against the LUT version that was active at the time. Six months later, your QC team can reproduce exactly what they saw.
- When the LUT is updated, the change is logged, and reviewers see the new mappings on the next case (or after the rollout window your lab defines).

It isn't a glamorous feature. It's also one of the highest-leverage things we've shipped for cytogenetics customers, because every minute saved on lookup is a minute spent on judgement.

## What we don't do

A few clarifications, because we get asked.

We don't ship a starter LUT. Yours is yours; we have no opinion about what should be in it.

We don't auto-update from external sources. If your lab pulls reference data from a public registry, that's a workflow you run; we ingest the result.

We don't lock you in. If you decide to take your LUT to another platform, export it as CSV. Done.

## If your scoring workflow has a Lookup Table

The pitch is simple: bring it inside the workflow. The lift is small, the downstream effect on throughput and reproducibility is not.

[Book a call if you want to talk through what this looks like for your assays.](/contact)


Why probe-to-interpretation Lookup Tables shouldn't live in a side spreadsheet. A practical argument from working with cytogenetics teams.

Cytogenetic LUTs belong inside the scoring workflow


We get asked some version of "does air-gapped really work?" on most discovery calls. The honest answer is yes, but the more useful answer is to describe what an air-gapped install actually looks like, because the experience differs from cloud installs in ways IT teams care about.

This post is a sanitized walkthrough of a recent fully-disconnected install. Customer details omitted.

## What "air-gapped" means here

The lab network had no outbound internet access. No traffic could leave the perimeter, including to package mirrors, license servers, or telemetry endpoints. The install needed to complete from a single offline bundle, and the running system needed to operate without any outbound traffic.

We've shipped DataTorch installs across a range of these constraints. This one was on the strict end.

## What we shipped

A single offline bundle (`datatorch-v0.4.x.tar.gz`) containing the application, dependency images, a Postgres bundle, a Redis bundle, and a small CLI to drive the install. Total size around 1.2 GB. The bundle was transferred onto the customer's network via their normal procedure (in this case a vetted USB drop from a clean staging machine), not over the network.

## The install itself

The customer's infrastructure team ran the install on their Kubernetes cluster (RHEL 8 nodes, Kubernetes 1.27). The CLI took a few minutes to:

1. Validate the environment (versions, resource quotas, namespace permissions)
2. Load images from the offline bundle into the cluster's internal registry
3. Start the services in dependency order
4. Run health checks
5. Print the local URL the SMEs would use

We verified one more thing before signing off: no outbound traffic. We watched the network monitor with the customer's security engineer. Nothing left the perimeter: no telemetry, no license phone-home, no support-channel callout. That was the point.

## What's harder air-gapped

A few things we tell customers upfront.

**Updates ship as bundles, not as `apt-get upgrade`.** When we release a new version, we ship you an updated offline bundle, and you run the same install flow. Cadence is yours, not ours.

**Support is via the channel you control.** We can't tail logs on your cluster. If something needs investigating, we work from logs and screen-shares your team initiates. Slower than SSH-in, by design.

**Some integrations are out.** If you wanted a Slack notification on pipeline completion, that's an outbound call. We won't make it. We'll send a webhook your internal notification system can route.

## When air-gapped is the right choice

The customers who choose this configuration are doing it for one of two reasons: a regulatory requirement that data and inference paths stay local, or an IP-driven decision (proprietary models or proprietary assays they don't want a vendor anywhere near). Sometimes both.

If your lab is in that bucket, the install pattern above is what we'd run with you. If you have constraints that make even this approach too open, we'd want to talk through them before recommending anything.

[Book a call to walk through your environment.](/contact)


What actually happens when you install DataTorch on a network with no internet egress. A runbook from a recent customer install.

Notes from an air-gapped deployment


Kromatid is our first regulated-lab customer and the design partner we built the on-prem workflow with. This is a longer version of the case study summary on our landing page, written so you can decide whether the pattern fits your lab.

## The work, briefly

Kromatid runs a directional Genomic Hybridization (dGH) cytogenetic assay. Without diving into the biology: it's a scoring workflow where a subject-matter expert (a cytogeneticist) reviews stained chromosome images, identifies events using a probe Lookup Table, and signs off on a result. The work is image-heavy, the expertise is rare, and the assay output feeds clinical decisions.

The throughput bottleneck is the cytogeneticist. The data is regulated. Both facts shaped what they needed.

## What got deployed

DataTorch runs inside Kromatid's environment. The pieces in the loop:

- **Lab data** stays in their storage, on their hardware.
- **Their in-house ML models** run as Job Agents called from a Pipeline. The pipeline runs against new images and surfaces predictions to the SME review surface.
- **Their cytogenetic LUT** (the probe-to-interpretation mapping their team has been refining for years) drives the scoring workflow. The LUT lives where the SME works.
- **Audit logs** are local, exportable, and shaped for their QC team's review process.

What's not in the loop: us. We don't have credentials into their environment. We don't see their data, their models, or their corrections. The deployment is validated once by their IT and security team, and we stay out of the data path after that.

## What changed for their team

Two things were the visible wins.

The first is throughput. With the SME reviewing predictions instead of starting from raw images, the per-case time dropped meaningfully. We're not going to publish a specific number until Kromatid is ready to put one in writing. But the direction is what matters: their cytogeneticists are doing more high-judgement work per shift.

The second is durability. The model in the loop is theirs, the LUT is theirs, the corrections are theirs. If they swap us out tomorrow, none of the assets they've built leave with us. That mattered to their leadership team during procurement.

## What didn't get easier

A few honest notes.

On-prem deployments take longer than cloud signups. We work through your IT and security review on your timeline, not ours. If your security team needs three weeks, that's three weeks. We can do many things; we can't accelerate your validation process.

Bringing your own model is great if you have a model. If you're still building one, DataTorch is something you grow into, not something that hands you an ML team.

## The lookalike pattern

If your lab looks like Kromatid's shape (SMEs as the throughput bottleneck, data and models that can't leave, an in-house ML effort that's either real or seriously planned), we should talk. The on-prem + Agents-and-Pipelines + workflow-LUT combination is what we built for that pattern.

[Book a 30-min call.](/contact)


Our first design-partner customer. What they deployed, what changed for their SMEs, and the constraints that drove the on-prem architecture.

How Kromatid runs cytogenetic dGH scoring entirely on-prem


Most cloud annotation platforms ship with a built-in model: you upload your data, their model pre-labels it, your reviewers correct, training data comes out. It's a clean story. It also doesn't work if you can't put your data in their cloud, or if your lab already has its own ML stack that took years to validate.

When we started talking to regulated life-sciences labs (cytogenetics groups, pathology teams, specialty diagnostics), we kept hearing the same shape of problem. Their SMEs were the bottleneck on scoring throughput. They had, or wanted, in-house models to take pressure off those SMEs. But every option on the market either required moving data to a vendor cloud, or required throwing out the model they'd built and using the vendor's.

That's not a feature gap. That's an architectural assumption. So we built around the opposite assumption: **your model is the model.**

## Agents and Pipelines, briefly

In DataTorch, a Pipeline is a workflow that runs against incoming data. The interesting unit inside a Pipeline is a Job Agent: a container, service, or runtime that emits structured outputs. You wrap your inference (whatever it is: Python service, Docker container, ONNX runtime) as a Job Agent and call it from a Pipeline. The pipeline runs your inference against new data, predictions land in the SME review surface, corrections feed back as labeled training data.

The agent owns the model. We own the orchestration, the review UI, and the audit trail. The split is deliberate.

## What this means for your lab

A few things flow from the split.

**Your ML team keeps their stack.** No rewriting inference to fit a vendor SDK. If your model runs as a Docker container today, that's the artifact you point a Job Agent at.

**Your data stays inside.** The Pipeline runs inside your environment; your model runs inside your environment; the SMEs review inside your environment. There is no point in the loop where your data has to leave the perimeter for the workflow to function.

**Your corrections compound.** Every SME correction is a labeled training example. You can export them on whatever schedule fits your retraining cycle: weekly, monthly, never. We don't see any of it.

## What it doesn't do

Worth being honest: Agents and Pipelines isn't a magic-model menu. We don't ship pre-trained models for cytogenetic scoring or digital pathology. If you don't have a model and don't want to build one, DataTorch is the wrong tool. The product assumes you've already decided to own your ML, or are actively heading that way.

If that sounds like your lab, we should talk.


Cloud labeling tools assume you'll use a vendor model. Regulated labs can't. Here's the architecture choice we made instead.

Why we built Agents and Pipelines for regulated labs


Welcome to DataTorch 0.4.1! This release improves the annotation import process with a streamlined drag-and-drop experience for COCO imports and official support for Pixel Mask annotations.

## Drag and Drop COCO Import

Importing COCO annotations has just got a lot simpler. We've updated the import modal to support **Drag and Drop**, allowing you to directly upload your COCO JSON files right from the browser.

![Drag and Drop COCO Import](/next/blog/datatorch-0-4-1-coco-drag-drop.png)

Simply open the import dialog, select "COCO" as the format, and drag your file into the release zone. The importer will automatically validate your COCO file structure to ensure compatibility before processing, giving you immediate feedback if there are any formatting issues.

This eliminates the need to use the CLI or Python SDK for quick imports, streamlining your workflow when moving data into DataTorch.

## Pixel Mask Import Support

We have also added support for **Pixel Mask** annotations. You can now import pixel-perfect segmentation masks directly into your projects. This feature is perfect for users working with **binary masks**, **semantic segmentation**, or simply a **folder of images with masks**.

![Pixel Mask Import](/next/blog/datatorch-0-4-1-pixel-mask-import.png)

Whether you are working with complex instance masks or standard binary masks, DataTorch now handles these data types natively. You can select "Pixel Mask" from the import dropdown to view specific instructions for importing your mask data, ensuring that your rigorous segmentation tasks are fully supported.

## Python SDK Updates

Along with these UI improvements, our Python SDK has been updated to support these new import workflows. You can programmatically upload pixel masks and COCO annotations using the latest version of the `datatorch` Python package.

Check out the updated documentation for examples on how to integrate these imports into your automated pipelines.


Drag and drop COCO imports and support for Pixel Mask annotations.

DataTorch Patch 0.4.1 Notes


Welcome to DataTorch 0.4.0, which comes with release of the Pipelines Tab in the annotator, allowing you to see real-time updates on the status of your pipelines run without needing to leave the annotator page.

We've also made some updates to the Python client, adding support for Python 3.12, as well as improvements to allow for smoother installation and programmatic uploading of files.

## Pipelines Tab

![Annotator with Pipelines Tab open](/next/blog/datatorch-0-4-0-annotator.png)

Power users of DataTorch have written pipelines with annotator triggers primarily to run their own models directly on the file being viewed in the browser. However, getting feedback on these pipelines has been a hassle, as users would have to go back into the Pipelines section of their project to see the logs of their actions, breaking workflow and adding complication.

To fix this, we are introducing the Pipelines Tab, a tab directly in the annotator that provides a real time update of each job that is being run in your project, showing the status and logs of each job and step.

![Pipelines Tab close up](/next/blog/datatorch-0-4-0-tab.png)

Now, users can enjoy real time feedback on the progress of their pipeine as soon as they press the trigger in the annotator, and resolve or debug issues in a far more streamlined manner than before. Give it a try with your next annotatorButton triggered pipeline!

## Python Client

We've updated our Python client to support Python version 3.12 and dropped support for Python 3.8. Furthermore, we have expanded the upload functions in the API and CLI tool to make it easier to upload directly to your project programmatically.

Some key changes include:

- Introduction of the upload_to_filesource function to the API, meant to replace the upload_to_default_filesource function, allowing you to specify which storage and dataset you wish to upload files into:

```python
def upload_to_filesource(
        self,
        project: Project,
        file: IO,
        storageId: str = None,
        storageFolderName:str = None,
        dataset: Dataset = None,
        **kwargs
    )
```

- Introduction of the upload option to the CLI, a wrapper around upload_to_filesource, letting you upload files and folders directly via CLI commands
- Example script in the Python repository (in the Examples folder) that demonstrates use of the upload_to_filesource function in a script:

```python
import os
import datatorch as dt

api = dt.api.ApiClient("your-api-key")
proj = api.project("user-name/project-name")
dset = proj.dataset("data-set-name")

folder_to_upload = "uploadme"
upload_to_storage_id = "your-storage-id"

# Get all the file names in the folder
files = [
    f
    for f in os.listdir(folder_to_upload)
    if os.path.isfile(os.path.join(folder_to_upload, f))
]

# Upload files to the selected storage and dataset using their IDs
try:
    for file_name in files:
        file_path = os.path.join(folder_to_upload, file_name)
        with open(file_path, "rb") as file:
            api.upload_to_filesource(
                project=proj,
                file=file,
                storageId=upload_to_storage_id,
                storageFolderName=None,
                dataset=dset,
            )
except Exception as e:
    print(f"Error Uploading: {e}")
```

## Bugfixes & QOL Changes

- Fixed typos on landing page
- Added better feedback for running pipelines in the annotator
- Added handling of uploading files of same name to cloud storages


The Pipelines tab in the annotator has arrived.

DataTorch Patch 0.4.0 Notes


We are thrilled to start the year of 2025 with the launch of our new landing page and blog!

We are thrilled to start the year of 2025 with the launch of our new landing page and blog.

DataTorch Blog

Platform

Solutions

Developers