Release 0.1.0: Pipelines, rebuilt

Pipelines is where DataTorch connects people and models, and this release rebuilds it. It is also the first DataTorch release with a number. Releases are numbered from here on.

Try it in three steps

1. Get a runner on the machine that will run the model (Python 3.10 or later). If you have one, upgrade it and restart it:

   pip install -U datatorch
   datatorch runner start

   If you do not, open Runners in your project, press Install runner and follow the steps, then add the runner to the project with Manage runners.

2. Open Pipelines in a test project and press New pipeline. Name it, choose From the annotator, and give the button a title and icon.

3. Press Continue. In Choose an action, open DataTorch library and pick huggingface. The editor puts a Download file step in front of it and passes it the file. On the huggingface step, paste a model id from the Hub into model, for example facebook/detr-resnet-50. This model knows everyday objects such as person, car and bottle, not cells, and it adds those as labels to the project the first time it finds them. Save, open a photo of a street or a room in the annotator (upload one if the project has none), and press your button. The first press installs PyTorch and Transformers on your runner and downloads the model, which takes a few minutes.

Build it in the editor

The editor keeps one yaml document beside the canvas (the YAML button in the toolbar opens it). Edit either and the other follows.

A pipeline starts by hand, on a schedule, or from a button in the annotator. You give the button a title and an icon, and choose what a press sends: the whole file, two points (opposite corners of a box), or four points (the top, bottom, left and right of an object). The Hugging Face action always reads the whole file. Points are for actions that take a region.

Run your own model

A step runs an action: a folder of code with a short yaml file that names its inputs and outputs. It can come from your runner, from the DataTorch library (ready-made actions we maintain), or from any public GitHub repository with an action yaml. Every library action now has a description, and we removed the ones that no longer worked.

Your own actions are local actions, folders on your runner at actions/local/<name>/<tag> inside the runner's folder (~/.config/datatorch/runner on Linux, ~/Library/Application Support/DataTorch/runner on a Mac). Keep private model code here rather than on GitHub. Create one with datatorch action create --local my-model, or in the editor choose On my runner, then Write a new action. It starts as a run.py that gets the image's path and returns a list of shapes in pixel coordinates, each a label name with a box, a polygon or points. Your model goes in between.

The person who created the runner can open and edit the code in the browser while the runner is online. The code stays in that folder, and DataTorch keeps only the action's yaml file.

An action with a requirements.txt gets its own Python environment on the runner, built on the first run and reused after that.

Hugging Face, no code

The huggingface library action runs a Hub model on one image: object detection, image segmentation, or image classification. It downloads the model to your runner and runs it there, so your images are not sent to Hugging Face.

It takes Transformers models and YOLO weights. A repository with a config.json runs through Transformers. One with .pt weights runs through Ultralytics. Set the task input to what the model does. Object detection is the default, so a segmentation or classification model needs it changed. If the task does not match the model, the run stops and says which task to set. The threshold input sets the lowest score kept, 0.5 by default. A private model needs HF_TOKEN in the runner's environment.

Results land on the file

A pipeline can return annotations. When the run completes, DataTorch places them on the file you pressed the button on and marks each one as made by that run. It writes them as the person who pressed the button, so they land only if that person may add annotations to the file.

When you pick a model action in the editor, the run creates any label the model returns that the project does not have yet. In the yaml, that is one line on the output:

outputs:
  annotations:
    value: ${{ jobs.main.outputs.annotations }}
    labels: create

To keep the project's labels fixed, set Labels the project does not have to Refuse the run and name them on the model's step. Then a run whose model returns an unknown label places nothing, and the run's row in the annotator's Pipelines tab names the missing labels.

A classification result (the top label and its score) shows in that run's row in the Pipelines tab. For anything else, add ordinary steps: send a Slack message or an email, or call a URL.

Known limits

• The first run of a model action installs its dependencies, which takes minutes. Later runs reuse them.
• Results land on one file, the one open in the annotator when the button was pressed. Putting a model's results on a whole dataset in one run is not in this release.
• Each press adds its own set of annotations. Pressing again does not replace the last run's boxes, so delete them before you try another model or threshold.
• The Hugging Face action reads every image as 8-bit RGB. A 16-bit TIFF, the format many microscopes write, loses most of its range, and a multi-page TIFF is read from its first page only. For those images, write a local action that scales the image the way your model expects.
• With several runners on a project, a pipeline that uses a local action runs only on the runner whose folder holds that action.

What comes next

We will post each new piece here as it ships. If something does not work the way this page says, or you want a capability sooner, write to datatorch@datatorch.io.