Import DICOM annotations

You can use the Python SDK to import DICOM annotations.

This page shows how to declare different annotation types (as Python dictionaries and NDJSON objects) and demonstrates the import process.

A Python notebook demonstrates these steps and can be run directly with Google CoLab.

Supported annotations

To import annotations in Labelbox, you need to create an annotation payload. This section shows how to declare the payload for each supported annotation type.

Labelbox supports two formats for the annotations payload:

Python annotation types (recommended)
NDJSON

Polyline

polyline_annotation = [
    lb_types.DICOMObjectAnnotation(
        name="line_dicom",
        group_key=lb_types.GroupKey.AXIAL,
        frame=1,
        value=lb_types.Line(points=[
            lb_types.Point(x=10, y=10),
            lb_types.Point(x=200, y=20),
            lb_types.Point(x=250, y=250)
        ]),
        segment_index=0,
        keyframe=True,
    ),
    lb_types.DICOMObjectAnnotation(
        name="line_dicom",
        group_key=lb_types.GroupKey.AXIAL,
        frame=20,
        value=lb_types.Line(points=[
            lb_types.Point(x=10, y=10),
            lb_types.Point(x=200, y=10),
            lb_types.Point(x=300, y=300)
        ]),
        segment_index=1,
        keyframe=True,
    ),    
]

polyline_annotation_ndjson = {
  'name': 'line_dicom',
  'groupKey': 'axial', # should be 'axial', 'sagittal', or 'coronal'
  'segments': [
    {
    'keyframes': [{
        'frame': 1,
        'line': [
            {'x': 10, 'y': 10},
            {'x': 200, 'y': 20},
            {'x': 250, 'y': 250},
        ]
    }]},
    {
    'keyframes' : [{
        'frame': 20,
        'line': [
            {'x': 10, 'y': 10},
            {'x': 200, 'y': 10},
            {'x': 300, 'y': 300},
        ]
    }]}
    ],
}

Segmentation Masks

MaskData is mask data declared as a uint8 array of [H, W, 3]. You can also convert a polygon annotation or a 2D array to MaskData. You can also specify a URL to a cloud-hosted mask (can be hosted on any cloud provider).

Masks cannot be larger than 9,000 in height or 9,000 pixels in width; otherwise, they will not import.

mask_annotation = [
    lb_types.DICOMMaskAnnotation(
    group_key='axial',
    frames=[
        lb_types.MaskFrame(
            index=1,
            instance_uri="https://storage.googleapis.com/labelbox-datasets/dicom-sample-data/sample-mask-1.png"
        ),
        lb_types.MaskFrame(
            index=5,
            instance_uri="https://storage.googleapis.com/labelbox-datasets/dicom-sample-data/sample-mask-1.png"
        )
    ],
    instances=[
        lb_types.MaskInstance(
            color_rgb=(255, 255, 255),
            name="segmentation_mask_dicom"
        )
    ])
]

mask_annotation_ndjson = {
    'groupKey': 'axial',
    'masks': {
        'frames': [{
            'index': 1,
            'instanceURI': "https://storage.googleapis.com/labelbox-datasets/dicom-sample-data/sample-mask-1.png"
        }, {
            'index': 5,
            'instanceURI': "https://storage.googleapis.com/labelbox-datasets/dicom-sample-data/sample-mask-1.png"
        }],
        'instances': [
            {
                'colorRGB': (255, 255, 255),
                'name': 'segmentation_mask_dicom'
            }
        ]
    }
}

Example: Import prelabels or ground truth

The steps for importing annotations as prelabels (for model assisted learning) or as ground truth labels are very similar. Steps 5 and 6 explain these differences in detail, when you create the annotation payload and perform the import.

Before you start

You will need to import these libraries to use the code examples in this section.

import labelbox as lb
import labelbox.types as lb_types
import uuid

Replace API key

API_KEY = ""
client = lb.Client(API_KEY)

Step 1: Import data rows into Catalog

global_key = "sample-dicom-1.dcm"
asset = {
    "row_data": "https://storage.googleapis.com/labelbox-datasets/dicom-sample-data/sample-dicom-1.dcm", 
    "global_key": global_key,
}

dataset = client.create_dataset(name="dicom_demo_dataset")
task = dataset.create_data_rows([asset])
task.wait_till_done()
print("Errors :",task.errors)
print("Failed data rows:" ,task.failed_data_rows)

Step 2: Set up ontology

Your project should have an ontology set up to support your annotations. To ensure schema matches, the tool names and classification instructions should match the name fields in your annotations.

To illustrate, the earlier polyline example set the name field to line_dicom.

We use the same name when defining the tools for the ontology. The same pattern must be followed for other tools and classifications used in the ontology.

ontology_builder = lb.OntologyBuilder(
    tools=[
        lb.Tool(tool=lb.Tool.Type.RASTER_SEGMENTATION, name="segmentation_mask_dicom"),
        lb.Tool(tool=lb.Tool.Type.LINE, name="line_dicom"),
    ]
)

ontology = client.create_ontology("Ontology DICOM Annotations", ontology_builder.asdict(), media_type=lb.MediaType.Dicom)

Step 3: Create labeling project

Connect the ontology to the labeling project

project = client.create_project(name="dicom_project_demo", media_type=lb.MediaType.Dicom)

## connect ontology to your project
project.setup_editor(ontology)

Step 4: Send data rows to project

# Create a batch to send to your MAL project
batch = project.create_batch(
  "first-batch-dicom-demo", # Each batch in a project must have a unique name
  global_keys=[global_key], # a list of data row objects, data row ids or global keys
  priority=5 # priority between 1(Highest) - 5(lowest)
)

print("Batch: ", batch)

Step 5: Create annotations payload

Create the annotations payload using earlier examples .

You can create annotations as Python dictionaries or NDJSON objects. The following examples show each format and show how to compose annotations into Labels attached to the data rows.

The resulting labels and label_ndjson from each approach will include every annotation (created earlier) supported by the respective method.


annotations_list = polyline_annotation + mask_annotation
labels = [
    lb_types.Label(
        data=lb_types.DicomData(global_key=global_key),
        annotations=annotations_list
    )
]

label_ndjson = []

for annotation in [
    polyline_annotation_ndjson, 
    mask_annotation_ndjson
]:      
  annotation.update({
      'dataRow': {
          'globalKey': global_key
      }
  })
  label_ndjson.append(annotation)

Step 6: Upload annotations project as prelabels or ground truth

For both options, pass either the labels or label_ndjson payload as the value for the predictions or labels parameter.

Option A: Upload as prelabels (model-assisted labeling)

upload_job_mal = lb.MALPredictionImport.create_from_objects(
    client = client, 
    project_id = project.uid, 
    name="mal_import_job-" + str(uuid.uuid4()), 
    predictions=labels)

upload_job_mal.wait_until_done();
print("Errors:", upload_job_mal.errors)
print("Status of uploads: ", upload_job_mal.statuses)

Option B: Upload as ground truth

upload_job_label_import = lb.LabelImport.create_from_objects(
    client = client,
    project_id = project.uid, 
    name = "label_import_job-" + str(uuid.uuid4()),
    labels=labels
)

upload_job_label_import.wait_until_done()
print("Errors:", upload_job_label_import.errors)
print("Status of uploads: ", upload_job_label_import.statuses)