Accelerated Pixel and Object Classification (APOC)

clesperanto meets scikit-learn to classify pixels, objects and edges in images, on a GPU using OpenCL.

TL;DR: Graphical abstract (source)

This repository contains the backend for Python developers. User-friendly plugins for Fiji and napari can be found here:

• napari-accelerated-pixel-and-object-classification
• clijx-accelerated-pixel-and-object-classification

For training classifiers from pairs of image and label-mask folders, please see this notebook.

Object segmentation

With a given blobs image and a corresponding annotation...

import apoc
from skimage.io import imread, imshow
import pyclesperanto_prototype as cle

image = imread('blobs.tif')
imshow(image)

manual_annotations = imread('annotations.tif')
imshow(manual_annotations, vmin=0, vmax=3)

... objects can be segmented (see full example):

# define features: original image, a blurred version and an edge image
features = apoc.PredefinedFeatureSet.medium_quick.value

# Training
clf = apoc.ObjectSegmenter(opencl_filename='object_segmenter.cl', positive_class_identifier=2)
clf.train(features, manual_annotations, image)

# Prediction
segmentation_result = clf.predict(image=image)
cle.imshow(segmentation_result, labels=True)

Object classification

With a given annotation, blobs can also be classified according to their shape (see full example).

features = 'area,mean_max_distance_to_centroid_ratio,standard_deviation_intensity'

# Create an object classifier
classifier = apoc.ObjectClassifier("object_classifier.cl")

# Training
classifier.train(features, segmentation_result, annotation, image)

# Prediction / determine object classification
classification_result = classifier.predict(segmentation_result, image)

cle.imshow(classification_result, labels=True)

Object selector

If the desired analysis goal is to select objects of a specific class, the object selector can be used (see full example).

features = 'area,mean_max_distance_to_centroid_ratio,standard_deviation_intensity'

cl_filename = "object_selector.cl"

# Create an object classifier
apoc.erase_classifier(cl_filename) # delete it if it was existing before
classifier = apoc.ObjectSelector(cl_filename, positive_class_identifier=1)

# train it
classifier.train(features, labels, annotation, image)

result = classifier.predict(labels, image)
cle.imshow(result, labels=True)

Object merger

APOC also comes with a ObjectMerger allowing to train a classifier on label edges for deciding to merge them or to keep them. (See full example)

feature_definition = "touch_portion mean_touch_intensity"

classifier_filename = "label_merger.cl"

apoc.erase_classifier(classifier_filename)
classifier = apoc.ObjectMerger(opencl_filename=classifier_filename)

classifier.train(features=feature_definition,
                 labels=oversegmented,
                 sparse_annotation=annotation,
                 image=background_subtracted) 

merged_labels = classifier.predict(labels=oversegmented, image=background_subtracted)
cle.imshow(merged_labels, labels=True)

More detailed examples

More example notebooks are available in this folder.

Installation

You can install apoc using conda or pip:

conda install -c conda-forge apoc-backend

OR:

conda install pyopencl
pip install apoc

Mac-users please also install this:

conda install -c conda-forge ocl_icd_wrapper_apple

Linux users please also install this:

conda install -c conda-forge ocl-icd-system

Contributing

Contributions are very welcome. Tests can be run with pytest, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the BSD-3 license, "apoc" is free and open source software

Issues

If you encounter any problems, please open a thread on image.sc along with a detailed description and tag @haesleinhuepf.