Discover our new platform: Learn more

A custom Cellpose model for H&E images

BioTuring AI & Algorithm team
BioTuring AI & Algorithm team
December 9, 2024

TLDR: We released a specialized model of TuringSegment for H&E images. Our model outperforms the original Cellpose model by a large margin, achieving approximately 40 percentage points higher AP50 on the Panuke and 48 for Nuinsseg datasets, and 43 percentage points higher on the Monuseg dataset.

The model has been integrated into our cell segmentation tool, TuringSegment, and is freely available on our GitHub repository.

Benchmark

All benchmarks are conducted in the test sets of Panuke, NuinsSeg and Monuseg data sets.

The figure above is the Average Precision at threshold 50% (AP50) comparison between our specialized model (TuringSegment) and the original Cellpose model (Cellpose) for each test dataset. Our model outperformed the original Cellpose model by a large margin, achieving approximately 40 percentage points higher AP50 on the Panuke and 48 for Nuinsseg datasets, and 43 percentage points higher on the Monuseg dataset.

Additionally, we assessed the accuracy across varying IoU thresholds using a combined dataset from all three test sets. Our model consistently outperformed the original Cellpose at every threshold, demonstrating better performance in both cell detection and segmentation quality.

Qualitative evaluation on whole-slice images

In this section, we highlight some differences between our model and the original model on real whole-slice images.

CellposeTuring Segment


The cellpose model cannot detect cells in dense regions. Furthermore, the detected nucleus are generally larger than their true size. On the other hand, our model is able to catch more cells accurately and the segmentation shapes fit better to the detected nucleus
CellposeTuring Segment
Our model correctly ignore red blood cells
CellposeTuring Segment
The Cellpose model doesn’t work with hard conditions such as dark regions. As show in the above figures, our model is much better at detecting cells in dark region with unclear cell boundary.

Conclusion

Our specialized model delivers a substantial improvement over Cellpose in detecting cells within H&E-stained images. The model is also integrated into Turing Segment‘s high-performance detection pipeline, providing researchers with more accurate segmentation results and significantly faster processing.

Visit our Turing Segment repository to learn how to use the model.

UPDATE: 02/01/2025

We have released version 2 of our H&E model, which demonstrates improved performance on challenging cases.

Below is the benchmarks of the new model on Panuke, NuinsSeg and Monuseg data sets. Although the accuracy is slightly lower compared to the first version, the model achieves robust performance on many difficult/out-of-distribution cases.

Qualitative evaluations are provided below:

CellposeTuring SegmentTuring Segment V2

Turing Segment V2 is capable of detecting and segmenting regions with blurred boundaries.

CellposeTuring SegmentTuring Segment V2

Turing Segment V2 performs effectively in low-contrast regions and better localize the nucleus compared to Cellpose.

CellposeTuring SegmentTuring Segment V2

The model clearly outperforms Cellpose and the V1 model on cells with weak signals

BioTuring AI & Algorithm team

0 comments