hovernet-histo-cin2-pan-v1

Hover-Net Model for Cervical Intraepithelial Neoplasia 2 (CIN2) Panoptic Segmentation

Model

• Panoptic Hover-Net: https://www.sciencedirect.com/science/article/pii/S1361841519301045
• Backbone encoder: pre-trained efficientnet_b5 from pytorch-image-models https://github.com/huggingface/pytorch-image-models

USAGE

1. Install histolytics and albumentations

pip install histolytics
pip install albumentations

2. Load trained model

from histolytics.models.hovernet_panoptic import HoverNetPanoptic

model = HoverNetPanoptic.from_pretrained("cin2_v1_efficientnet_b5")

3. Run inference for one image

from albumentations import Resize, Compose
from histolytics.utils import FileHandler
from histolytics.transforms import MinMaxNormalization

model.set_inference_mode()

# Resize to multiple of 32 of your own choosing
transform = Compose([Resize(1024, 1024), MinMaxNormalization()])

im = FileHandler.read_img(IMG_PATH)
im = transform(image=im)["image"]

prob = model.predict(im)
out = model.post_process(prob)
# out = {"nuc": [(nuc instances (H, W), nuc types (H, W))], "tissue": [tissues (H, W)], "cyto": None}

3.1 Run inference for image batch

import torch
from histolytics.utils import FileHandler

model.set_inference_mode()

# dont use random matrices IRL
batch = torch.rand(8, 3, 1024, 1024)

prob = model.predict(im)
out = model.post_process(prob)
# out = {
#  "nuc": [
#    (nuc instances (H, W), nuc types (H, W)),
#    (nuc instances (H, W), nuc types (H, W)),
#    .
#    .
#    .
#    (nuc instances (H, W), nuc types (H, W))    
#  ],
#  "tissue": [
#    (nuc instances (H, W), nuc types (H, W)),
#    (nuc instances (H, W), nuc types (H, W)),
#    .
#    .
#    .
#    (nuc instances (H, W), nuc types (H, W))    
#  ],
#  "cyto": None,
#}

4. Visualize output

from matplotlib import pyplot as plt
from skimage.color import label2rgb

fig, ax = plt.subplots(1, 4, figsize=(24, 6))
ax[0].imshow(im)
ax[1].imshow(label2rgb(out["nuc"][0][0], bg_label=0)) # inst_map
ax[2].imshow(label2rgb(out["nuc"][0][1], bg_label=0)) # type_map
ax[3].imshow(label2rgb(out["tissue"][0], bg_label=0)) # tissue_map

Dataset Details

Semi-manually annotated CIN2 samples from a (private) cohort of Helsinki University Hospital

Contains:

• 370 varying sized image crops at 20x magnification.
• 168 640 annotated nuclei
• 570 872 983 pixels of annotated tissue region

Dataset classes

nuc_classes = {
    0: "background",
    1: "neoplastic",
    2: "inflammatory",
    3: "connective",
    4: "dead",
    5: "glandular_epithelial",
    6: "squamous_epithelial",
}

tissue_classes = {
    0: "background",
    1: "stroma",
    2: "cin",
    3: "squamous_epithelium",
    4: "glandular_epithelium",
    5: "slime",
    6: "blood",
}

Dataset Class Distribution

Nuclei:

• connective nuclei: 46 222 (~27.3%)
• neoplastic nuclei: 49 493 (~29.4%)
• inflammatory nuclei 27 226 (~16.1%)
• dead nuclei 195 (~0.11%)
• glandular epithelial 14 310 (~8.5%)
• squamous epithelial 31194 (~18.5%)

Tissues:

• stromal tissue: 28.2%
• CIN tissue: 23.4%
• squamous epithelium: 24.7%
• glandular epithelium 7.7%
• slime 6.5%
• blood 2.5%

Model Training Details:

First, the image crops in the training data were tiled into 224x224px patches with a sliding window (stride=32px).

Rest of the training procedures follow this notebook: [link]

Citation

histolytics:

@article{
}

Hover-Net original paper:

@article{Graham2019,
  title = {Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images},
  volume = {58},
  ISSN = {1361-8415},
  url = {http://dx.doi.org/10.1016/j.media.2019.101563},
  DOI = {10.1016/j.media.2019.101563},
  journal = {Medical Image Analysis},
  publisher = {Elsevier BV},
  author = {Graham,  Simon and Vu,  Quoc Dang and Raza,  Shan E Ahmed and Azam,  Ayesha and Tsang,  Yee Wah and Kwak,  Jin Tae and Rajpoot,  Nasir},
  year = {2019},
  month = dec,
  pages = {101563}
}

Licence

These model weights are released under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Additional Terms

While the Apache 2.0 License grants broad permissions, we kindly request that users adhere to the following guidelines: Medical or Clinical Use: This model is not intended for use in medical diagnosis, treatment, or prevention of disease of real patients. It should not be used as a substitute for professional medical advice.