Few-shot hypercolumn-based mitochondria segmentation in cardiac and outer hair cells in focused ion beam-scanning electron microscopy (FIB-SEM) data

Julia Dietlmeier, Kevin McGuinness, Sandra Rugonyi, Teresa Wilson, Alfred Nuttall, Noel E. O'Connor

Research output: Contribution to journalArticle

Abstract

We present a novel AI-based approach to the few-shot automated segmentation of mitochondria in large-scale electron microscopy images. Our framework leverages convolutional features from a pre-trained deep multilayer convolutional neural network, such as VGG-16. We then train a binary gradient boosting classifier on the resulting high-dimensional feature hypercolumns. We extract VGG-16 features from the first four convolutional blocks and apply bilinear upsampling to resize the obtained maps to the input image size. This procedure yields a 2688-dimensional feature hypercolumn for each pixel in a 224 × 224 input image. We then apply L1-regularized logistic regression for supervised active feature selection to reduce dependencies among the features, to reduce overfitting, as well as to speed-up gradient boosting-based training. During inference we block process 1728 × 2022 large microscopy images. Our experiments show that in such a formulation of transfer learning our processing pipeline is able to achieve high-accuracy results on very challenging datasets containing a large number of irregularly shaped mitochondria in cardiac and outer hair cells. Our proposed few-shot training approach gives competitive performance with the state-of-the-art using far less training data.

Original languageEnglish (US)
Pages (from-to)521-528
Number of pages8
JournalPattern Recognition Letters
Volume128
DOIs
StatePublished - Dec 1 2019

Fingerprint

Mitochondria
Focused ion beams
Cells
Scanning electron microscopy
Electron microscopy
Logistics
Feature extraction
Microscopic examination
Multilayers
Classifiers
Pipelines
Pixels
Neural networks
Processing
Experiments

Keywords

  • Cardiac and outer hair cells
  • Deep learning
  • Few-shot learning
  • Gradient boosting
  • Mitochondria segmentation
  • Transfer learning

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Few-shot hypercolumn-based mitochondria segmentation in cardiac and outer hair cells in focused ion beam-scanning electron microscopy (FIB-SEM) data. / Dietlmeier, Julia; McGuinness, Kevin; Rugonyi, Sandra; Wilson, Teresa; Nuttall, Alfred; O'Connor, Noel E.

In: Pattern Recognition Letters, Vol. 128, 01.12.2019, p. 521-528.

Research output: Contribution to journalArticle

@article{234006ad4bff4c55a0a518ea8cf03fc5,
title = "Few-shot hypercolumn-based mitochondria segmentation in cardiac and outer hair cells in focused ion beam-scanning electron microscopy (FIB-SEM) data",
abstract = "We present a novel AI-based approach to the few-shot automated segmentation of mitochondria in large-scale electron microscopy images. Our framework leverages convolutional features from a pre-trained deep multilayer convolutional neural network, such as VGG-16. We then train a binary gradient boosting classifier on the resulting high-dimensional feature hypercolumns. We extract VGG-16 features from the first four convolutional blocks and apply bilinear upsampling to resize the obtained maps to the input image size. This procedure yields a 2688-dimensional feature hypercolumn for each pixel in a 224 × 224 input image. We then apply L1-regularized logistic regression for supervised active feature selection to reduce dependencies among the features, to reduce overfitting, as well as to speed-up gradient boosting-based training. During inference we block process 1728 × 2022 large microscopy images. Our experiments show that in such a formulation of transfer learning our processing pipeline is able to achieve high-accuracy results on very challenging datasets containing a large number of irregularly shaped mitochondria in cardiac and outer hair cells. Our proposed few-shot training approach gives competitive performance with the state-of-the-art using far less training data.",
keywords = "Cardiac and outer hair cells, Deep learning, Few-shot learning, Gradient boosting, Mitochondria segmentation, Transfer learning",
author = "Julia Dietlmeier and Kevin McGuinness and Sandra Rugonyi and Teresa Wilson and Alfred Nuttall and O'Connor, {Noel E.}",
year = "2019",
month = "12",
day = "1",
doi = "10.1016/j.patrec.2019.10.031",
language = "English (US)",
volume = "128",
pages = "521--528",
journal = "Pattern Recognition Letters",
issn = "0167-8655",
publisher = "Elsevier",

}

TY - JOUR

T1 - Few-shot hypercolumn-based mitochondria segmentation in cardiac and outer hair cells in focused ion beam-scanning electron microscopy (FIB-SEM) data

AU - Dietlmeier, Julia

AU - McGuinness, Kevin

AU - Rugonyi, Sandra

AU - Wilson, Teresa

AU - Nuttall, Alfred

AU - O'Connor, Noel E.

PY - 2019/12/1

Y1 - 2019/12/1

N2 - We present a novel AI-based approach to the few-shot automated segmentation of mitochondria in large-scale electron microscopy images. Our framework leverages convolutional features from a pre-trained deep multilayer convolutional neural network, such as VGG-16. We then train a binary gradient boosting classifier on the resulting high-dimensional feature hypercolumns. We extract VGG-16 features from the first four convolutional blocks and apply bilinear upsampling to resize the obtained maps to the input image size. This procedure yields a 2688-dimensional feature hypercolumn for each pixel in a 224 × 224 input image. We then apply L1-regularized logistic regression for supervised active feature selection to reduce dependencies among the features, to reduce overfitting, as well as to speed-up gradient boosting-based training. During inference we block process 1728 × 2022 large microscopy images. Our experiments show that in such a formulation of transfer learning our processing pipeline is able to achieve high-accuracy results on very challenging datasets containing a large number of irregularly shaped mitochondria in cardiac and outer hair cells. Our proposed few-shot training approach gives competitive performance with the state-of-the-art using far less training data.

AB - We present a novel AI-based approach to the few-shot automated segmentation of mitochondria in large-scale electron microscopy images. Our framework leverages convolutional features from a pre-trained deep multilayer convolutional neural network, such as VGG-16. We then train a binary gradient boosting classifier on the resulting high-dimensional feature hypercolumns. We extract VGG-16 features from the first four convolutional blocks and apply bilinear upsampling to resize the obtained maps to the input image size. This procedure yields a 2688-dimensional feature hypercolumn for each pixel in a 224 × 224 input image. We then apply L1-regularized logistic regression for supervised active feature selection to reduce dependencies among the features, to reduce overfitting, as well as to speed-up gradient boosting-based training. During inference we block process 1728 × 2022 large microscopy images. Our experiments show that in such a formulation of transfer learning our processing pipeline is able to achieve high-accuracy results on very challenging datasets containing a large number of irregularly shaped mitochondria in cardiac and outer hair cells. Our proposed few-shot training approach gives competitive performance with the state-of-the-art using far less training data.

KW - Cardiac and outer hair cells

KW - Deep learning

KW - Few-shot learning

KW - Gradient boosting

KW - Mitochondria segmentation

KW - Transfer learning

UR - http://www.scopus.com/inward/record.url?scp=85074452161&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074452161&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2019.10.031

DO - 10.1016/j.patrec.2019.10.031

M3 - Article

AN - SCOPUS:85074452161

VL - 128

SP - 521

EP - 528

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

SN - 0167-8655

ER -