Software (developed at University at Buffalo)

  • image

    HistoCloud: A user-friendly tool for cloud-based WSI segmentation

    We made available for the community our cloud-based instance - HistoCloud for digital pahology WSI visualization, automated segmentation, and error correction for training a semantic segmentation model.  We made available the source code, docker image, and a video overview for the community.

    .
    image

    The user interface of HistoCloud. 

    Selected Reference

    • B. Lutnick, D. Manthey, J. U. Becker, B. Ginley, K. Moos, J. E. Zuckerman, L. Rodrigues, A. J. Gallan, L. Barisoni, C. E. Alpers, X. X. Wang, K. Myakala, B. A. Jones, M. Levi, J. B. Kopp, T. Yoshida, S. S. Han, S. Jain, A. Z. Rosenberg, K. Y. Jen, and P. Sarder, for the Kidney Precision Medicine Project, "A user-friendly tool for cloud-based whole slide image segmentation, with examples from renal histopathology." In revision for Communications Medicine - Nature.
  • image

    PodoSighter: Label-free podocyte detection in renal tissue WSIs

    PodoSighter is a cloud-based tool for automated, label-free podocyte detection and three-dimensional quantification from periodic acid-Schiff-stained histologic sections.  The cloud based instance is available via our HistoCloud portal.  All the codes, WSIs, trained models, video tutorial, as well as a docker image to set a user friendly cloud instance of the tool are made available for the community.

    .
    image

    Podosighter plugin on the cloud. 

    Selected Reference

    • D. Govind, J. U. Becker, J. Miecznikowski, A. Rosenberg, J. Dang, P. L. Tharaux, R. Yacoub, F. Thaiss, P. F. Hoyer, D. Manthey, B. Lutnick, A. M. Worral, I. Mohammad, V. Walavalkar, J. E. Tomaszewski, K. Y. Jen, and P. Sarder, "PodoSighter: A cloud-based tool for label-free podocyte detection in kidney whole slide images," Journal of the American Society of Nephrology, vol. 32, no. 11, pp. 2795-2813, Nov. 2021.
  • image

    HistoFetch: On-the-fly processing of gigapixel whole slide images

    HistoFetch extracts random patches and labels from pathology whole slide images (WSIs) for input to a neural network on-the-fly.  To demonstrate the usefulness of HistoFetch in training downstream deep learning tasks, we modified two popular generative adversarial networks (GAN) architectures, enabling direct implementation on WSIs.  HistoFetch and modified GAN codes with trained models are available at: Link 1, Link 2, and Link 3. ProGAN generated in silico images are available here.

    .
    image

    Schematic overview of histo-fetch and comparison with the traditional method. 

    Selected Reference

    • B.  Lutnick, L.  K.  Murali, B.  Ginley, A.  Rosenberg, and P. Sarder, "Histo-fetch – On-the-fly processing of gigapixel whole slide images simplifies and speeds neural network training." To appear in Journal of Pathology Informatics.
  • image

    Computational segmentation of interstitial fibrosis and tubular atrophy

    This software conducts automated segmentation and quantification of interstitial fibrosis and tubular atrophy (IFTA) and glomerulosclerosis from renal tissue whole slide images.  All the codes, images and trained models used for evaluation are made available for the community.  A schematic overview is shown below.

    .
    image

    Schematic overview of IFTA segmentation using the H-AI-L pipeline. 

    Selected Reference

    • B. Ginley, K. Y. Jen, S. S. Han, L. Rodrigues, S. Jain, A. Fogo, J. Zuckerman, V. Walavalkar, J. C. Miecznikowski, Y. Wen, F. Wen, D. Yun, K. C. Moon, A. Rosenberg, C. Parikh, and P. Sarder, "Automated computational detection of interstitial fibrosis, tubular atrophy, and glomerulosclerosis," Journal of the American Society of Nephrology, vol. 32, no. 4, pp. 837-50, Apr. 2021.
  • image

    Synaptophysin-KI-Estimator (SKIE)

    This software automates Ki-67 index quantitation via whole-slide image (WSI) analysis for gastrointestinal neuroendocrine tumors (GI-NETs) and defines tumor grade. We also have introduced deep-SKIE, a deep learner-based approach where a Ki-67 index heatmap is generated throughout the tumor. Source codes used to derive the results are available here.  Raw data used to derive the results are available here. A schematic pipeline of SKIE is available here. A schematic of deep-SKIE is shown below.

    .
    image

    Deep-SKIE computational pipeline. 

    Selected Reference

    • D. Govind, K. Y. Jen, K. Matsukuma, G. Gao, K. A. Olson, D. Gui, G. Wilding, S. P. Border, and P. Sarder, "Improving the accuracy of gastrointestinal neuroendocrine tumor grading with deep learning," Scientific Reports - Nature, vol. 10, pp. 11064: 1–11, Jul. 2020.
  • image

    Computational segmentation and classification of diabetic glomerulosclerosis

    This software performs Tervaert classification of renal biopsies from patients with diabetic nephropathy.  Source codes used to derive the results are available here.  Raw data used to derive the results are available here.

    .
    image

    Overview of the computational pipeline. 

    Selected References

    • B. Ginley, B. Lutnick, K. Y. Jen, A. Fogo, S. Jain, A. Rosenberg, V. Walavalkar, G. Wilding, J. E. Tomaszewski, R. Yacoub, G. M. Rossi, and P. Sarder, "Computational segmentation and classification of diabetic glomerulosclerosis," Journal of American Society of Nephrology, vol. 30, no. 10, pp. 1953-67, Oct. 2019.
  • image

    Human Artificial Intelligence Loop (HAIL)

    This Python based software facilitates iterative annotation of structures from whole slide images by significantly reducing annotation burden.  Source codes used to derive the results are available here.  Raw data used to derive the results are available here.

    .
    image

    Overview of computational iterative annotation. 

    Selected References

    • B. Lutnick, B. Ginley, D. Govind, S. D. McGarry, P. S. LaViolette, R. Yacoub, S. Jain, J. E. Tomaszewski, K. Y. Jen, and P. Sarder, "An integrated iterative annotation technique for easing neural network training in medical image analysis," Nature Machine Intelligence, vol. 1, no. 2, pp. 112–19, Feb. 2019.
  • image

    Automated Erythrocyte Quantification from Whole Slide Images

    This MATLAB based software automatically detects, segments, and quantifies erythrocytes from whole slide images.  Source code and images used to derive the results are available here.

    .
    image

    Detection of optimally spaced blood cells in a blood smear.  (A) Whole slide image of a blood smear; the demarcation in red indicates monolayer, with barely touching cells.  (B) Estimated optimal region where blood cells are minimally touching.  (C) Segmentation of individual blood cells.

    Selected References

    • D. Govind, B. Lutnick, J. E. Tomaszewski, and P. Sarder, "Automated erythrocyte detection and classification from whole slide images," Journal of Medical Imaging - SPIE, vol. 5, no. 2, pp. 027501: 1–11, Apr. 2018.
  • image

    Image Analytics of Neutrophil Extracellular Traps

    This MATLAB based software identifies neutrophil extracellular traps from in vitro human samples visualized using the ImageStream® platform (Millipore Sigma, Darmstadt, Germany), and confocal images of thin tissue section images.  Source code and images used to derive the results are available here.

    .
    image

    Computational pipeline to identify in vitro neutrophil extracellular traps (NETs). (A–E) are NET-positive and (F–J) are NET-negative.  (A & F) Respective raw examples of a NET object and non-NET object.  NET indicated with white arrow in (A).  (B & G) Grayscale versions of the preceding images.  (C & H) Enhanced contrast using contrast limited adaptive histogram equalization.  (D & I) Ellipse fitting of the binary region, minor and major axes are demonstrated.  (E & J) Convex hull fitting of binary regions.  (K) Distribution of the three most separable morphological features for both image classes, displaying the optimal support vector hyperplane for n=1455 images.  (L) Morphological distributions for n=568 unstimulated neutrophils and n=294 stimulated neutrophils from an independent experiment.

    Selected References

    • B. Ginley, T. Emmons, B. Lutnick, C. Urban, B. H. Segal, and P. Sarder, "Computational detection and quantification of human and mouse neutrophil extracellular traps in flow cytometry and confocal microscopy," Scientific Reports - Nature, vol. 7, pp. 17755:1–11, Dec. 2017.
  • image

    Rapid Glomerular Detection in Whole Slide Histopathology Images

    This MATLAB based software detects glomeruli from whole-slide images (WSIs) of renal tissue sections marked with histological stains.  Multi-radial color local binary pattern features extracted from glomerular and non-glomerular regions are used to train a support vector machine (SVM), deployed in tandem with a deep convolutional neural network trained for glomerular recognition.  Using 5 Intel(R) Core(TM) i7-4790 CPUs with 40 GB RAM, the method takes ~2 min to extract glomeruli from a biopsy WSI (~106 pixels).  Source code and images used to derive the results are available here.

    .
    image

    Performance of a multi-radial color local binary pattern (mrcLBP) feature extraction method to localize glomerular locations in renal tissue images. Human renal biopsy images with automatically detected glomeruli locations (green squares), using mrcLBP feature vector to train a support vector machine.

    Selected References

    • O. Simon, R. Yacoub, S. Jain, J. E. Tomaszewski, and P. Sarder, "Multi-radial LBP features as a tool for rapid glomerular detection and assessment in whole slide histopathology images," Scientific Reports - Nature, vol. 8, pp. 2032:1–11, Feb. 2018.
  • image

    Segmentation of Glomerular Textural Boundary

    This MATLAB based software segments complex glomerulus boundary in histopathological images using an integrated method, based on Gabor filter bank based textural segmentation, statistical F-testing, and distance transform.  We have developed our method to segment glomerular boundaries in five common histopathological stains: hematoxylin & eosin, periodic acid-Schiff (PAS), Gömöri's trichrome, Congo red (CR), and Jones silver.  Our method localizes the exact glomerular boundary with mean sensitivity/ specificity of 0.88/0.96 and accuracy of 0.92 on 1000 images.  The H&E and trichrome images show the most specific performance as compared to manual annotation, with a mean sensitivity/specificity of 0.87/0.97.  Jones silver staining is least specific but also most sensitive, with mean sensitivity/specificity of 0.90/0.92.  CR and PAS staining fall between the other contenders, with sensitivity/specificity of 0.88/0.95 each. We will be happy to share our code and raw input data for pursuing collaborative research.

    .
    image

    Computational pipeline for segmenting the glomerular boundary. (A) Hematoxylin and eosin stained glomerular tissue image.  (B) Grayscale version of the image shown in A.  (C) Gaussian blurred image of B.  (D) Intensity image of the first principal component of the Gabor filter bank outputs using as input the image shown in C.  (E) K-means clustering was used to find final Gabor boundary.  (F) F-testing examines the entire image for similarity with E, and outputs 0 or 1 for each pixel.  (G) Morphological noise removal for the image in F.  (H–I) Respective binary masks obtained from F-testing and Gabor filter bank were distance transformed.  (J) Intensity image of a spatial weighting intensity map obtained from A.  (K) Heatmap of an average of intensity images in H–J. (L) Final segmentation after thresholding shown using green.  The segmentation obtained from initial Gabor (see E) is shown using black.  The green boundary depicts improved detection of glomerulus.

    Selected References

    • B. Ginley, J. E. Tomaszewski, R. Yacoub, F. Chen, and P. Sarder, "Unsupervised labeling of glomerular boundaries using Gabor filters and statistical testing in renal histology," Journal of Medical Imaging - SPIE, vol. 4, no. 2, pp. 021102:1–13, Feb. 2017.

Software (developed at Washington University School of Medicine in St.  Louis)

  • image

    Multiscale Segmentation of Fluorescence Microscopy Images

    This C++ based software automatically segments fluorescence microscopy images for a given resolution.  Users can control the desired levels of detail in the segmented images by tuning a resolution parameter while segmenting using our software.  In the resulting segmentation, low resolution leads to larger segments, and high resolution leads to smaller segments.

    .
    image

    (Left) Fluorescence lifetime (FLT) imaging microscopy (FLIM) image of live cells containing fluorescent dyes of different FLTs.  Different FLTs here depict molecular events in cellular microenvironment.  Color map was chosen manually to show different cellular regions with different FLTs.  (Right) Automatic segmentation at increasing resolution (γ) for the image shown in left.  Colors depict different segments.  Number of segments in the processed image increases with resolution, identifies the correct number of segments at γ = 1-10, and introduces noisy segments at higher resolution.

    Selected References (* indicates equal contribution)

    • D. Hu*, P. Sarder*, P. Ronhovde, S. Orthaus, S. Achilefu, and Z. Nussinov, "Automatic segmentation of fluorescence lifetime microscopy images of cells using multiresolution community detection–a first study," Journal of Microscopy, vol. 253, no. 1, pp. 54–64, 2014.
  • image

    Segmentation of DIC Microscopy Images

    This MATLAB based software takes as input any 2D DIC microscopy image of cells, and automatically segments the cells by press of a button.  It can also take fluorescence images of the cells as additional input, and can provide a table, listing the cell area, fluorescence mean and variance, and structural heterogeneity of each cell.  A snap of the software is provided below.

    .
    image
  • image

    Classification of Small Peptides based on Synthesis Yield

    This python based software predicts the probability defining how easily a new peptide sequence can be synthesized when such a sequence is considered for synthesis in a chemistry lab.  If the prediction probability is low, the software suggests minimally modified sequences that can be synthesized with ease.  Potential application of this software can be exploring new molecular targets in cancer.

    .

    First Four Steps:

    .
    image

    Step1: Upload training & test sequences

    image

    Step2: Prediction

    .
    image

    Step3: Performance analysis

    image

    Step4: Estimation of the stability likelihood of the amino acids in the sequence

    .

    Performance:

    image

    (Left) Performance of the proposed method (NBC) in predicting how easy a peptide is synthesizable.  A total of 161 peptides were considered for learning.  Information obtained from the peptide sequences alone allows our method (NBC1) to achieve prediction with AUC = 0.89.  Additional information on the peptides, such as their sequence length, hydrophobicity, charge, oxidation, and β-sheet formation, in addition with their sequence information, improves prediction of the method (NBC2) with AUC = 0.97.  (Right) Proposed method outperforms commercially available software Peptide Companion (PC).  Same peptide sequences as considered in the previous example were considered.  NBC performs prediction with AUC = 0.97, while PC was only able to achieve an AUC of 0.83 in predicting synthesis feasibility of a peptide.