Underline indicates corresponding author.
* indicates equal contribution.
†Copyright (2013) American Institute of Physics.  This article may be downloaded for personal use only.  Any other use requires prior permission of the author and the American Institute of Physics.
‡The pdf file may not be exactly the same as that in the published book chapter, but should be similar.
Neurodegenerative diseases including Alzheimer's affect millions around the world, and this number is projected to increase over the years unless a breakthrough is made.  There are several theories on the pathogenesis of neurodegenerative diseases, with the amyloid cascade and tau theory being the most prominent ones.  The formation of amyloid plaques and tau tangles collapses capillaries in the brain, thereby inducing hypoxia and destruction of neurons from loss of nourishment.  While we do understand some of the changes that occur in the brain's vasculature form the pathogenesis of these diseases, they have not yet been mathematically characterized with precision.  A computational pipeline is presented here to analyze optically sectioned mice brain sections imaged via two-photon microscopy and characterize various vasculature parameters which are known to deteriorate from neurogenerative diseases.  Our proposed pipeline aims to quantify various brain vasculature parameters, such as, vessel tortuosity, diameter, volume and length, as well the degree of difference to understand disease pathogenesis with the eventual hope of providing drug intervention to regress or minimize these changes.
In diabetic nephropathy (DN), hyperglycemia drives a progressive thickening of glomerular filtration surfaces, increased cell proliferation as well as mesangial expansion and a constriction of capillary lumens.  This leads to progressive structural changes inside the glomeruli.  In this work, we make a study of structural glomerular changes in DN from a graph-theoretic standpoint, using features extracted from minimal spanning trees (MSTs) constructed over intercellular distances in order to classify the "packing signatures" of different DN stages.  We further investigate the significance of the competing effects of volume change measured here in 2-dimensional pixel span area on one hand and increased cell proliferation on the other in determining the packing patterns.  Towards that we formulate the problem as dynamic Bayesian network (DBN).  From our preliminary results we do postulate that volume expansion caused by internal pressure as capillary lumens constriction has perhaps has a greater effect in the early stages.
Neural networks promise to bring robust, quantitative analysis to medical fields. However, their adoption is limited by the technicalities of training these networks and the required volume and quality of human-generated annotations. To address this gap in the field of pathology, we have created an intuitive interface for data annotation and the display of neural network predictions within a commonly used digital pathology whole-slide viewer.  This strategy used a 'human-in-the-loop' to reduce the annotation burden.  We demonstrate that segmentation of human and mouse renal micro compartments is repeatedly improved when humans interact with automatically generated annotations throughout the training process.  Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data.
Schizophrenia is a neurodevelopmental disorder characterized by complex aberrations in the structure, wiring, and chemistry of multiple neuronal systems.  The abnormal developmental trajectory of the brain is established during gestation, long before clinical manifestation of the disease.  Over 200 genes and even greater numbers of single nucleotide polymorphisms and copy number variations have been linked with schizophrenia.  How does altered function of such a variety of genes lead to schizophrenia?  We propose that the protein products of these altered genes converge on a common neurodevelopmental pathway responsible for the development of brain neural circuit and neurotransmitter systems.  The results of a multichanneled investigation using induced pluripotent stem cell (iPSCs)- and embryonic stem cell (ESCs)-derived neuronal committed cells (NCCs) indicate an early (preneuronal) developmental-genomic etiology of schizophrenia and that the dysregulated developmental gene networks are common to genetically unrelated cases of schizophrenia.  The results support a "watershed" mechanism in which mutations within diverse signaling pathways affect the common pan-ontogenic mechanism, integrative nuclear (n)FGFR1 signaling (INFS).  Dysregulation of INFS in schizophrenia NCCs deconstructs coordinated gene networks and leads to formation of new networks by the dysregulated genes.  This genome deprograming affects critical gene programs and pathways for neural development and functions.  Studies show that the genomic deprograming reflect an altered nFGFR1—genome interactions and deregulation of miRNA genes by nFGFR1.  In addition, changes in chromatin topology imposed by nFGFR1 may play a role in coordinate gene dysregulation in schizophrenia.
While the in vivo efficacy of Sn-2 phosphatidylcholine prodrugs incorporated into targeted, non-pegylated lipid-encapsulated nanoparticles was demonstrated in prior preclinical studies, the microscopic details of cell prodrug internalization and trafficking events are unknown.  Classic fluorescence microscopy, fluorescence lifetime imaging microscopy, and single-molecule super-resolution microscopy were used to investigate the cellular handling of doxorubicin-prodrug and AlexaFluor-488-prodrug.  Sn-2 phosphatidylcholine prodrugs delivered by hemifusion of nanoparticle and cell phospholipid membranes functioned as phosphatidylcholine mimics, circumventing the challenges of endosome sequestration and release.  Phosphatidylcholine prodrugs in the outer cell membrane leaflet translocated to the inner membrane leaflet by ATP-dependent and ATP-independent mechanisms and distributed broadly within the cytosolic membranes over the next 12 h.  A portion of the phosphatidylcholine prodrug populated vesicle membranes trafficked to the perinuclear Golgi/ER region, where the drug was enzymatically liberated and activated.  Native doxorubicin entered the cells, passed rapidly to the nucleus, and bound to dsDNA, whereas DOX was first enzymatically liberated from DOX-prodrug within the cytosol, particularly in the perinuclear region, before binding nuclear dsDNA.  Much of DOX-prodrug was initially retained within intracellular membranes.  In vitro anti-proliferation effectiveness of the two drug delivery approaches was equivalent at 48 h, suggesting that residual intracellular DOX-prodrug may constitute a slow-release drug reservoir that enhances effectiveness.  We have demonstrated that Sn-2 phosphatidylcholine prodrugs function as phosphatidylcholine mimics following reported pathways of phosphatidylcholine distribution and metabolism.  Drug complexed to the Sn-2 fatty acid is enzymatically liberated and reactivated over many hours, which may enhance efficacy over time.
Blood smear is a crucial diagnostic aid.  Quantification of both solitary and overlapping erythrocytes within these smears, directly from their whole slide images (WSIs), remains a challenge.  Existing software designed to accomplish the computationally extensive task of hematological whole slide image analysis are too expensive and are widely unavailable.  We have thereby developed a fully automated software targeted for erythrocyte detection and quantification from WSIs.  In this paper, we define an optimal region within the smear which contain cells which are neither too scarce or damaged nor too crowded.  We detect the optimal regions within the smear and subsequently extract all the cells from these regions, both solitary and overlapped, the latter of which undergoes a clump splitting before extraction.  The performance was systematically tested on 28 WSIs of blood smears obtained from 13 different species from three classes of the subphylum vertebrata including birds, mammals, and reptiles.  This data poses as an immensely variant erythrocyte database with diversity in size, shape, intensity and textural features.  Our method detected ~3.02 times more cells than that detected from the traditional monolayer and resulted in a testing accuracy of 99.14% for the classification into their respective class (bird, mammal, or reptile), and a testing accuracy of 84.73% for the classification into their respective species.  The results suggest the potential employment of this software for the diagnosis of hematological disorders like sickle cell anemia.
We present a rapid, scalable, and high throughput computational pipeline to accurately detect and segment the glomerulus from renal histopathology images with high precision and accuracy.  Our proposed method integrates information from fluorescence and bright-field microscopy imaging of renal tissues.  For computation, we exploit the simplicity, yet extreme robustness of Butterworth bandpass filter to extract the glomeruli by utilizing the information inherent in the renal tissue stained with immunofluorescence marker sensitive at blue emission wavelength as well as tissue auto-fluorescence.  The resulting output is in-turn used to detect and segment multiple glomeruli within the field-of-view in the same tissue section post-stained with histopathological stains.  Our approach, optimized over 40 images, produced a sensitivity/specificity of 0.95/0.84 on n = 66 test images, each containing one or more glomeruli.  The work not only has implications in renal histopathology involving diseases with glomerular structural damages, which is vital to track the progression of the disease, but also aids in the development of a tool to rapidly generate a database of glomeruli from whole slide images, essential for training neural networks.  The current practice to detect glomerular structural damage is by the manual examination of biopsied renal tissues, which is laborious, time intensive and tedious.  Existing automated pipelines employ complex neural networks which are computationally extensive, demand expensive high-performance hardware and require large expert-annotated datasets for training.  Our automated method to detect glomerular boundary will aid in rapid extraction of glomerular compartmental features from large renal histopathological images.
The adoption of deep learning techniques in medical applications has thus far been limited by the availability of the large labeled datasets required to robustly train neural networks, as well as difficulty interpreting these networks.  However, recent techniques for unsupervised training of neural networks promise to address these issues, leveraging only structure to model input data.  We propose the use of a variational autoencoder (VAE) which utilizes data from an animal model to augment the training set and non-linear dimensionality reduction to map this data to human sets.  This architecture utilizes variational inference, performed on latent parameters, to statistically model the probability distribution of training data in a latent feature space.  We show the feasibility of VAEs, using images of mouse and human renal glomeruli from various pathological stages of diabetic nephropathy (DN), to model the progression of structural changes which occur in DN.  When plotted in a 2-dimentional latent space, human and mouse glomeruli, show separation with some overlap, suggesting that the data is continuous, and can be statistically correlated.  When DN stage is plotted in this latent space, trends in disease pathology are visualized.
In diabetic nephropathy (DN), hyperglycemia drives a progressive thickening of and damage to the glomerular filtration surfaces, as well as mesangial expansion and a constriction of capillary lumens.  This leads at first to high blood pressure, increased glomerular filtration and micro-proteinuria, and later (if untreated) to severe proteinuria and end-stage renal disease (ESRD).  Though, it is well known that DN is accompanied by marked histopathological changes, the assessment of these structural changes is to a degree subjective and hence varies between pathologists.  In this work, we make a first study of glomerular changes in DN from a graph-theoretical and distance-based standpoint, using minimal spanning trees (MSTs) and distance matrices to generate statistical distributions that can potentially provide a “fingerprint” of DN.  We apply these tools to detect notable differences between normal and DN glomeruli in both human disease and in a streptozotocin-induced (STZ) mouse model.  We also introduce an automated pipeline for rapidly generating MSTs and evaluating their properties with respect to DN, and make a first pass at three-dimensional MST structures.  We envision these approaches may provide a better understanding not only of the processes underway in DN progression, but of key differences between actual human disease and current experimental models.
The glomerulus is the primary compartment of blood filtration in the kidney.  It is a sphere of bundled, fenestrated capillaries that selectively allows solute loss.  Structural damages to glomerular micro-compartments lead to physiological failures which influence filtration efficacy.  The sole way to confirm glomerular structural damage in renal pathology is by examining histopathological or immunofluorescence stained needle biopsies under a light microscope.  However, this method is extremely tedious and time consuming, and requires manual scoring on the number and volume of structures.  Computational image analysis is the perfect tool to ease this burden.  The major obstacle to development of digital histopathological quantification protocols for renal pathology is the extreme heterogeneity present within kidney tissue.  Here we present an automated computational pipeline to 1) segment glomerular compartment boundaries and 2) quantify features of compartments, in healthy and diseased renal tissue.  The segmentation involves a two stage process, one step for rough segmentation generation and another for refinement.  Using a Naïve Bayesian classifier on the resulting feature set, this method was able to distinguish pathological stage IIa from III with 0.89/0.93 sensitivity/specificity and stage IIb from III with 0.7/0.8 sensitivity/specificity, on n = 514 glomeruli taken from n = 13 human biopsies with diagnosed diabetic nephropathy, and n = 5 human renal tissues with no histological abnormalities.  Our method will simplify computational partitioning of glomerular micro-compartments and subsequent quantification.  We aim for our methods to ease manual labor associated with clinical diagnosis of renal disease.
We demonstrate a simple and effective automated method for the localization of glomeruli in large (~1 gigapixel) histopathological whole-slide images (WSIs) of thin renal tissue sections and biopsies, using an adaptation of the well-known local binary patterns (LBP) image feature vector to train a support vector machine (SVM) model.  Our method offers high precision (>90%) and reasonable recall (>70%) for glomeruli from WSIs, is readily adaptable to glomeruli from multiple species, including mouse, rat, and human, and is robust to diverse slide staining methods.  Using 5 Intel(R) Core(TM) i7-4790 CPUs with 40 GB RAM, our method typically requires ~15 sec for training and ~2 min to extract glomeruli reproducibly from a WSI.  Deploying a deep convolutional neural network trained for glomerular recognition in tandem with the SVM suffices to reduce false positives to below 3%.  We also apply our LBP-based descriptor to successfully detect pathologic changes in a mouse model of diabetic nephropathy.  We envision potential clinical and laboratory applications for this approach in the study and diagnosis of glomerular disease, and as a means of greatly accelerating the construction of feature sets to fuel deep learning studies into tissue structure and pathology.
Quantitative and noninvasive measurement of protease activities has remained an imaging challenge in deep tissues such as the lungs.  Here, we designed a dual-radiolabeled probe for reporting the activities of proteases such as matrix metalloproteinases (MMPs) with multispectral single photon emission computed tomography (SPECT) imaging.  A gold nanoparticle (NP) was radiolabeled with 125I and 111In and functionalized with an MMP9-cleavable peptide to form a multispectral SPECT imaging contrast agent.  In another design, incorporation of 199Au radionuclide into the metal crystal structure of gold NPs provided a superior and stable reference signal in lungs, and 111In was linked to the NP surface via a protease-cleavable substrate, which can serve as an enzyme activity reporter.  This work reveals strategies to correlate protease activities with diverse pathologies in a tissue-depth independent manner.
Neutrophil extracellular traps (NETs) are extracellular defense mechanisms used by neutrophils, where chromatin is expelled together with histones and granular/cytoplasmic proteins.  They have become an immunology hotspot, implicated in infections, but also in a diverse array of diseases such as systemic lupus erythematosus, diabetes, and cancer.  However, the precise assessment of in vivo relevance in different disease settings has been hampered by limited tools to quantify occurrence of extracellular traps in experimental models and human samples.  To expedite progress towards improved quantitative tools, we have developed computational pipelines to identify extracellular traps from an in vitro human samples visualized using the ImageStream® platform (Millipore Sigma, Darmstadt, Germany), and confocal images of an in vivo mouse disease model of aspergillus fumigatus pneumonia.  Our two in vitro methods, tested on n = 363/n = 145 images respectively, achieved holdout sensitivity/specificity 0.98/0.93 and 1/0.92.  Our unsupervised method for thin lung tissue sections in murine fungal pneumonia achieved sensitivity/specificity 0.99/0.98 in n = 14 images.  Our supervised method for thin lung tissue classified NETs with sensitivity/specificity 0.86/0.90. We expect that our approach will be of value for researchers, and have application in infectious and inflammatory diseases.
Studies of induced pluripotent stem cells (iPSCs) from schizophrenia patients and control individuals revealed that the disorder is programmed at the preneuronal stage, involves a common dysregulated mRNA transcriptome, and identified Integrative Nuclear FGFR1 Signaling a common dysregulated mechanism.  We used human embryonic stem cell (hESC) and iPSC-derived cerebral organoids from four controls and three schizophrenia patients to model the first trimester of in utero brain development.  The schizophrenia organoids revealed an abnormal scattering of proliferating Ki67+ neural progenitor cells (NPCs) from the ventricular zone (VZ), throughout the intermediate (IZ) and cortical (CZ) zones.  TBR1 pioneer neurons and reelin, which guides cortico-petal migration, were restricted from the schizophrenia cortex.  The maturing neurons were abundantly developed in the subcortical regions, but were depleted from the schizophrenia cortex.  The decreased intracortical connectivity was denoted by changes in the orientation and morphology of calretinin interneurons.  In schizophrenia organoids, nuclear (n)FGFR1 was abundantly expressed by developing subcortical cells, but was depleted from the neuronal committed cells (NCCs) of the CZ.  Transfection of dominant negative and constitutively active nFGFR1 caused widespread disruption of the neuro-ontogenic gene networks in hESC-derived NPCs and NCCs.  The fgfr1 gene was the most prominent FGFR gene expressed in NPCs and NCCs, and blocking with PD173074 reproduced both the loss of nFGFR1 and cortical neuronal maturation in hESC cerebral organoids.  We report for the first time, progression of the cortical malformation in schizophrenia and link it to altered FGFR1 signaling.  Targeting INFS may offer a preventive treatment of schizophrenia.
The watershed-hypothesis of schizophrenia asserts that over 200 different mutations dysregulate distinct pathways that converge on an unspecified common mechanism(s) that controls disease ontogeny.  Consistent with this hypothesis, our RNA-sequencing of neuron committed cells (NCCs) differentiated from established iPSCs of 4 schizophrenia patients and 4 control subjects uncovered a dysregulated transcriptome of 1349 mRNAs common to all patients.  Data reveals a global dysregulation of developmental genome, deconstruction of coordinated mRNA networks, and the formation of aberrant, new coordinated mRNA networks indicating a concerted action of the responsible factor(s).  Sequencing of miRNA transcriptomes demonstrated an overexpression of 16 miRNAs and deconstruction of interactive miRNA–mRNA networks in schizophrenia NCCs.  ChiPseq revealed that the nuclear (n) form of FGFR1, a pan-ontogenic regulator, is overexpressed in schizophrenia NCCs and overtargets dysregulated mRNA and miRNA genes.  The nFGFR1 targeted 54% of all human gene promoters and 84.4% of schizophrenia dysregulated genes.  The upregulated genes reside within major developmental pathways that control neurogenesis and neuron formation, whereas downregulated genes are involved in oligodendrogenesis.  Our results indicate (i) an early (preneuronal) genomic etiology of schizophrenia, (ii) dysregulated genes and new coordinated gene networks are common to unrelated cases of schizophrenia, (iii) gene dysregulations are accompanied by increased nFGFR1-genome interactions, and (iv) modeling of increased nFGFR1 by an overexpression of a nFGFR1 lead to up or downregulation of selected genes as observed in schizophrenia NCCs.  Together our results designate nFGFR1 signaling as a potential common dysregulated mechanism in investigated patients and potential therapeutic target in schizophrenia.
Clinical pathology relies on manual compartmentalization and quantification of biological structures, which is time consuming and often error-prone.  Application of computer vision segmentation algorithms to histopathological image analysis, in contrast, can offer fast, reproducible, and accurate quantitative analysis to aid pathologists.  Algorithms tunable to different biologically relevant structures can allow accurate, precise, and reproducible estimates of disease states.  In this direction, we have developed a fast, unsupervised computational method for simultaneously separating all biologically relevant structures from histopathological images in multi-scale.  Segmentation is achieved by solving an energy optimization problem.  Representing the image as a graph, nodes (pixels) are grouped by minimizing a Potts model Hamiltonian, adopted from theoretical physics, modeling interacting electron spins.  Pixel relationships (modeled as edges) are used to update the energy of the partitioned graph.  By iteratively improving the clustering, the optimal number of segments is revealed.  To reduce computational time, the graph is simplified using a Cantor pairing function to intelligently reduce the number of included nodes.  The classified nodes are then used to train a multiclass support vector machine to apply the segmentation over the full image.  Accurate segmentations of images with as many as 106 pixels can be completed only in 5 sec, allowing for attainable multi-scale visualization.  To establish clinical potential, we employed our method in renal biopsies to quantitatively visualize for the first time scale variant compartments of heterogeneous intra- and extraglomerular structures simultaneously.  Implications of the utility of our method extend to fields such as oncology, genomics, and non-biological problems.
The glomerulus, a specialized bundle of capillaries, is the blood filtering unit of the kidney.  Each human kidney contains about 1 million glomeruli.  Structural damages in the glomerular micro-compartments give rise to several renal conditions; most severe of which is proteinuria, where excessive blood proteins flow freely to the urine.  The sole way to confirm glomerular structural damage in renal pathology is by examining histopathological or immunofluorescence stained needle biopsies under a light microscope.  However, this method is extremely tedious and time consuming, and requires manual scoring on the number and volume of structures.  Computational quantification of equivalent features promises to greatly ease this manual burden.  The largest obstacle to computational quantification of renal tissue is the ability to recognize complex glomerular textural boundaries automatically.  Here we present a computational pipeline to accurately identify glomerular boundaries with high precision and accuracy.  The computational pipeline employs an integrated approach composed of Gabor filtering, Gaussian blurring, statistical F-testing, and distance transform, and performs significantly better than standard Gabor based textural segmentation method.  Our integrated approach provides mean accuracy/precision of 0.89/0.97 on n = 200 Hematoxylin and Eosin (H&E) glomerulus images, and mean 0.88/0.94 accuracy/precision on n = 200 Periodic Acid Schiff (PAS) glomerulus images.  Respective accuracy/precision of the Gabor filter bank based method is 0.83/0.84 for H&E and 0.78/0.8 for PAS.  Our method will simplify computational partitioning of glomerular micro-compartments hidden within dense textural boundaries.  Automatic quantification of glomeruli will streamline structural analysis in clinic, and can help realize real time diagnoses and interventions.
Neutrophil extracellular trap (NET) formation is an alternate immunologic weapon used mainly by neutrophils.  Chromatin backbones fused with proteins derived from granules are shot like projectiles onto foreign invaders.  It is thought that this mechanism is highly anti-microbial, aids in preventing bacterial dissemination, is used to break down structures several sizes larger than neutrophils themselves, and may have several more uses yet unknown.  NETs have been implied to be involved in a wide array of systemic host immune defenses, including sepsis, autoimmune diseases, and cancer.  Existing methods used to visually quantify NETotic versus non-NETotic shapes are extremely time-consuming and subject to user bias.  These limitations are obstacles to developing NETs as prognostic biomarkers and therapeutic targets.  We propose an automated pipeline for quantitatively detecting neutrophil and NET shapes captured using a flow cytometry-imaging system.  Our method uses contrast limited adaptive histogram equalization to improve signal intensity in dimly illuminated NETs.  From the contrast improved image, fixed value thresholding is applied to convert the image to binary.  Feature extraction is performed on the resulting binary image, by calculating region properties of the resulting foreground structures.  Classification of the resulting features is performed using Support Vector Machine.  Our method classifies NETs from neutrophils without traps at 0.97/0.96 sensitivity/specificity on n = 387 images, and is 1500X faster than manual classification, per sample.  Our method can be extended to rapidly analyze whole-slide immunofluorescence tissue images for NET classification, and has potential to streamline the quantification of NETs for patients with diseases associated with cancer and autoimmunity.
The glomerulus is the blood filtering unit of the kidney.  Each human kidney contains ~1 million glomeruli.  Several renal conditions originate from structural damage to glomerular microcompartments, such as proteinuria, the excessive loss of blood proteins into urine.  The gold standard for evaluating structural damage in renal pathology is histopathological and immunofluorescence examination of needle biopsies under a light microscope.  This method is limited by qualitative or semiquantitative manual scoring approaches to the evaluation of glomerular structural features.  Computational quantification of equivalent features promises to improve the precision of glomerular structural analysis.  One large obstacle to the computational quantification of renal tissue is the identification of complex glomerular boundaries automatically.  To mitigate this issue, we developed a computational pipeline capable of extracting and exactly defining glomerular boundaries.  Our method, composed of Gabor filtering, Gaussian blurring, statistical F-testing, and distance transform, is able to accurately identify glomerular boundaries with mean sensitivity/specificity of 0.88/0.96 and accuracy of 0.92, on n = 1000 glomeruli images stained with standard renal histological stains.  Our method will simplify computational partitioning of glomerular microcompartments hidden within dense textural boundaries.  Automatic quantification of glomeruli will streamline structural analysis in clinic and can pioneer real-time diagnoses and interventions for renal care.
The branch of pathology concerned with excess blood serum proteins being excreted in the urine pays particular attention to the glomerulus, a small intertwined bunch of capillaries located at the beginning of the nephron.  Normal glomeruli allow moderate amount of blood proteins to be filtered; proteinuric glomeruli allow large amount of blood proteins to be filtered.  Diagnosis of proteinuric diseases requires time intensive manual examination of the structural compartments of the glomerulus from renal biopsies.  Pathological examination includes cellularity of individual compartments, Bowman's and luminal space segmentation, cellular morphology, glomerular volume, capillary morphology, and more.  Long examination times may lead to increased diagnosis time and/or lead to reduced precision of the diagnostic process.  Automatic quantification holds strong potential to reduce renal diagnostic time.  We have developed a computational pipeline capable of automatically segmenting relevant features from renal biopsies.  Our method first segments glomerular compartments from renal biopsies by isolating regions with high nuclear density.  Gabor texture segmentation is used to accurately define glomerular boundaries.  Bowman's and luminal spaces are segmented using morphological operators.  Nuclei structures are segmented using color deconvolution, morphological processing, and bottleneck detection.  Average computation time of feature extraction for a typical biopsy, comprising of ~12 glomeruli, is ~69 s using an Intel(R) Core(TM) i7-4790 CPU, and is ~65X faster than manual processing.  Using images from rat renal tissue samples, automatic glomerular structural feature estimation was reproducibly demonstrated for 15 biopsy images, which contained 148 individual glomeruli images.  The proposed method holds immense potential to enhance information available while making clinical diagnoses.
Enhanced glycolysis and poor perfusion in most solid malignant tumors create an acidic extracellular environment, which enhances tumor growth, invasion, and metastasis.  Complex molecular systems have been explored for imaging and treating these tumors.  Here, we report the development of a small molecule, LS662, that emits near-infrared (NIR) fluorescence upon protonation by the extracellular acidic pH environment of diverse solid tumors.  Protonation of LS662 induces selective internalization into tumor cells and retention in the tumor microenvironment.  Noninvasive NIR imaging demonstrates selective retention of the pH sensor in diverse tumors, and two-photon microscopy of ex vivo tumors reveals significant retention of LS662 in tumor cells and the acid tumor microenvironment.  Passive and active internalization processes combine to enhance NIR fluorescence in tumors over time.  The low background fluorescence allows tumors to be detected with high sensitivity, as well as dead or dying cells to be delineated from healthy cells.  In addition to demonstrating the feasibility of using small molecule pH sensors to image multiple aggressive solid tumor types via a protonation-induced internalization and retention pathway, the study reveals the potential of using LS662 to monitor treatment response and tumor-targeted drug delivery.
Open surgery requiring cytoreduction still remains the primary treatment course for many cancers.  The extent of resection is vital for the outcome of surgery, greatly affecting patients' follow-up treatment including need for revision surgery in the case of positive margins, choice of chemotherapy, and overall survival.  Existing imaging modalities such as computed tomography, magnetic resonance imaging, and positron emission tomography are useful in the diagnostic stage and long-term monitoring but do not provide the level of temporal or spatial resolution needed for intraoperative surgical guidance.  Surgeons must instead rely on visual evaluation and palpation in order to distinguish tumors from surrounding tissues.  Fluorescence imaging provides high-resolution, real-time mapping with the use of a contrast agent and can greatly enhance intraoperative imaging.  Here we demonstrate an intraoperative, real-time fluorescence imaging system for direct highlighting of target tissues for surgical guidance, optical projection of acquired luminescence (OPAL).  Image alignment, accuracy, and resolution was determined in vitro prior to demonstration of feasibility for operating room use in large animal models of sentinel lymph node biopsy.  Fluorescence identification of regional lymph nodes after intradermal injection of indocyanine green was performed in pigs with surgical guidance from the OPAL system.  Acquired fluorescence images were processed and rapidly reprojected to highlight indocyanine green within the true surgical field.  OPAL produced enhanced visualization for resection of lymph nodes at each anatomical location.  Results show the optical projection of acquired luminescence system can successfully use fluorescence image capture and projection to provide aligned image data that is invisible to the human eye in the operating room setting.
Visualization of biological processes and pathologic conditions at the cellular and tissue levels largely relies on the use of fluorescence intensity signals from fluorophores or their bioconjugates.  To overcome the concentration dependency of intensity measurements, evaluate subtle molecular interactions, and determine biochemical status of intracellular or extracellular microenvironments, fluorescence lifetime (FLT) imaging has emerged as a reliable imaging method complementary to intensity measurements.  Driven by a wide variety of dyes exhibiting stable or environment-responsive FLTs, information multiplexing can be readily accomplished without the need for ratiometric spectral imaging.  With knowledge of the fluorescent states of the molecules, it is entirely possible to predict the functional status of biomolecules or microevironment of cells.  Whereas the use of FLT spectroscopy and microscopy in biological studies is now well-established, in vivo imaging of biological processes based on FLT imaging techniques is still evolving.  This review summarizes recent advances in the application of the FLT of molecular probes for imaging cells and small animal models of human diseases.  It also highlights some challenges that continue to limit the full realization of the potential of using FLT molecular probes to address diverse biological problems and outlines areas of potential high impact in the future.
Multiresolution community detection (CD) method has been suggested in a recent work as an efficient method for performing unsupervised segmentation of fluorescence lifetime (FLT) images of live cell images containing fluorescent molecular probes.  In the current paper, we further explore this method in FLT images of ex vivo tissue slices.  The image processing problem is framed as identifying clusters with respective average FLTs against a background or solvent in FLT imaging microscopy (FLIM) images derived using NIR fluorescent dyes.  We have identified significant multiresolution structures using replica correlations in these images, where such correlations are manifested by information theoretic overlaps of the independent solutions (replicas) attained using the multiresolution CD method from different starting points.  In this paper, our method is found to be more efficient than a current state-of-the-art image segmentation method based on mixture of Gaussian distributions. It offers more than 1:25 times diversity based on Shannon index than the latter method, in selecting clusters with distinct average FLTs in NIR FLIM images.
We report two methods for quantitatively determining maximal imaging depth from thick tissue images captured using all–near-infrared (NIR) multiphoton microscopy (MPM).  All-NIR MPM is performed using 1550 nm laser excitation with NIR detection.  This method enables imaging more than five-fold deep in thick tissues in comparison with other NIR excitation microscopy methods.  In this study, we show a correlation between the multiphoton signal along the depth of tissue samples and the shape of the corresponding empirical probability density function (pdf) of the photon counts.  Histograms from this analysis become increasingly symmetric with the imaging depth.  This distribution transitions toward the background distribution at higher imaging depths.  Inspired by these observations, we propose two independent methods based on which one can automatically determine maximal imaging depth in the all-NIR MPM images of thick tissues.  At this point, the signal strength is expected to be weak and similar to the background.  The first method suggests the maximal imaging depth corresponds to the deepest image plane where the ratio between the mean and median of the empirical photon-count pdf is outside the vicinity of 1.  The second method suggests the maximal imaging depth corresponds to the deepest image plane where the squared distance between the empirical photon-count mean obtained from the object and the mean obtained from the background is greater than a threshold.  We demonstrate the application of these methods in all-NIR MPM images of mouse kidney tissues to study maximal depth penetration in such tissues.
Fluorescence quenching properties of copper(II) ions have been used for designing Cu(II) sensitive fluorescent molecular probes.  In this paper, we demonstrate that static quenching plays a key role in free Cu(II)-mediated fluorescence quenching of a near infrared (NIR) fluorescent dye cypate.  The Stern-Volmer quenching constant was calculated to be KSV = 970,000 M-1 in 25 mM MES buffer, pH 6.5 at room temperature.  We synthesized LS835, a compound containing cypate attached covalently to chelated Cu(II) to study fluorescence quenching by chelated Cu(II).  The fluorescence quenching mechanism of chelated Cu(II) is predominantly dynamic or collisional quenching.  The quenching efficiency of chelated Cu(II) was calculated to be 58%±6% in dimethylsulfoxide at room temperature.  Future work will involve further characterization of the mechanism of NIR fluorescence quenching by Cu(II) and testing its reversibility for potential applications in designing fluorophore-quencher based molecular probes for biological imaging.
We propose an analytical framework to build a microfluidic microsphere-trap array device that enables simultaneous, efficient, and accurate screening of multiple biological targets in a single microfluidic channel.  By optimizing the traps' geometric parameters, the trap arrays in the channel of the device can immobilize microspheres of different sizes at different regions, obeying hydrodynamically engineered trapping mechanism.  Different biomolecules can be captured by the ligands on the surfaces of microspheres of different sizes.  They are thus detected according to the microspheres' positions (position encoding), which simplifies screening and avoids target identification errors.  To demonstrate the proposition, we build a device for simultaneous detection of two target types by trapping microspheres of two sizes.  We evaluate the device performance using finite element fluidic dynamics simulations and microsphere-trapping experiments.  These results validate that the device efficiently achieves position encoding of the two-sized microspheres with few fluidic errors, providing the promise to utilize our framework to build devices for simultaneous detection of more targets.  We also envision utilizing the device to separate, sort, or enumerate cells, such as circulating tumor cells and blood cells, based on cell size and deformability.  Therefore, the device is promising to become a cost-effective and point-of-care miniaturized disease diagnostic tool.
Inspired by a multiresolution community detection based network segmentation method, we suggest an automatic method for segmenting fluorescence lifetime (FLT) imaging microscopy (FLIM) images of cells in a first pilot investigation on two selected images.  The image processing problem is framed as identifying segments with respective average FLTs against the background in FLIM images.  The proposed method segments a FLIM image for a given resolution of the network defined using image pixels as the nodes and similarity between the FLTs of the pixels as the edges.  In the resulting segmentation, low network resolution leads to larger segments, and high network resolution leads to smaller segments.  Furthermore, using the proposed method, the mean-square error in estimating the FLT segments in a FLIM image was found to consistently decrease with increasing resolution of the corresponding network.  The multiresolution community detection method appeared to perform better than a popular spectral clustering-based method in performing FLIM image segmentation.  At high resolution, the spectral segmentation method introduced noisy segments in its output, and it was unable to achieve a consistent decrease in mean-square error with increasing resolution.
Optical imaging enables real-time visualization of intrinsic and exogenous contrast within biological tissues.  Applications in human medicine have demonstrated the power of fluorescence imaging to enhance visualization in dermatology, endoscopic procedures, and open surgery.  Although few optical contrast agents are available for human medicine at this time, fluorescence imaging is proving to be a powerful tool in guiding medical procedures.  Recently, intraoperative detection of fluorescent molecular probes that target cell-surface receptors has been reported for improvement in oncologic surgery in humans.  We have developed a novel system, optical projection of acquired luminescence (OPAL), to further enhance real-time guidance of open oncologic surgery.  In this method, collected fluorescence intensity maps are projected onto the imaged surface rather than via wall-mounted display monitor.  To demonstrate proof-of-principle for OPAL applications in oncologic surgery, lymphatic transport of indocyanine green was visualized in live mice for intraoperative identification of sentinel lymph nodes.  Subsequently, peritoneal tumors in a murine model of breast cancer metastasis were identified using OPAL after systemic administration of a tumor-selective fluorescent molecular probe.  These initial results clearly show that OPAL can enhance adoption and ease-of-use of fluorescence imaging in oncologic procedures relative to existing state-of-the-art intraoperative imaging systems.
The era of molecular medicine has ushered in the development of microscopic methods that can report molecular processes in thick tissues with high spatial resolution.  A commonality in deep-tissue microscopy is the use of near-infrared (NIR) lasers with single- or multiphoton excitations.  However, the relationship between different NIR excitation microscopic techniques and the imaging depths in tissue has not been established.  We compared such depth limits for three NIR excitation techniques: NIR single-photon confocal microscopy (NIR SPCM), NIR multiphoton excitation with visible detection (NIR/VIS MPM), and all-NIR multiphoton excitation with NIR detection (NIR/NIR MPM).  Homologous cyanine dyes provided the fluorescence.  Intact kidneys were harvested after administration of kidney-clearing cyanine dyes in mice.  NIR SPCM and NIR/VIS MPM achieved similar maximum imaging depth of ~100 μm.  The NIR/NIR MPM enabled greater than fivefold imaging depth (>500 μm) using the harvested kidneys.  Although the NIR/NIR MPM used 1550-nm excitation where water absorption is relatively high, cell viability and histology studies demonstrate that the laser did not induce photothermal damage at the low laser powers used for the kidney imaging.  This study provides guidance on the imaging depth capabilities of NIR excitation-based microscopic techniques and reveals the potential to multiplex information using these platforms.
Position-encoded microsphere arrays are a promising technology for identifying biological targets and quantifying their concentrations.  In this paper we analyze the statistical performance of these arrays in imaging targets at typical low signal-to-noise ratio (SNR) levels.  We compute the Ziv-Zakai bound (ZZB) on the errors in estimating the unknown parameters, including the target concentrations.  We find the SNR level below which the ZZB provides a more accurate prediction of the error than the posterior Cramér-Rao bound (PCRB), through numerical examples.  We further apply the ZZB to select the optimal design parameters of the microsphere array device and investigate the effects of the experimental variables such as microscope point-spread function.  An imaging experiment on microspheres with protein targets verifies the optimal design parameters using the ZZB.
Microarray devices are powerful for detecting and analyzing biological targets.  However, the potential of these devices may not be fully realized due to the lack of optimization of their design and implementation.  In this work, we consider a microsphere-trap array device by employing microfluidic techniques and a hydrodynamic trapping mechanism.  We design a novel geometric structure of the trap array in the device, and develop a comprehensive and robust framework to optimize the values of the geometric parameters to maximize the microsphere arrays' packing density.  We also simultaneously optimize multiple criteria, such as efficiently immobilizing a single microsphere in each trap, effectively eliminating fluidic errors such as channel clogging and multiple microspheres in a single trap, minimizing errors in subsequent imaging experiments, and easily recovering targets.  We use finite element simulations to validate the trapping mechanism of the device, and to study the effects of the optimization geometric parameters.  We further perform microsphere-trapping experiments using the optimized device and a device with randomly selected geometric parameters, which we denote as the un-optimized device.  These experiments demonstrate easy control of the transportation and manipulation of the microspheres in the optimized device.  They also show that the optimized device greatly outperforms the un-optimized device by increasing the packing density by a factor of two, improving the microsphere trapping efficiency from 58% to 99%, and reducing fluidic errors from 48% to a negligible level (less than 1%).  The optimization framework lays the foundation for the future goal of developing a modular, reliable, efficient, and inexpensive lab-on-a-chip system.
We build a microfluidic trap-based microsphere array device.  In the device, we design a novel geometric structure of the trap array and employ the hydrodynamic trapping mechanism to immobilize the microspheres.  We develop a comprehensive and robust framework to optimize the values of the geometric parameters to maximize the microsphere arrays' packing density.  We also simultaneously optimize multiple criteria, such as efficiently immobilizing a single microsphere in each trap, effectively eliminating fluidic errors such as channel clogging and multiple microspheres in a single trap, minimizing errors in subsequent imaging experiments, and easily recovering targets.  Microsphere-trapping experiments have been performed using the optimized device and a device with un-optimized geometric parameters.  These experiments demonstrate easy control of the transportation and manipulation of the microspheres in the optimized device.  They also show that the optimized device greatly outperforms the un-optimized one.
Microsphere arrays can be used to effectively detect, identify, and quantify biological targets, such as mRNAs, proteins, antibodies, and cells.  In this work, we design a microfluidic microsphere-trap array device that enables simultaneous, efficient, and accurate screening of multiple targets on a single platform.  Different types of targets are captured on the surfaces of microspheres of different sizes.  By optimizing the geometric parameters of the traps, the trap arrays in this device can immobilize microspheres of different sizes at different regions with microfluidic hydrodynamic trapping.  The targets are thus detected according to the microspheres' positions (position-encoding), which simplifies screening and avoids errors in target identification.  We validate the design using fluid dynamics finite element simulations by COMSOL Multiphysics software using microsphere of two different sizes.  We also performed preliminary microspheretrapping experiments on a fabricated device using microspheres of one size.  Our results demonstrate that the proposed device can achieve the position-encoding of the microspheres with few fluidic errors.  This device is promising for simultaneous detection of multiple targets and become a cheap and fast disease diagnostic tool.
Although single-photon fluorescence lifetime imaging microscopy (FLIM) is widely used to image molecular processes using a wide range of excitation wavelengths, the captured emission of this technique is confined to the visible spectrum.  Here, we explore the feasibility of utilizing near-infrared (NIR) fluorescent molecular probes with emission >700 nm for FLIM of live cells.  The confocal microscope is equipped with a 785 nm laser diode, a red-enhanced photomultiplier tube, and a time-correlated single photon counting card.  We demonstrate that our system reports the lifetime distributions of NIR fluorescent dyes, cypate and DTTCI, in cells.  In cells labelled separately or jointly with these dyes, NIR FLIM successfully distinguishes their lifetimes, providing a method to sort different cell populations.  In addition, lifetime distributions of cells co-incubated with these dyes allow estimate of the dyes' relative concentrations in complex cellular microenvironments.  With the heightened interest in fluorescence lifetime-based small animal imaging using NIR fluorophores, this technique further serves as a bridge between in vitro spectroscopic characterization of new fluorophore lifetimes and in vivo tissue imaging.
We design microsphere arrays with predetermined positions of microspheres, for capturing targets at low concentrations.  To optimize the design parameters, we compute the Ziv-Zakai bound (ZZB) on the errors in estimating the target concentrations.  We numerically demonstrate our design by computing the minimal distance between the microspheres and the optimal imaging temperature, for a desired level of errors.  We also validate that, at low target concentrations, the statistical design using the ZZB is more precise than that using the posterior Cramér-Rao bound.  We further quantitatively evaluate the effect of the fluorescence microscope point-spread function on the design performance, which provide useful guides to the device design and implementation.  The key advantages of the proposed microsphere arrays are error-free target identification, simplified data analysis, high packing density, and reduced cost.
We propose a microsphere array device with microspheres having controllable positions for error-free target identification.  We conduct a statistical design analysis to select the optimal distance between the microspheres as well as the optimal temperature.  Our design simplifies the imaging and ensures a desired statistical performance for a given sensor cost.  Specifically, we compute the posterior Cramér-Rao bound on the errors in estimating the unknown target concentrations.  We use this performance bound to compute the optimal design variables.  We discuss both uniform and sparse concentration levels of targets, and replace the unknown imaging parameters with their maximum likelihood estimates.  We illustrate our design concept using numerical examples.  The proposed microarray has high sensitivity, efficient packing, and guaranteed imaging performance.  It simplifies the imaging analysis significantly by identifying targets based on the known positions of the microspheres.  Potential applications include molecular recognition, specificity of targeting molecules, protein-protein dimerization, high throughput screening assays for enzyme inhibitors, drug discovery, and gene sequencing.
We modify the Google Page-Rank algorithm, which is primarily used for ranking web pages, to analyze the gene reachability in complex gene co-expression networks.  Our modification is based on the average connections per gene.  We propose a new method to compute the metric of average connections per gene, inspired by the Page-Rank algorithm.  We calculate this average as eight for human genome data and three to seven for yeast genome data.  Our algorithm provides clustering of genes.  The proposed analogy between web pages and genes may offer a new way to interpret gene networks.
In this paper, we propose a gene regulatory network (GRN) estimation method, which assumes that such networks are typically sparse, using time-series microarray datasets.  We represent the regulatory relationships between the genes using weights, with the "net" regulation influence on a gene's expression being the summation of the independent regulatory inputs.  We estimate the weights using a Bayesian linear regression method for sparse parameter vectors.  We apply our proposed method to the extraction of differential gene expression software selected genes of a human buffy-coat microarray expression profile dataset of ventilator-associated pneumonia (VAP), and compare the estimation result with the GRNs estimated using both a correlation coefficient method and a database-based method ingenuity pathway analysis.  A biological analysis of the resulting consensus network that is derived using the GRNs, estimated with both our and the correlation-coefficient methods results in four biologically meaningful subnetworks.  Also, our method performs either better than or competitively with the existing well-established GRN estimation methods.  Moreover, it performs comparatively with respect to: 1) the ground-truth GRNs for the in silico 50- and 100-gene datasets reported recently in the DREAM3 challenge and 2) the GRN estimated using a mutual information-based method for the top-ranked Bayesian analysis of time series (a Bayesian user-friendly software for analyzing time-series microarray experiments) selected genes of the VAP dataset.
We propose a three-dimensional (3D) microarray device with fixed distance between the microspheres.  The microspheres in the new device have controllable positions.  To design the layout of our proposed device, we consider an optimal statistical performance analysis for imaging the microspheres.  We compute the posterior Cramér-Rao bound on the error in estimating the unknown imaging parameters and use the bound to compute the minimal distance between the microspheres, which provides a desired level of statistical imaging performance.  We illustrate our concept using a numerical example.  The proposed microarray has high sensitivity, efficient packing density, and guaranteed imaging performance.  It identifies targets based on the microspheres' locations, allowing significantly simplified imaging analysis.
We develop a maximum likelihood (ML)-based parametric image deconvolution technique to locate quantum-dot (q-dot) encoded microparticles from three-dimensional (3-D) images of an ultra-high density 3-D microarray.  A potential application of the proposed microarray imaging is assay analysis of gene, protein, antigen, and antibody targets.  This imaging is performed using a wide-field fluorescence microscope.  We first describe our problem of interest and the pertinent measurement model by assuming additive Gaussian noise.  We use a 3-D Gaussian point-spread-function (PSF) model to represent the blurring of the widefield microscope system.  We employ parametric spheres to represent the light intensity profiles of the q-dot-encoded microparticles.  We then develop the estimation algorithm for the single-sphere-object image assuming that the microscope PSF is totally unknown.  The algorithm is tested numerically and compared with the analytical Cramér-Rao bounds (CRB).  To apply our analysis to real data, we first segment a section of the blurred 3-D image of the multiple microparticles using a k-means clustering algorithm, obtaining 3-D images of single-sphere-objects.  Then, we process each of these images using our proposed estimation technique.  In the numerical examples, our method outperforms the blind deconvolution (BD) algorithms in high signal-to-noise ratio (SNR) images.  For the case of real data, our method and the BD-based methods perform similarly for the well-separated microparticle images.
In oligonucleotide microarray experiments, noise is a challenging problem, as biologists now are studying their organisms not in isolation but in the context of a natural environment.  In low photomultiplier tube (PMT) voltage images, weak gene signals and their interactions with the background fluorescence noise are most problematic.  In addition, nonspecific sequences bind to array spots intermittently causing inaccurate measurements.  Conventional techniques cannot precisely separate the foreground and the background signals.  In this paper, we propose analytically based estimation technique.  We assume a priori spot-shape information using a circular outer periphery with an elliptical center hole.  We assume Gaussian statistics for modeling both the foreground and background signals.  The mean of the foreground signal quantifies the weak gene signal corresponding to the spot, and the variance gives the measure of the undesired binding that causes fluctuation in the measurement.  We propose a foreground-signal and shape-estimation algorithm using the Gibbs sampling method.  We compare our developed algorithm with the existing Mann-Whitney (MW)- and expectation maximization (EM)/iterated conditional modes (ICM)-based methods.  Our method outperforms the existing methods with considerably smaller mean-square error (MSE) for all signal-to-noise ratios (SNRs) in computer-generated images and gives better qualitative results in low-SNR real-data images.  Our method is computationally relatively slow because of its inherent sampling operation and hence only applicable to very noisy-spot images.  In a realistic example using our method, we show that the gene-signal fluctuations on the estimated foreground are better observed for the input noisy images with relatively higher undesired bindings.
Fluorescence microscopy imaging is widely used in biomedical research, astronomical speckle imaging, remote sensing, positron-emission tomography, and many other applications.  In companion papers, we developed a maximum likelihood (ML)-based image deconvolution technique to quantify fluorescence signals from a three-dimensional (3D) image of a target captured microparticle ensemble.  We assumed both the additive Gaussian and Poisson statistics for the noise.  Imaging is performed by using a confocal fluorescence microscope system.  Potential application of microarray technology includes security, environmental monitoring, analyzing assays for DNA or protein targets, functional genomics, and drug development.  We proposed a new parametric model of the fluorescence microscope 3D point-spread function (PSF) in terms of basis functions.  In this paper, we present a performance analysis of the ML-based deconvolution techniques for both the noise models.
This paper presents an overview of various deconvolution techniques of 3D fluorescence microscopy images.  It describes the subject of image deconvolution for 3D fluorescence microscopy images and provides an overview of the distortion issues in different areas.  The paper presents a brief schematic description of fluorescence microscope systems and provides a summary of the microscope point-spread function (PSF), which often creates the most severe distortion in the acquired 3D image.  Finally, it discusses the ongoing research work in the area and provides a brief review of performance measures of 3D deconvolution microscopy techniques.  It also provides a summary of the numerical results using simulated data and presents the results obtained from the real data.