Front Med (Lausanne). 2024 Jun 25;11:1394262. doi: 10.3389/fmed.2024.1394262. eCollection 2024.

ABSTRACT

Rectal cancer (RC) is a globally prevalent malignant tumor, presenting significant challenges in its management and treatment. Currently, magnetic resonance imaging (MRI) offers superior soft tissue contrast and radiation-free effects for RC patients, making it the most widely used and effective detection method. In early screening, radiologists rely on patients' medical radiology characteristics and their extensive clinical experience for diagnosis. However, diagnostic accuracy may be hindered by factors such as limited expertise, visual fatigue, and image clarity issues, resulting in misdiagnosis or missed diagnosis. Moreover, the distribution of surrounding organs in RC is extensive with some organs having similar shapes to the tumor but unclear boundaries; these complexities greatly impede doctors' ability to diagnose RC accurately. With recent advancements in artificial intelligence, machine learning techniques like deep learning (DL) have demonstrated immense potential and broad prospects in medical image analysis. The emergence of this approach has significantly enhanced research capabilities in medical image classification, detection, and segmentation fields with particular emphasis on medical image segmentation. This review aims to discuss the developmental process of DL segmentation algorithms along with their application progress in lesion segmentation from MRI images of RC to provide theoretical guidance and support for further advancements in this field.

PMID:38983364 | PMC:PMC11231084 | DOI:10.3389/fmed.2024.1394262

Cas Lek Cesk. 2024;162(7-8):283-289.

ABSTRACT

In recent years healthcare is undergoing significant changes due to technological innovations, with Artificial Intelligence (AI) being a key trend. Particularly in radiodiagnostics, according to studies, AI has the potential to enhance accuracy and efficiency. We focus on AI's role in diagnosing pulmonary lesions, which could indicate lung cancer, based on chest X-rays. Despite lower sensitivity in comparison to other methods like chest CT, due to its routine use, X-rays often provide the first detection of lung lesions. We present our deep learning-based solution aimed at improving lung lesion detection, especially during early-stage of illness. We then share results from our previous studies validating this model in two different clinical settings: a general hospital with low prevalence findings and a specialized oncology center. Based on a quantitative comparison with the conclusions of radiologists of different levels of experience, our model achieves high sensitivity, but lower specificity than comparing radiologists. In the context of clinical requirements and AI-assisted diagnostics, the experience and clinical reasoning of the doctor play a crucial role, therefore we currently lean more towards models with higher sensitivity over specificity. Even unlikely suspicions are presented to the doctor. Based on these results, it can be expected that in the future artificial intelligence will play a key role in the field of radiology as a supporting tool for evaluating specialists. To achieve this, it is necessary to solve not only technical but also medical and regulatory aspects. It is crucial to have access to quality and reliable information not only about the benefits but also about the limitations of machine learning and AI in medicine.

PMID:38981713

Cas Lek Cesk. 2024;162(7-8):275-278.

ABSTRACT

The aim of the article to present the development of artificial intelligence (AI) methods and their applications in medicine and health care. Current technological development contributes to generation of large volumes of data that cannot be evaluated only manually. We describe the process of patient care and its individual parts that can be supported by technology and data analysis methods. There are many successful applications that help in the decision support process, in processing complex multidimensional heterogeneous and/or long-term data. On the other side, failures appear in AI methods applications. In recent years, deep learning became very popular and to a certain extend it delivered promising results. However, it has certain flaws that might lead to misclassification. The correct methodological steps in design and implementation of selected methods to data processing are briefly presented.

PMID:38981711

Phys Med Biol. 2024 Jul 9. doi: 10.1088/1361-6560/ad611c. Online ahead of print.

ABSTRACT

OBJECTIVE: Bifurcation detection in intravascular optical coherence tomography (IVOCT) images plays a significant role in guiding optimal revascularization strategies for percutaneous coronary intervention (PCI). We propose a bifurcation detection method using vision transformer (ViT) based deep learning in IVOCT.

APPROACH: Instead of relying on lumen segmentation, the proposed method identifies the bifurcation image using a ViT-based classification model and then estimate bifurcation ostium points by a ViT-based landmark detection model.
Main results.By processing 8640 clinical images, the Accuracy and F1-score of bifurcation identification by the proposed ViT-based model are 2.54% and 16.08% higher than that of traditional non-deep learning methods, are similar to the best performance of convolutional neural networks (CNNs) based methods, respectively. The ostium distance error of the ViT-based model is 0.305mm, which is reduced 68.5% compared with the traditional non-deep learning method and reduced 24.81% compared with the best performance of CNNs based methods. The results also show that the proposed ViT-based method achieves the highest success detection rate (SDR) are 11.3% and 29.2% higher than the non-deep learning method, and 4.6% and 2.5% higher than the best performance of CNNs based methods when the distance threshold is 0.1 mm and 0.2 mm, respectively.

SIGNIFICANCE: The proposed ViT-based method enhances the performance of bifurcation detection of IVOCT images, which maintains a high correlation and consistency between the automatic detection results and the expert manual results. It is of great significance in guiding the selection of PCI treatment strategies.&#xD.

PMID:38981596 | DOI:10.1088/1361-6560/ad611c

Phys Med Biol. 2024 Jul 9. doi: 10.1088/1361-6560/ad611d. Online ahead of print.

ABSTRACT

OBJECTIVE: Deep learning models that aid in medical image assessment tasks must be both accurate and reliable to be deployed within clinical settings. While deep learning models have been shown to be highly accurate across a variety of tasks, measures that indicate the reliability of these models are less established. Increasingly, uncertainty quantification (UQ) methods are being introduced to inform users on the reliability of model outputs. However, most existing methods cannot be augmented to previously validated models because they are not post hoc, and they change a model's output. In this work, we overcome these limitations by introducing a novel post hoc UQ method, termed Local Gradients UQ, and demonstrate its utility for deep learning-based metastatic disease delineation.

APPROACH: This method leverages a trained model's localized gradient space to assess sensitivities to trained model parameters. We compared the Local Gradients UQ method to non-gradient measures defined using model probability outputs. The performance of each uncertainty measure was assessed in four clinically relevant experiments: (1) response to artificially degraded image quality, (2) comparison between matched high- and low-quality clinical images, (3) false positive (FP) filtering, and (4) correspondence with physician-rated disease likelihood.

MAIN RESULTS: (1) Response to artificially degraded image quality was enhanced by the Local Gradients UQ method, where the median percent difference between matching lesions in non-degraded and most degraded images was consistently higher for the Local Gradients uncertainty measure than the non-gradient uncertainty measures (e.g., 62.35% vs. 2.16% for additive Gaussian noise). (2) The Local Gradients UQ measure responded better to high- and low-quality clinical images (p<0.05 vs p>0.1 for both non-gradient uncertainty measures). (3) FP filtering performance was enhanced by the Local Gradients UQ method when compared to the non-gradient methods, increasing the area under the receiver operating characteristic curve (ROC AUC) by 20.1% and decreasing the false positive rate by 26%. (4) The Local Gradients UQ method also showed more favorable correspondence with physician-rated likelihood for malignant lesions by increasing ROC AUC for correspondence with physician-rated disease likelihood by 16.2%.

SIGNIFICANCE: In summary, this work introduces and validates a novel gradient-based UQ method for deep learning-based medical image assessments to enhance user trust when using deployed clinical models.

PMID:38981594 | DOI:10.1088/1361-6560/ad611d

Phys Med Biol. 2024 Jul 9. doi: 10.1088/1361-6560/ad6117. Online ahead of print.

ABSTRACT

OBJECTIVE: Positron Emission Tomography and Magnetic Resonance Imaging (PET-MRI) systems can obtain functional and anatomical scans. But PET suffers from a low signal-to-noise ratio, while MRI are time-consuming. To address time-consuming, an effective strategy involves reducing k-space data collection, albeit at the cost of lowering image quality. This study aims to leverage the inherent complementarity within PET-MRI data to enhance the image quality of PET-MRI. Apporach: A novel PET-MRI joint reconstruction model, termed MC-Diffusion, is proposed in the Bayesian framework. The joint reconstruction problem is transformed into a joint regularization problem, where data fidelity terms of PET and MRI are expressed independently. The regular term, the derivative of the logarithm of the joint probability distribution of PET and MRI, employs a joint score-based diffusion model for learning. The diffusion model involves the forward diffusion process and the reverse diffusion process. The forward diffusion process adds noise to transform a complex joint data distribution into a known joint prior distribution for PET and MRI simultaneously, resembling a denoiser. The reverse diffusion process removes noise using a denoiser to revert the joint prior distribution to the original joint data distribution, effectively utilizing joint probability distribution to describe the correlations of PET and MRI for improved quality of joint reconstruction.

MAIN RESULTS: Qualitative and quantitative improvements are observed with the MC-Diffusion model. Comparative analysis against LPLS and Joint ISAT-net on the ADNI dataset demonstrates superior performance by exploiting complementary information between PET and MRI. The MC-Diffusion model effectively enhances the quality of PET and MRI images.

SIGNIFICANCE: This study employs the MC-Diffusion model to enhance the quality of PET-MRI images by integrating the fundamental principles of PET and MRI modalities and their inherent complementarity. The MC-Diffusion model facilitates a more profound comprehension of the priors obtained through deep learning.

PMID:38981592 | DOI:10.1088/1361-6560/ad6117

Cell Rep Methods. 2024 Jul 5:100817. doi: 10.1016/j.crmeth.2024.100817. Online ahead of print.

ABSTRACT

Deep-learning tools that extract prognostic factors derived from multi-omics data have recently contributed to individualized predictions of survival outcomes. However, the limited size of integrated omics-imaging-clinical datasets poses challenges. Here, we propose two biologically interpretable and robust deep-learning architectures for survival prediction of non-small cell lung cancer (NSCLC) patients, learning simultaneously from computed tomography (CT) scan images, gene expression data, and clinical information. The proposed models integrate patient-specific clinical, transcriptomic, and imaging data and incorporate Kyoto Encyclopedia of Genes and Genomes (KEGG) and Reactome pathway information, adding biological knowledge within the learning process to extract prognostic gene biomarkers and molecular pathways. While both models accurately stratify patients in high- and low-risk groups when trained on a dataset of only 130 patients, introducing a cross-attention mechanism in a sparse autoencoder significantly improves the performance, highlighting tumor regions and NSCLC-related genes as potential biomarkers and thus offering a significant methodological advancement when learning from small imaging-omics-clinical samples.

PMID:38981473 | DOI:10.1016/j.crmeth.2024.100817

Med Image Anal. 2024 Jul 1;97:103257. doi: 10.1016/j.media.2024.103257. Online ahead of print.

ABSTRACT

The alignment of tissue between histopathological whole-slide-images (WSI) is crucial for research and clinical applications. Advances in computing, deep learning, and availability of large WSI datasets have revolutionised WSI analysis. Therefore, the current state-of-the-art in WSI registration is unclear. To address this, we conducted the ACROBAT challenge, based on the largest WSI registration dataset to date, including 4,212 WSIs from 1,152 breast cancer patients. The challenge objective was to align WSIs of tissue that was stained with routine diagnostic immunohistochemistry to its H&E-stained counterpart. We compare the performance of eight WSI registration algorithms, including an investigation of the impact of different WSI properties and clinical covariates. We find that conceptually distinct WSI registration methods can lead to highly accurate registration performances and identify covariates that impact performances across methods. These results provide a comparison of the performance of current WSI registration methods and guide researchers in selecting and developing methods.

PMID:38981282 | DOI:10.1016/j.media.2024.103257

Comput Biol Med. 2024 Jul 8;179:108826. doi: 10.1016/j.compbiomed.2024.108826. Online ahead of print.

ABSTRACT

Researchers face the challenge of defining subject selection criteria when training algorithms for human activity recognition tasks. The ongoing uncertainty revolves around which characteristics should be considered to ensure algorithmic robustness across diverse populations. This study aims to address this challenge by conducting an analysis of heterogeneity in the training data to assess the impact of physical characteristics and soft-biometric attributes on activity recognition performance. The performance of various state-of-the-art deep neural network architectures (tCNN, hybrid-LSTM, Transformer model) processing time-series data using the IntelliRehab (IRDS) dataset was evaluated. By intentionally introducing bias into the training data based on human characteristics, the objective is to identify the characteristics that influence algorithms in motion analysis. Experimental findings reveal that the CNN-LSTM model achieved the highest accuracy, reaching 88%. Moreover, models trained on heterogeneous distributions of disability attributes exhibited notably higher accuracy, reaching 51%, compared to those not considering such factors, which scored an average of 33%. These evaluations underscore the significant influence of subjects' characteristics on activity recognition performance, providing valuable insights into the algorithm's robustness across diverse populations. This study represents a significant step forward in promoting fairness and trustworthiness in artificial intelligence by quantifying representation bias in multi-channel time-series activity recognition data within the healthcare domain.

PMID:38981215 | DOI:10.1016/j.compbiomed.2024.108826

Radiat Oncol. 2024 Jul 9;19(1):89. doi: 10.1186/s13014-024-02467-w.

ABSTRACT

BACKGROUND AND PURPOSE: To investigate the feasibility of synthesizing computed tomography (CT) images from magnetic resonance (MR) images in multi-center datasets using generative adversarial networks (GANs) for rectal cancer MR-only radiotherapy.

MATERIALS AND METHODS: Conventional T2-weighted MR and CT images were acquired from 90 rectal cancer patients at Peking University People's Hospital and 19 patients in public datasets. This study proposed a new model combining contrastive learning loss and consistency regularization loss to enhance the generalization of model for multi-center pelvic MRI-to-CT synthesis. The CT-to-sCT image similarity was evaluated by computing the mean absolute error (MAE), peak signal-to-noise ratio (SNRpeak), structural similarity index (SSIM) and Generalization Performance (GP). The dosimetric accuracy of synthetic CT was verified against CT-based dose distributions for the photon plan. Relative dose differences in the planning target volume and organs at risk were computed.

RESULTS: Our model presented excellent generalization with a GP of 0.911 on unseen datasets and outperformed the plain CycleGAN, where MAE decreased from 47.129 to 42.344, SNRpeak improved from 25.167 to 26.979, SSIM increased from 0.978 to 0.992. The dosimetric analysis demonstrated that most of the relative differences in dose and volume histogram (DVH) indicators between synthetic CT and real CT were less than 1%.

CONCLUSION: The proposed model can generate accurate synthetic CT in multi-center datasets from T2w-MR images. Most dosimetric differences were within clinically acceptable criteria for photon radiotherapy, demonstrating the feasibility of an MRI-only workflow for patients with rectal cancer.

PMID:38982452 | DOI:10.1186/s13014-024-02467-w

BMC Med Imaging. 2024 Jul 9;24(1):170. doi: 10.1186/s12880-024-01352-y.

ABSTRACT

OBJECTIVES: To develop and validate a novel interpretable artificial intelligence (AI) model that integrates radiomic features, deep learning features, and imaging features at multiple semantic levels to predict the prognosis of intracerebral hemorrhage (ICH) patients at 6 months post-onset.

MATERIALS AND METHODS: Retrospectively enrolled 222 patients with ICH for Non-contrast Computed Tomography (NCCT) images and clinical data, who were divided into a training cohort (n = 186, medical center 1) and an external testing cohort (n = 36, medical center 2). Following image preprocessing, the entire hematoma region was segmented by two radiologists as the volume of interest (VOI). Pyradiomics algorithm library was utilized to extract 1762 radiomics features, while a deep convolutional neural network (EfficientnetV2-L) was employed to extract 1000 deep learning features. Additionally, radiologists evaluated imaging features. Based on the three different modalities of features mentioned above, the Random Forest (RF) model was trained, resulting in three models (Radiomics Model, Radiomics-Clinical Model, and DL-Radiomics-Clinical Model). The performance and clinical utility of the models were assessed using the Area Under the Receiver Operating Characteristic Curve (AUC), calibration curve, and Decision Curve Analysis (DCA), with AUC compared using the DeLong test. Furthermore, this study employs three methods, Shapley Additive Explanations (SHAP), Grad-CAM, and Guided Grad-CAM, to conduct a multidimensional interpretability analysis of model decisions.

RESULTS: The Radiomics-Clinical Model and DL-Radiomics-Clinical Model exhibited relatively good predictive performance, with an AUC of 0.86 [95% Confidence Intervals (CI): 0.71, 0.95; P < 0.01] and 0.89 (95% CI: 0.74, 0.97; P < 0.01), respectively, in the external testing cohort.

CONCLUSION: The multimodal explainable AI model proposed in this study can accurately predict the prognosis of ICH. Interpretability methods such as SHAP, Grad-CAM, and Guided Grad-Cam partially address the interpretability limitations of AI models. Integrating multimodal imaging features can effectively improve the performance of the model.

CLINICAL RELEVANCE STATEMENT: Predicting the prognosis of patients with ICH is a key objective in emergency care. Accurate and efficient prognostic tools can effectively prevent, manage, and monitor adverse events in ICH patients, maximizing treatment outcomes.

PMID:38982357 | DOI:10.1186/s12880-024-01352-y

Adv Sci (Weinh). 2024 Jul 9:e2404211. doi: 10.1002/advs.202404211. Online ahead of print.

ABSTRACT

Dysphagia is more common in conditions such as stroke, Parkinson's disease, and head and neck cancer. This can lead to pneumonia, choking, malnutrition, and dehydration. Currently, the diagnostic gold standard uses radiologic imaging, the videofluoroscopic swallow study (VFSS); however, it is expensive and necessitates specialized facilities and trained personnel. Although several devices attempt to address the limitations, none offer the clinical-grade quality and accuracy of the VFSS. Here, this study reports a wireless multimodal wearable system with machine learning for automatic, accurate clinical assessment of swallowing behavior and diagnosis of silent aspirations from dysphagia patients. The device includes a kirigami-structured electrode that suppresses changes in skin contact impedance caused by movements and a microphone with a gel layer that effectively blocks external noise for measuring high-quality electromyograms and swallowing sounds. The deep learning algorithm offers the classification of swallowing patterns while diagnosing silent aspirations, with an accuracy of 89.47%. The demonstration with post-stroke patients captures the system's significance in measuring multiple physiological signals in real-time for detecting swallowing disorders, validated by comparing them with the VFSS. The multimodal electronics can ensure a promising future for dysphagia healthcare and rehabilitation therapy, providing an accurate, non-invasive alternative for monitoring swallowing and aspiration events.

PMID:38981027 | DOI:10.1002/advs.202404211

ESC Heart Fail. 2024 Jul 9. doi: 10.1002/ehf2.14918. Online ahead of print.

ABSTRACT

AIMS: Assessing the risk for HF rehospitalization is important for managing and treating patients with HF. To address this need, various risk prediction models have been developed. However, none of them used deep learning methods with real-world data. This study aimed to develop a deep learning-based prediction model for HF rehospitalization within 30, 90, and 365 days after acute HF (AHF) discharge.

METHODS AND RESULTS: We analysed the data of patients admitted due to AHF between January 2014 and January 2019 in a tertiary hospital. In performing deep learning-based predictive algorithms for HF rehospitalization, we use hyperbolic tangent activation layers followed by recurrent layers with gated recurrent units. To assess the readmission prediction, we used the AUC, precision, recall, specificity, and F1 measure. We applied the Shapley value to identify which features contributed to HF readmission. Twenty-two prognostic features exhibiting statistically significant associations with HF rehospitalization were identified, consisting of 6 time-independent and 16 time-dependent features. The AUC value shows moderate discrimination for predicting readmission within 30, 90, and 365 days of follow-up (FU) (AUC:0.63, 0.74, and 0.76, respectively). The features during the FU have a relatively higher contribution to HF rehospitalization than features from other time points.

CONCLUSIONS: Our deep learning-based model using real-world data could provide valid predictions of HF rehospitalization in 1 year follow-up. It can be easily utilized to guide appropriate interventions or care strategies for patients with HF. The closed monitoring and blood test in daily clinics are important for assessing the risk of HF rehospitalization.

PMID:38981003 | DOI:10.1002/ehf2.14918

ACS Appl Mater Interfaces. 2024 Jul 9. doi: 10.1021/acsami.4c04223. Online ahead of print.

ABSTRACT

Intelligent colorimetric freshness indicator is a low-cost way to intuitively monitor the freshness of fresh food. A colorimetric strip sensor array was prepared by p-dimethylaminocinnamaldehyde (PDL)-doped poly(vinyl alcohol) (PVA) and chitosan (Chit) for the quantitative analysis of indole, which is an indicator of shrimp freshness. As a result of indole simulation, the array strip turned from faint yellow to pink or mulberry color with the increasing indole concentration, like a progress bar. The indicator film exhibited excellent permeability, mechanical and thermal stability, and color responsiveness to indole, which was attributed to the interactions between PDL and Chit/PVA. Furthermore, the colorimetric strip sensor array provided a good relationship between the indole concentration and the color intensity within a range of 50-350 ppb. The pathogens and spoilage bacteria of shrimp possessed the ability to produce indole, which caused the color changes of the strip sensor array. In the shrimp freshness monitoring experiment, the color-changing progress of the strip sensor array was in agreement with the simulation and could distinguish the shrimp freshness levels. The image classification system based on deep learning were developed, the accuracies of four DCNN algorithms are above 90%, with VGG16 achieving the highest accuracy at 97.89%. Consequently, a "progress bar" strip sensor array has the potential to realize nondestructive, more precise, and commercially available food freshness monitoring using simple visual inspection and intelligent equipment identification.

PMID:38980942 | DOI:10.1021/acsami.4c04223

PLoS One. 2024 Jul 9;19(7):e0293777. doi: 10.1371/journal.pone.0293777. eCollection 2024.

ABSTRACT

An improved algorithm based on Yolov8s is proposed to address the slower speed, higher number of parameters, and larger computational cost of deep learning in coal gangue target detection. A lightweight network, Fasternet, is used as the backbone to increase the speed of object detection and reduce the model complexity. By replacing Slimneck with the C2F part in the HEAD module, the aim is to reduce model complexity and improve detection accuracy. The detection accuracy is effectively improved by replacing the Detect layer with Detect-DyHead. The introduction of DIoU loss function instead of CIoU loss function and the combination of BAM block attention mechanism makes the model pay more attention to critical features, which further improves the detection performance. The results show that the improved model compresses the storage size of the model by 28%, reduces the number of parameters by 28.8%, reduces the computational effort by 34.8%, and improves the detection accuracy by 2.5% compared to the original model. The Yolov8s-change model provides a fast, real-time and efficient detection solution for gangue sorting. This provides a strong support for the intelligent sorting of coal gangue.

PMID:38980881 | DOI:10.1371/journal.pone.0293777

IEEE Trans Neural Netw Learn Syst. 2024 Jul 9;PP. doi: 10.1109/TNNLS.2024.3419250. Online ahead of print.

ABSTRACT

Protein function prediction is crucial for understanding species evolution, including viral mutations. Gene ontology (GO) is a standardized representation framework for describing protein functions with annotated terms. Each ontology is a specific functional category containing multiple child ontologies, and the relationships of parent and child ontologies create a directed acyclic graph. Protein functions are categorized using GO, which divides them into three main groups: cellular component ontology, molecular function ontology, and biological process ontology. Therefore, the GO annotation of protein is a hierarchical multilabel classification problem. This hierarchical relationship introduces complexities such as mixed ontology problem, leading to performance bottlenecks in existing computational methods due to label dependency and data sparsity. To overcome bottleneck issues brought by mixed ontology problem, we propose ProFun-SOM, an innovative multilabel classifier that utilizes multiple sequence alignments (MSAs) to accurately annotate gene ontologies. ProFun-SOM enhances the initial MSAs through a reconstruction process and integrates them into a deep learning architecture. It then predicts annotations within the cellular component, molecular function, biological process, and mixed ontologies. Our evaluation results on three datasets (CAFA3, SwissProt, and NetGO2) demonstrate that ProFun-SOM surpasses state-of-the-art methods. This study confirmed that utilizing MSAs of proteins can effectively overcome the two main bottlenecks issues, label dependency and data sparsity, thereby alleviating the root problem, mixed ontology. A freely accessible web server is available at http://bliulab.net/ ProFun-SOM/.

PMID:38980781 | DOI:10.1109/TNNLS.2024.3419250

IEEE J Biomed Health Inform. 2024 Jul 9;PP. doi: 10.1109/JBHI.2024.3425434. Online ahead of print.

ABSTRACT

Image analysis can play an important role in supporting histopathological diagnoses of lung cancer, with deep learning methods already achieving remarkable results. However, due to the large scale of whole-slide images (WSIs), creating manual pixel-wise annotations from expert pathologists is expensive and time-consuming. In addition, the heterogeneity of tumors and similarities in the morphological phenotype of tumor subtypes have caused inter-observer variability in annotations, which limits optimal performance. Effective use of weak labels could potentially alleviate these issues. In this paper, we propose a two-stage transformer-based weakly supervised learning framework called Simple Shuffle-Remix Vision Transformer (SSRViT). Firstly, we introduce a Shuffle-Remix Vision Transformer (SRViT) to retrieve discriminative local tokens and extract effective representative features. Then, the token features are selected and aggregated to generate sparse representations of WSIs, which are fed into a simple transformer-based classifier (SViT) for slide-level prediction. Experimental results demonstrate that the performance of our proposed SSRViT is significantly improved compared with other state-of-the-art methods in discriminating between adenocarcinoma, pulmonary sclerosing pneumocytoma and normal lung tissue (accuracy of 96.9% and AUC of 99.6%).

PMID:38980777 | DOI:10.1109/JBHI.2024.3425434

IEEE J Biomed Health Inform. 2024 Jul 9;PP. doi: 10.1109/JBHI.2024.3424869. Online ahead of print.

ABSTRACT

Marker-based motion capture (mocap) is a conventional method used in biomechanics research to precisely analyze human movement. However, the time-consuming marker placement process and extensive post-processing limit its wider adoption. Therefore, markerless mocap systems that use deep learning to estimate 2D keypoint from images have emerged as a promising alternative, but annotation errors in training datasets used by deep learning models can affect estimation accuracy. To improve the precision of 2D keypoint annotation, we present a method that uses anatomical landmarks based on marker-based mocap. Specifically, we use multiple RGB cameras synchronized and calibrated with a marker-based mocap system to create a high-quality dataset (RRIS40) of images annotated with surface anatomical landmarks. A deep neural network is then trained to estimate these 2D anatomical landmarks and a ray-distance-based triangulation is used to calculate the 3D marker positions. We conducted extensive evaluations on our RRIS40 test set, which consists of 10 subjects performing various movements. Compared against a marker-based system, our method achieves a mean Euclidean error of 13.23 mm in 3D marker position, which is comparable to the precision of marker placement itself. By learning directly to predict anatomical keypoints from images, our method outperforms OpenCap's augmentation of 3D anatomical landmarks from triangulated wild keypoints. This highlights the potential of facilitating wider integration of markerless mocap into biomechanics research. The RRIS40 test set is made publicly available for research purposes at koonyook.github.io/rris40.

PMID:38980775 | DOI:10.1109/JBHI.2024.3424869

J Imaging Inform Med. 2024 Jul 9. doi: 10.1007/s10278-024-01128-4. Online ahead of print.

ABSTRACT

Accurate image classification and retrieval are of importance for clinical diagnosis and treatment decision-making. The recent contrastive language-image pre-training (CLIP) model has shown remarkable proficiency in understanding natural images. Drawing inspiration from CLIP, pathology-dedicated CLIP (PathCLIP) has been developed, utilizing over 200,000 image and text pairs in training. While the performance the PathCLIP is impressive, its robustness under a wide range of image corruptions remains unknown. Therefore, we conduct an extensive evaluation to analyze the performance of PathCLIP on various corrupted images from the datasets of osteosarcoma and WSSS4LUAD. In our experiments, we introduce eleven corruption types including brightness, contrast, defocus, resolution, saturation, hue, markup, deformation, incompleteness, rotation, and flipping at various settings. Through experiments, we find that PathCLIP surpasses OpenAI-CLIP and the pathology language-image pre-training (PLIP) model in zero-shot classification. It is relatively robust to image corruptions including contrast, saturation, incompleteness, and orientation factors. Among the eleven corruptions, hue, markup, deformation, defocus, and resolution can cause relatively severe performance fluctuation of the PathCLIP. This indicates that ensuring the quality of images is crucial before conducting a clinical test. Additionally, we assess the robustness of PathCLIP in the task of image-to-image retrieval, revealing that PathCLIP performs less effectively than PLIP on osteosarcoma but performs better on WSSS4LUAD under diverse corruptions. Overall, PathCLIP presents impressive zero-shot classification and retrieval performance for pathology images, but appropriate care needs to be taken when using it.

PMID:38980627 | DOI:10.1007/s10278-024-01128-4

J Imaging Inform Med. 2024 Jul 9. doi: 10.1007/s10278-024-01181-z. Online ahead of print.

ABSTRACT

Malposition of a nasogastric tube (NGT) can lead to severe complications. We aimed to develop a computer-aided detection (CAD) system to localize NGTs and detect NGT malposition on portable chest X-rays (CXRs). A total of 7378 portable CXRs were retrospectively retrieved from two hospitals between 2015 and 2020. All CXRs were annotated with pixel-level labels for NGT localization and image-level labels for NGT presence and malposition. In the CAD system, DeepLabv3 + with backbone ResNeSt50 and DenseNet121 served as the model architecture for segmentation and classification models, respectively. The CAD system was tested on images from chronologically different datasets (National Taiwan University Hospital (National Taiwan University Hospital)-20), geographically different datasets (National Taiwan University Hospital-Yunlin Branch (YB)), and the public CLiP dataset. For the segmentation model, the Dice coefficients indicated accurate delineation of the NGT course (National Taiwan University Hospital-20: 0.665, 95% confidence interval (CI) 0.630-0.696; National Taiwan University Hospital-Yunlin Branch: 0.646, 95% CI 0.614-0.678). The distance between the predicted and ground-truth NGT tips suggested accurate tip localization (National Taiwan University Hospital-20: 1.64 cm, 95% CI 0.99-2.41; National Taiwan University Hospital-Yunlin Branch: 2.83 cm, 95% CI 1.94-3.76). For the classification model, NGT presence was detected with high accuracy (area under the receiver operating characteristic curve (AUC): National Taiwan University Hospital-20: 0.998, 95% CI 0.995-1.000; National Taiwan University Hospital-Yunlin Branch: 0.998, 95% CI 0.995-1.000; CLiP dataset: 0.991, 95% CI 0.990-0.992). The CAD system also detected NGT malposition with high accuracy (AUC: National Taiwan University Hospital-20: 0.964, 95% CI 0.917-1.000; National Taiwan University Hospital-Yunlin Branch: 0.991, 95% CI 0.970-1.000) and detected abnormal nasoenteric tube positions with favorable performance (AUC: 0.839, 95% CI 0.807-0.869). The CAD system accurately localized NGTs and detected NGT malposition, demonstrating excellent potential for external generalizability.

PMID:38980623 | DOI:10.1007/s10278-024-01181-z