Sci Rep. 2024 Sep 5;14(1):20754. doi: 10.1038/s41598-024-71795-4.

ABSTRACT

To ensure the reliability of machining quality, it is crucial to predict tool wear accurately. In this paper, a novel deep learning-based model is proposed, which synthesizes the advantages of power spectral density (PSD), convolutional neural networks (CNN), and vision transformer model (ViT), namely PSD-CVT. PSD maps can provide a comprehensive understanding of the spectral characteristics of the signals. It makes the spectral characteristics more obvious and makes it easy to analyze and compare different signals. CNN focuses on local feature extraction, which can capture local information such as the texture, edge, and shape of the image, while the attention mechanism in ViT can effectively capture the global structure and long-range dependencies present in the image. Two fully connected layers with a ReLU function are used to obtain the predicted tool wear values. The experimental results on the PHM 2010 dataset demonstrate that the proposed model has higher prediction accuracy than the CNN model or ViT model alone, as well as outperforms several existing methods in accurately predicting tool wear. The proposed prediction method can also be applied to predict tool wear in other machining fields.

PMID:39237695 | DOI:10.1038/s41598-024-71795-4

NPJ Precis Oncol. 2024 Sep 5;8(1):189. doi: 10.1038/s41698-024-00678-8.

ABSTRACT

Pathological complete response (pCR) serves as a critical measure of the success of neoadjuvant chemotherapy (NAC) in breast cancer, directly influencing subsequent therapeutic decisions. With the continuous advancement of artificial intelligence, methods for early and accurate prediction of pCR are being extensively explored. In this study, we propose a cross-modal multi-pathway automated prediction model that integrates temporal and spatial information. This model fuses digital pathology images from biopsy specimens and multi-temporal ultrasound (US) images to predict pCR status early in NAC. The model demonstrates exceptional predictive efficacy. Our findings lay the foundation for developing personalized treatment paradigms based on individual responses. This approach has the potential to become a critical auxiliary tool for the early prediction of NAC response in breast cancer patients.

PMID:39237596 | DOI:10.1038/s41698-024-00678-8

Sci Rep. 2024 Sep 5;14(1):20685. doi: 10.1038/s41598-024-71775-8.

ABSTRACT

Radio frequency interference (RFI) poses challenges in the analysis of synthetic aperture radar (SAR) images. Existing RFI suppression systems rely on prior knowledge of the presence of RFI. This paper proposes a lightweight neural network-based algorithm for detecting and segmenting RFI (LDNet) in the time-frequency domain. The network accurately delineates RFI pixel regions in time-frequency spectrograms. To mitigate the impact on the operational speed of the entire RFI suppression system, lightweight modules and pruning operations are introduced. Compared to threshold-based RFI detection algorithms, deep learning-based segmentation networks, and AC-UNet specifically designed for RFI detection, LDNet achieves improvements in mean intersection over union (MIoU) by 24.56%, 13.29%, and 7.54%, respectively.Furthermore, LDNet reduces model size by 99.03% and inference latency by 24.53% compared to AC-UNet.

PMID:39237592 | DOI:10.1038/s41598-024-71775-8

Sci Rep. 2024 Sep 6;14(1):20774. doi: 10.1038/s41598-024-71020-2.

ABSTRACT

Type 2 diabetes mellitus (T2DM) is a prevalent health challenge faced by countries worldwide. In this study, we propose a novel large language multimodal models (LLMMs) framework incorporating multimodal data from clinical notes and laboratory results for diabetes risk prediction. We collected five years of electronic health records (EHRs) dating from 2017 to 2021 from a Taiwan hospital database. This dataset included 1,420,596 clinical notes, 387,392 laboratory results, and more than 1505 laboratory test items. Our method combined a text embedding encoder and multi-head attention layer to learn laboratory values, and utilized a deep neural network (DNN) module to merge blood features with chronic disease semantics into a latent space. In our experiments, we observed that integrating clinical notes with predictions based on textual laboratory values significantly enhanced the predictive capability of the unimodal model in the early detection of T2DM. Moreover, we achieved an area greater than 0.70 under the receiver operating characteristic curve (AUC) for new-onset T2DM prediction, demonstrating the effectiveness of leveraging textual laboratory data for training and inference in LLMs and improving the accuracy of new-onset diabetes prediction.

PMID:39237580 | DOI:10.1038/s41598-024-71020-2

Light Sci Appl. 2024 Sep 4;13(1):231. doi: 10.1038/s41377-024-01544-9.

ABSTRACT

In recent years, the integration of deep learning techniques with biophotonic setups has opened new horizons in bioimaging. A compelling trend in this field involves deliberately compromising certain measurement metrics to engineer better bioimaging tools in terms of e.g., cost, speed, and form-factor, followed by compensating for the resulting defects through the utilization of deep learning models trained on a large amount of ideal, superior or alternative data. This strategic approach has found increasing popularity due to its potential to enhance various aspects of biophotonic imaging. One of the primary motivations for employing this strategy is the pursuit of higher temporal resolution or increased imaging speed, critical for capturing fine dynamic biological processes. Additionally, this approach offers the prospect of simplifying hardware requirements and complexities, thereby making advanced imaging standards more accessible in terms of cost and/or size. This article provides an in-depth review of the diverse measurement aspects that researchers intentionally impair in their biophotonic setups, including the point spread function (PSF), signal-to-noise ratio (SNR), sampling density, and pixel resolution. By deliberately compromising these metrics, researchers aim to not only recuperate them through the application of deep learning networks, but also bolster in return other crucial parameters, such as the field of view (FOV), depth of field (DOF), and space-bandwidth product (SBP). Throughout this article, we discuss various biophotonic methods that have successfully employed this strategic approach. These techniques span a wide range of applications and showcase the versatility and effectiveness of deep learning in the context of compromised biophotonic data. Finally, by offering our perspectives on the exciting future possibilities of this rapidly evolving concept, we hope to motivate our readers from various disciplines to explore novel ways of balancing hardware compromises with compensation via artificial intelligence (AI).

PMID:39237561 | DOI:10.1038/s41377-024-01544-9

Int Dent J. 2024 Sep 4:S0020-6539(24)01469-2. doi: 10.1016/j.identj.2024.08.010. Online ahead of print.

ABSTRACT

This study aims to provide an overview of the global research trends in the detection and diagnosis of dental caries in the past 20 years. A literature search was conducted in the Scopus Database to retrieve studies on the diagnostic approaches for dental caries published from January 2003 to December 2023. The diagnostic approaches in the retrieved studies were examined and the studies were categorized according to the diagnostic approaches investigated. Bibliometric data including journals, countries, affiliations, authors, and numbers of citations of the publications were summarised. The publications' keyword co-occurrence was analysed using VOSviewer. This bibliometric analysis included 1879 publications investigating seven categories of caries diagnostic approaches, including visual and/or tactile (n = 459; 19%), radiation-based (n = 662; 27%), light-based (n = 771; 32%), ultrasound-based (n = 28; 1%), electric-based (n = 51; 2%), molecular-based (n = 196; 8%) diagnostic approaches, as well as AI-based diagnostic interpretation aids (n = 265; 11%). An increase in the annual number of publications on caries diagnostic approaches was observed in the past 20 years. Caries Research (n = 103) presented the highest number of publications on caries diagnostic approaches. The country with the highest number of publications was the United States (n = 1092). The University of São Paulo was the institution that published the highest number of articles (n = 195). The publication with the highest citation has been cited 932 times. VOS viewer revealed that the most frequently occurring keywords were 'Deep Learning', 'Artificial Intelligence', 'Laser Fluorescence' and 'Radiography'. This bibliometric analysis highlighted an emerging global research trend in the detection and diagnosis approaches for dental caries in the past 20 years. An evident increase in publications on molecular-based caries diagnostic approaches and AI-based diagnostic interpretation aids was perceived over the last 5 years.

PMID:39237399 | DOI:10.1016/j.identj.2024.08.010

ISA Trans. 2024 Sep 2:S0019-0578(24)00408-7. doi: 10.1016/j.isatra.2024.08.027. Online ahead of print.

ABSTRACT

Deep learning has been increasingly used in health management and maintenance decision-making for rotating machinery. However, some challenges must be addressed to make this technology more effective. For example, the collected data is assumed to follow the same feature distribution, and sufficient labeled training data are available. Unfortunately, domain shifts occur inevitably in real-world scenarios due to different working conditions, and acquiring sufficient labeled samples is time-consuming and expensive in complex environments. This study proposes a novel domain adaptive framework called deep Multiscale Conditional Adversarial Networks (MCAN) for machinery fault diagnosis to address these shortcomings. The MCAN model comprises two key components. Constructed by a novel multiscale module with an attention mechanism, the first component is a shared feature generator that captures rich features at different internal perceptual scales, and the attention mechanism determines the weights assigned to each scale, enhancing the model's dynamic adjustment and self-adaptation capabilities. The second component consists of two domain classifiers based on Bidirectional Long Short-Term Memory (BiLSTM) leveraging spatiotemporal features at various levels to achieve domain adaptation in the output space. The deep domain classifier also captures the cross-covariance dependencies between feature representations and classifier predictions, thereby improving the predictions' discriminability. The proposed method has been evaluated using two publicly available fault diagnosis datasets and one condition monitoring experiment. The results of cross-domain transfer tasks demonstrated that the proposed method outperformed several state-of-the-art methods in terms of transferability and stability. This result is a significant step forward in deep learning for health management and maintenance decision-making for rotating machinery, and it has the potential to revolutionize its future application.

PMID:39237396 | DOI:10.1016/j.isatra.2024.08.027

Proc Natl Acad Sci U S A. 2024 Sep 10;121(37):e2318296121. doi: 10.1073/pnas.2318296121. Epub 2024 Sep 5.

ABSTRACT

Anthropogenic habitat destruction and climate change are reshaping the geographic distribution of plants worldwide. However, we are still unable to map species shifts at high spatial, temporal, and taxonomic resolution. Here, we develop a deep learning model trained using remote sensing images from California paired with half a million citizen science observations that can map the distribution of over 2,000 plant species. Our model-Deepbiosphere-not only outperforms many common species distribution modeling approaches (AUC 0.95 vs. 0.88) but can map species at up to a few meters resolution and finely delineate plant communities with high accuracy, including the pristine and clear-cut forests of Redwood National Park. These fine-scale predictions can further be used to map the intensity of habitat fragmentation and sharp ecosystem transitions across human-altered landscapes. In addition, from frequent collections of remote sensing data, Deepbiosphere can detect the rapid effects of severe wildfire on plant community composition across a 2-y time period. These findings demonstrate that integrating public earth observations and citizen science with deep learning can pave the way toward automated systems for monitoring biodiversity change in real-time worldwide.

PMID:39236239 | DOI:10.1073/pnas.2318296121

IEEE Trans Image Process. 2024 Sep 5;PP. doi: 10.1109/TIP.2024.3451939. Online ahead of print.

ABSTRACT

Image restoration (IR) via deep learning has been vigorously studied in recent years. However, due to the ill-posed nature of the problem, it is challenging to recover the high-quality image details from a single distorted input especially when images are corrupted by multiple distortions. In this paper, we propose a multi-stage IR approach for progressive restoration of multi-degraded images via transferring similar edges/textures from the reference image. Our method, called a Reference-based Image Restoration Transformer (Ref-IRT), operates via three main stages. In the first stage, a cascaded U-Transformer network is employed to perform the preliminary recovery of the image. The proposed network consists of two U-Transformer architectures connected by feature fusion of the encoders and decoders, and the residual image is estimated by each U-Transformer in an easy-to-hard and coarse-to-fine fashion to gradually recover the high-quality image. The second and third stages perform texture transfer from a reference image to the preliminarily-recovered target image to further enhance the restoration performance. To this end, a quality-degradation-restoration method is proposed for more accurate content/texture matching between the reference and target images, and a texture transfer/reconstruction network is employed to map the transferred features to the high-quality image. Experimental results tested on three benchmark datasets demonstrate the effectiveness of our model as compared with other state-of-the-art multi-degraded IR methods. Our code and dataset are available at https://vinelab.jp/refmdir/.

PMID:39236125 | DOI:10.1109/TIP.2024.3451939

Forensic Sci Med Pathol. 2024 Sep 5. doi: 10.1007/s12024-024-00877-6. Online ahead of print.

ABSTRACT

Shoe prints are one of the most common types of evidence found at crime scenes, second only to fingerprints. However, studies involving modern approaches such as machine learning and deep learning for the detection and analysis of shoe prints are quite limited in this field. With advancements in technology, positive results have recently emerged for the detection of 2D shoe prints. However, few studies focusing on 3D shoe prints. This study aims to use deep learning methods, specifically the PointNet architecture, for binary classification applications of 3D shoe prints, utilizing two different shoe brands. A 3D dataset created from 160 pairs of shoes was employed for this research. This dataset comprises 797 images from the Adidas brand and 2445 images from the Nike brand. The dataset used in the study includes worn shoe prints. According to the results obtained, the training phase achieved an accuracy of 96%, and the validation phase achieved an accuracy of 93%. These study results are highly positive and indicate promising potential for classifying 3D shoe prints. This study is described as the first classification study conducted using a deep learning method specifically on 3D shoe prints. It provides proof of concept that deep learning research can be conducted on 3D shoeprints. While the developed binary classification of these 3D shoeprints may not fully meet current forensic needs, it will serve as a source of motivation for future research and for the creation of 3D datasets intended for forensic purposes.

PMID:39235752 | DOI:10.1007/s12024-024-00877-6

Transl Vis Sci Technol. 2024 Sep 3;13(9):11. doi: 10.1167/tvst.13.9.11.

ABSTRACT

PURPOSE: The purpose of this study was to develop a deep learning algorithm for detecting and quantifying incomplete retinal pigment epithelium and outer retinal atrophy (iRORA) and complete retinal pigment epithelium and outer retinal atrophy (cRORA) in optical coherence tomography (OCT) that generalizes well to data from different devices and to validate in an intermediate age-related macular degeneration (iAMD) cohort.

METHODS: The algorithm comprised a domain adaptation (DA) model, promoting generalization across devices, and a segmentation model for detecting granular biomarkers defining iRORA/cRORA, which are combined into iRORA/cRORA segmentations. Manual annotations of iRORA/cRORA in OCTs from different devices in the MACUSTAR study (168 patients with iAMD) were compared to the algorithm's output. Eye level classification metrics included sensitivity, specificity, and quadratic weighted Cohen's κ score (κw). Segmentation performance was assessed quantitatively using Bland-Altman plots and qualitatively.

RESULTS: For ZEISS OCTs, sensitivity and specificity for iRORA/cRORA classification were 38.5% and 93.1%, respectively, and 60.0% and 96.4% for cRORA. For Spectralis OCTs, these were 84.0% and 93.7% for iRORA/cRORA, and 62.5% and 97.4% for cRORA. The κw scores for 3-way classification (none, iRORA, and cRORA) were 0.37 and 0.73 for ZEISS and Spectralis, respectively. Removing DA reduced κw from 0.73 to 0.63 for Spectralis.

CONCLUSIONS: The DA-enabled iRORA/cRORA segmentation algorithm showed superior consistency compared to human annotations, and good generalization across OCT devices.

TRANSLATIONAL RELEVANCE: The application of this algorithm may help toward precise and automated tracking of iAMD-related lesion changes, which is crucial in clinical settings and multicenter longitudinal studies on iAMD.

PMID:39235402 | DOI:10.1167/tvst.13.9.11

Transl Vis Sci Technol. 2024 Sep 3;13(9):7. doi: 10.1167/tvst.13.9.7.

ABSTRACT

PURPOSE: To develop a Vision Transformer (ViT) model based on the mixed attention feature enhancement mechanism, ChoroidSeg-ViT, for choroid layer segmentation in optical coherence tomography (OCT) images.

METHODS: This study included a dataset of 100 OCT B-scans images. Ground truths were carefully labeled by experienced ophthalmologists. An end-to-end local-enhanced Transformer model, ChoroidSeg-ViT, was designed to segment the choroid layer by integrating the local enhanced feature extraction and semantic feature fusion paths. Standard segmentation metrics were selected to evaluate ChoroidSeg-ViT.

RESULTS: Experimental results demonstrate that ChoroidSeg-ViT exhibited superior segmentation performance (mDice: 98.31, mIoU: 96.62, mAcc: 98.29) compared to other deep learning approaches, thus indicating the effectiveness and superiority of this proposed model for the choroid layer segmentation task. Furthermore, ablation and generalization experiments validated the reasonableness of the module design.

CONCLUSIONS: We developed a novel Transformer model to precisely and automatically segment the choroid layer and achieved the state-of-the-art performance.

TRANSLATIONAL RELEVANCE: ChoroidSeg-ViT could segment precise and smooth choroid layers and form the basis of an automatic choroid analysis system that would facilitate future choroidal research in ophthalmology.

PMID:39235399 | DOI:10.1167/tvst.13.9.7

Comput Methods Biomech Biomed Engin. 2024 Sep 5:1-11. doi: 10.1080/10255842.2024.2400318. Online ahead of print.

ABSTRACT

Machine learning (ML) has been used to predict lower extremity joint torques from joint angles and surface electromyography (sEMG) signals. This study trained three bidirectional Long Short-Term Memory (LSTM) models, which utilize joint angle, sEMG, and combined modalities as inputs, using a publicly accessible dataset to estimate joint torques during normal walking and assessed the performance of models, that used specific inputs independently plus the accuracy of the joint-specific torque prediction. The performance of each model was evaluated using normalized root mean square error (nRMSE) and Pearson correlation coefficient (PCC). Each model's median scores for the PCC and nRMSE values were highly convergent and the bulk of the mean nRMSE values of all joints were less than 10%. The ankle joint torque was the most successfully predicted output, having a mean nRMSE of less than 9% for all models. The knee joint torque prediction has reached the highest accuracy with a mean nRMSE of 11% and the hip joint torque prediction of 10%. The PCC values of each model were significantly high and remarkably comparable for the ankle (∼ 0.98), knee (∼ 0.92), and hip (∼ 0.95) joints. The model obtained significantly close accuracy with single and combined input modalities, indicating that one of either input may be sufficient for predicting the torque of a particular joint, obviating the need for the other in certain contexts.

PMID:39235388 | DOI:10.1080/10255842.2024.2400318

Adv Sci (Weinh). 2024 Sep 5:e2406878. doi: 10.1002/advs.202406878. Online ahead of print.

ABSTRACT

Emerging reconfigurable metasurfaces offer various possibilities for programmatically manipulating electromagnetic waves across spatial, spectral, and temporal domains, showcasing great potential for enhancing terahertz applications. However, they are hindered by limited tunability, particularly evident in relatively small phase tuning over 270°, due to the design constraints with time-intensive forward design methodologies. Here, a multi-bit programmable metasurface is demonstrated capable of terahertz beam steering facilitated by a developed physics-informed inverse design (PIID) approach. Through integrating a modified coupled mode theory (MCMT) into residual neural networks, the PIID algorithm not only significantly increases the design accuracy compared to conventional neural networks but also elucidates the intricate physical relations between the geometry and the modes. Without decreasing the reflection intensity, the method achieves the enhanced phase tuning as large as 300°. Additionally, the inverse-designed programmable beam steering metasurface is experimentally validated, which is adaptable across 1-bit, 2-bit, and tri-state coding schemes, yielding a deflection angle up to 68° and broadened steering coverage. The demonstration provides a promising pathway for rapidly exploring advanced metasurface devices, with potentially great impact on communication and imaging technologies.

PMID:39235322 | DOI:10.1002/advs.202406878

Nano Lett. 2024 Sep 5. doi: 10.1021/acs.nanolett.4c03069. Online ahead of print.

ABSTRACT

Super-resolution fluorescence imaging has offered unprecedented insights and revolutionized our understanding of biology. In particular, localized plasmonic structured illumination microscopy (LPSIM) achieves video-rate super-resolution imaging with ∼50 nm spatial resolution by leveraging subdiffraction-limited nearfield patterns generated by plasmonic nanoantenna arrays. However, the conventional trial-and-error design process for LPSIM arrays is time-consuming and computationally intensive, limiting the exploration of optimal designs. Here, we propose a hybrid inverse design framework combining deep learning and genetic algorithms to refine LPSIM arrays. A population of designs is evaluated using a trained convolutional neural network, and a multiobjective optimization method optimizes them through iteration and evolution. Simulations demonstrate that the optimized LPSIM substrate surpasses traditional substrates, exhibiting higher reconstruction accuracy, robustness against noise, and increased tolerance for fewer measurements. This framework not only proves the efficacy of inverse design for tailoring LPSIM substrates but also opens avenues for exploring new plasmonic nanostructures in imaging applications.

PMID:39234957 | DOI:10.1021/acs.nanolett.4c03069

Brief Bioinform. 2024 Jul 25;25(5):bbae439. doi: 10.1093/bib/bbae439.

ABSTRACT

The internal ribosome entry site (IRES) is a cis-regulatory element that can initiate translation in a cap-independent manner. It is often related to cellular processes and many diseases. Thus, identifying the IRES is important for understanding its mechanism and finding potential therapeutic strategies for relevant diseases since identifying IRES elements by experimental method is time-consuming and laborious. Many bioinformatics tools have been developed to predict IRES, but all these tools are based on structure similarity or machine learning algorithms. Here, we introduced a deep learning model named DeepIRES for precisely identifying IRES elements in messenger RNA (mRNA) sequences. DeepIRES is a hybrid model incorporating dilated 1D convolutional neural network blocks, bidirectional gated recurrent units, and self-attention module. Tenfold cross-validation results suggest that DeepIRES can capture deeper relationships between sequence features and prediction results than other baseline models. Further comparison on independent test sets illustrates that DeepIRES has superior and robust prediction capability than other existing methods. Moreover, DeepIRES achieves high accuracy in predicting experimental validated IRESs that are collected in recent studies. With the application of a deep learning interpretable analysis, we discover some potential consensus motifs that are related to IRES activities. In summary, DeepIRES is a reliable tool for IRES prediction and gives insights into the mechanism of IRES elements.

PMID:39234953 | DOI:10.1093/bib/bbae439

Biomed Pap Med Fac Univ Palacky Olomouc Czech Repub. 2024 Sep 4. doi: 10.5507/bp.2024.027. Online ahead of print.

ABSTRACT

Artificial Intelligence (AI) has evolved significantly over the past decades, from its early concepts in the 1950s to the present era of deep learning and natural language processing. Advanced large language models (LLMs), such as Chatbot Generative Pre-Trained Transformer (ChatGPT) is trained to generate human-like text responses. This technology has the potential to revolutionize various aspects of gastroenterology, including diagnosis, treatment, education, and decision-making support. The benefits of using LLMs in gastroenterology could include accelerating diagnosis and treatment, providing personalized care, enhancing education and training, assisting in decision-making, and improving communication with patients. However, drawbacks and challenges such as limited AI capability, training on possibly biased data, data errors, security and privacy concerns, and implementation costs must be addressed to ensure the responsible and effective use of this technology. The future of LLMs in gastroenterology relies on the ability to process and analyse large amounts of data, identify patterns, and summarize information and thus assist physicians in creating personalized treatment plans. As AI advances, LLMs will become more accurate and efficient, allowing for faster diagnosis and treatment of gastroenterological conditions. Ensuring effective collaboration between AI developers, healthcare professionals, and regulatory bodies is essential for the responsible and effective use of this technology. By finding the right balance between AI and human expertise and addressing the limitations and risks associated with its use, LLMs can play an increasingly significant role in gastroenterology, contributing to better patient care and supporting doctors in their work.

PMID:39234774 | DOI:10.5507/bp.2024.027

Stud Health Technol Inform. 2024 Aug 30;317:356-365. doi: 10.3233/SHTI240878.

ABSTRACT

INTRODUCTION: Glioblastoma (GB) is one of the most aggressive tumors of the brain. Despite intensive treatment, the average overall survival (OS) is 15-18 months. Therefore, it is helpful to be able to assess a patient's OS to tailor treatment more specifically to the course of the disease. Automated analysis of routinely generated MRI sequences (FLAIR, T1, T1CE, and T2) using deep learning-based image classification has the potential to enable accurate OS predictions.

METHODS: In this work, a method was developed and evaluated that classifies the OS into three classes - "short", "medium" and "long". For this purpose, the four MRI sequences of a person were corrected using bias-field correction and merged into one image. The pipeline was realized by a bagging model using 5-fold cross-validation and the ResNet50 architecture.

RESULTS: The best model was able to achieve an F1-score of 0.51 and an accuracy of 0.67. In addition, this work enabled a largely clear differentiation of the "short" and "long" classes, which offers high clinical significance as decision support.

CONCLUSION: Automated analysis of MRI scans using deep learning-based image classification has the potential to enable accurate OS prediction in glioblastomas.

PMID:39234740 | DOI:10.3233/SHTI240878

Stud Health Technol Inform. 2024 Aug 30;317:270-279. doi: 10.3233/SHTI240867.

ABSTRACT

INTRODUCTION: A modern approach to ensuring privacy when sharing datasets is the use of synthetic data generation methods, which often claim to outperform classic anonymization techniques in the trade-off between data utility and privacy. Recently, it was demonstrated that various deep learning-based approaches are able to generate useful synthesized datasets, often based on domain-specific analyses. However, evaluating the privacy implications of releasing synthetic data remains a challenging problem, especially when the goal is to conform with data protection guidelines.

METHODS: Therefore, the recent privacy risk quantification framework Anonymeter has been built for evaluating multiple possible vulnerabilities, which are specifically based on privacy risks that are considered by the European Data Protection Board, i.e. singling out, linkability, and attribute inference. This framework was applied to a synthetic data generation study from the epidemiological domain, where the synthesization replicates time and age trends previously found in data collected during the DONALD cohort study (1312 participants, 16 time points). The conducted privacy analyses are presented, which place a focus on the vulnerability of outliers.

RESULTS: The resulting privacy scores are discussed, which vary greatly between the different types of attacks.

CONCLUSION: Challenges encountered during their implementation and during the interpretation of their results are highlighted, and it is concluded that privacy risk assessment for synthetic data remains an open problem.

PMID:39234731 | DOI:10.3233/SHTI240867

Stud Health Technol Inform. 2024 Aug 30;317:49-58. doi: 10.3233/SHTI240837.

ABSTRACT

INTRODUCTION: Data-driven medical research (DDMR) needs multimodal data (MMD) to sufficiently capture the complexity of clinical cases. Methods for early multimodal data integration (MMDI), i.e. integration of the data before performing a data analysis, vary from basic concatenation to applying Deep Learning, each with distinct characteristics and challenges. Besides early MMDI, there exists late MMDI which performs modality-specific data analyses and then combines the analysis results.

METHODS: We conducted a scoping review, following PRISMA guidelines, to find and analyze 21 reviews on methods for early MMDI between 2019 and 2024.

RESULTS: Our analysis categorized these methods into four groups and summarized group-specific characteristics that are relevant for choosing the optimal method combination for MMDI pipelines in DDMR projects. Moreover, we found that early MMDI is often performed by executing several methods subsequently in a pipeline. This early MMDI pipeline is usually subject to manual optimization.

DISCUSSION: Our focus was on structural integration in DDMR. The choice of MMDI method depends on the research setting, complexity, and the researcher team's expertise. Future research could focus on comparing early and late MMDI approaches as well as automating the optimization of MMDI pipelines to integrate vast amounts of real-world medical data effectively, facilitating holistic DDMR.

PMID:39234706 | DOI:10.3233/SHTI240837