Comput Biol Chem. 2024 Jul 18;112:108129. doi: 10.1016/j.compbiolchem.2024.108129. Online ahead of print.

ABSTRACT

DNA methylation at the N6 position of adenine (N6-methyladenine, 6 mA), which refers to the attachment of a methyl group to the N6 site of the adenine (A) of DNA, is an important epigenetic modification in prokaryotic and eukaryotic genomes. Accurately predicting the 6 mA binding sites can provide crucial insights into gene regulation, DNA repair, disease development and so on. Wet experiments are commonly used for analyzing 6 mA binding sites. However, they suffer from high cost and expensive time. Therefore, various deep learning methods have been widely used to predict 6 mA binding sites recently. In this study, we develop a framework based on multi-scale DNA language model named "iDNA6mA-MDL". "iDNA6mA-MDL" integrates multiple kmers and the nucleotide property and frequency method for feature embedding, which can capture a full range of DNA sequence context information. At the prediction stage, it also leverages DNABERT to compensate for the incomplete capture of global DNA information. Experiments show that our framework obtains average AUC of 0.981 on a classic 6 mA rice gene dataset, going beyond all existing advanced models under fivefold cross-validations. Moreover, "iDNA6mA-MDL" outperforms most of the popular state-of-the-art methods on another 11 6 mA datasets, demonstrating its effectiveness in 6 mA binding sites prediction.

PMID:39067351 | DOI:10.1016/j.compbiolchem.2024.108129

Comput Biol Chem. 2024 Jul 14;112:108146. doi: 10.1016/j.compbiolchem.2024.108146. Online ahead of print.

ABSTRACT

Many drug molecules contain functional groups, resulting in a torsional barrier corresponding to rotation around the bond linking the fragments. In medicinal chemistry and pharmaceutical sciences, inclusive of drug design studies, the exact calculation of the potential energy surface (PES) of these molecular torsions is extremely important and precious. Machine learning (ML), including deep learning (DL), is currently one of the most rapidly evolving tools in computer-aided drug discovery and molecular simulations. In this work, we used ANI-1x neural network potential as a quantum-level ML to predict the PESs of the L-3,4-dihydroxyphenylalanine (Levodopa) antiparkinsonian drug molecule. The electronic energies and structural parameters calculated by density functional theory (DFT) using the wB97X method and all possible Pople's basis sets indicated the 6-31G(d) basis set, when used with the wB97X functional, exhibits behavior similar to that of the ANI-1x model. The vibrational frequencies investigation showed a linear correlation between DFT and ML data. All ANI-1x calculations were completed quickly in a very short computing time. From this perspective, we expect the ANI-1x dataset applied in this work to be appreciably efficient and effective in computational structure-based drug design studies.

PMID:39067350 | DOI:10.1016/j.compbiolchem.2024.108146

J Environ Manage. 2024 Jul 26;367:121913. doi: 10.1016/j.jenvman.2024.121913. Online ahead of print.

ABSTRACT

With the increasing importance of environmental and economic sustainability concerns, the concept of Environmental, Social, and Governance (ESG) has gained significant attention. In the era of digitalization, a research approach called carbon sentiment analysis has emerged as an innovative method. This study aims to explore the connections between carbon sentiment, ESG, and corporate sustainable growth within the context of the green economy. By using Ordinary Least Squares (OLS) regression analysis and establishing a panel data model of ESG performance and sustainable growth for Chinese listed companies, a notable positive correlation between the two variables was observed. Endogeneity was addressed using the two-stage instrumental variable method (2SLS) and the dynamic panel Generalized Method of Moments (GMM) model, with the results remaining robust both before and after the COVID-19 pandemic. Carbon-related news and textual information were collected and analyzed using advanced deep learning methods in Natural Language Processing (NLP), specifically Bidirectional Encoder Representations from Transformers (BERT) and Long Short-Term Memory (LSTM) models. This analysis enabled sentiment analysis and identification of the sentiment orientation of carbon news. The obtained sentiment scores were then integrated with company data to establish a moderation effect model. The findings of the study reveal that carbon sentiment significantly and positively moderates ESG performance in relation to corporate sustainable growth. Furthermore, the construction of a mediation effect model showed that carbon sentiment can moderate ESG performance by reducing environmental uncertainty, enhancing social trust, and alleviating financing constraints, thereby influencing corporate sustainable growth. The results of the heterogeneity group regression analysis demonstrate that the impact of ESG performance driven by carbon sentiment on sustainable growth is more pronounced in carbon market pilot regions, non-heavily polluting industries, and labor-intensive industries. This research provides a fresh perspective for understanding the dynamics of ESG, online carbon sentiment, and their implications for corporate sustainable growth. Additionally, it contributes to the development of the green economy and the formulation of environmental management policies.

PMID:39067346 | DOI:10.1016/j.jenvman.2024.121913

J Electrocardiol. 2024 Jul 20;86:153759. doi: 10.1016/j.jelectrocard.2024.153759. Online ahead of print.

ABSTRACT

BACKGROUND: Atrial fibrillation (AF) is often asymptomatic and thus under-observed. Given the high risks of stroke and heart failure among patients with AF, early prediction and effective management are crucial. Given the prevalence of obstructive sleep apnea among AF patients, electrocardiogram (ECG) analysis from polysomnography (PSG) offers a unique opportunity for early AF prediction. Our aim is to identify individuals at high risk of AF development from single‑lead ECGs during standard PSG.

METHODS: We analyzed 18,782 single‑lead ECG recordings from 13,609 subjects undergoing PSG at the Massachusetts General Hospital sleep laboratory. AF presence was identified using ICD-9/10 codes. The dataset included 15,913 recordings without AF history and 2054 recordings from patients diagnosed with AF between one month to fifteen years post-PSG. Data were partitioned into training, validation, and test cohorts ensuring that individual patients remained exclusive to each cohort. The test set was held out during the training process. We employed two different methods for feature extraction to build a final model for AF prediction: Extraction of hand-crafted ECG features and a deep learning method. For extraction of ECG-hand-crafted features, recordings were split into 30-s windows, and those with a signal quality index (SQI) below 0.95 were discarded. From each remaining window, 150 features were extracted from the time, frequency, time-frequency domains, and phase-space reconstructions of the ECG. A compilation of 12 statistical features summarized these window-specific features per recording, resulting in 1800 features (12 × 150). A pre-trained deep neural network from the PhysioNet Challenge 2021 was updated using transfer learning to discriminate recordings with and without AF. The model processed PSG ECGs in 16-s windows to generate AF probabilities, from which 13 statistical features were extracted. Combining 1800 features from feature extraction with 13 from the deep learning model, we performed a feature selection and subsequently trained a shallow neural network to predict future AF and evaluated its performance on the test cohort.

RESULTS: On the test set, our model exhibited sensitivity, specificity, and precision of 0.67, 0.81, and 0.3, respectively, for AF prediction. Survival analysis revealed a hazard ratio of 8.36 (p-value: 1.93 × 10-52) for AF outcomes using the log-rank test.

CONCLUSIONS: Our proposed ECG analysis method, utilizing overnight PSG data, shows promise in AF prediction despite modest precision, suggesting false positives. This approach could enable low-cost screening and proactive treatment for high-risk patients. Refinements, including additional physiological parameters, may reduce false positives, enhancing clinical utility and accuracy.

PMID:39067281 | DOI:10.1016/j.jelectrocard.2024.153759

Sensors (Basel). 2024 Jul 22;24(14):4761. doi: 10.3390/s24144761.

ABSTRACT

Visual object tracking is an important technology in camera-based sensor networks, which has a wide range of practicability in auto-drive systems. A transformer is a deep learning model that adopts the mechanism of self-attention, and it differentially weights the significance of each part of the input data. It has been widely applied in the field of visual tracking. Unfortunately, the security of the transformer model is unclear. It causes such transformer-based applications to be exposed to security threats. In this work, the security of the transformer model was investigated with an important component of autonomous driving, i.e., visual tracking. Such deep-learning-based visual tracking is vulnerable to adversarial attacks, and thus, adversarial attacks were implemented as the security threats to conduct the investigation. First, adversarial examples were generated on top of video sequences to degrade the tracking performance, and the frame-by-frame temporal motion was taken into consideration when generating perturbations over the depicted tracking results. Then, the influence of perturbations on performance was sequentially investigated and analyzed. Finally, numerous experiments on OTB100, VOT2018, and GOT-10k data sets demonstrated that the executed adversarial examples were effective on the performance drops of the transformer-based visual tracking. White-box attacks showed the highest effectiveness, where the attack success rates exceeded 90% against transformer-based trackers.

PMID:39066157 | DOI:10.3390/s24144761

Sensors (Basel). 2024 Jul 22;24(14):4747. doi: 10.3390/s24144747.

ABSTRACT

In large public places such as railway stations and airports, dense pedestrian detection is important for safety and security. Deep learning methods provide relatively effective solutions but still face problems such as feature extraction difficulties, image multi-scale variations, and high leakage detection rates, which bring great challenges to the research in this field. In this paper, we propose an improved dense pedestrian detection algorithm GR-yolo based on Yolov8. GR-yolo introduces the repc3 module to optimize the backbone network, which enhances the ability of feature extraction, adopts the aggregation-distribution mechanism to reconstruct the yolov8 neck structure, fuses multi-level information, achieves a more efficient exchange of information, and enhances the detection ability of the model. Meanwhile, the Giou loss calculation is used to help GR-yolo converge better, improve the detection accuracy of the target position, and reduce missed detection. Experiments show that GR-yolo has improved detection performance over yolov8, with a 3.1% improvement in detection means accuracy on the wider people dataset, 7.2% on the crowd human dataset, and 11.7% on the people detection images dataset. Therefore, the proposed GR-yolo algorithm is suitable for dense, multi-scale, and scene-variable pedestrian detection, and the improvement also provides a new idea to solve dense pedestrian detection in real scenes.

PMID:39066144 | DOI:10.3390/s24144747

Sensors (Basel). 2024 Jul 21;24(14):4738. doi: 10.3390/s24144738.

ABSTRACT

This study presents a concrete stress monitoring method utilizing 1D CNN deep learning of raw electromechanical impedance (EMI) signals measured with a capsule-like smart aggregate (CSA) sensor. Firstly, the CSA-based EMI measurement technique is presented by depicting a prototype of the CSA sensor and a 2 degrees of freedom (2 DOFs) EMI model for the CSA sensor embedded in a concrete cylinder. Secondly, the 1D CNN deep regression model is designed to adapt raw EMI responses from the CSA sensor for estimating concrete stresses. Thirdly, a CSA-embedded cylindrical concrete structure is experimented with to acquire EMI responses under various compressive loading levels. Finally, the feasibility and robustness of the 1D CNN model are evaluated for noise-contaminated EMI data and untrained stress EMI cases.

PMID:39066134 | DOI:10.3390/s24144738

Sensors (Basel). 2024 Jul 21;24(14):4733. doi: 10.3390/s24144733.

ABSTRACT

This work presents TTFDNet, a transformer-based and transfer learning network for end-to-end depth estimation from single-frame fringe patterns in fringe projection profilometry. TTFDNet features a precise contour and coarse depth (PCCD) pre-processor, a global multi-dimensional fusion (GMDF) module and a progressive depth extractor (PDE). It utilizes transfer learning through fringe structure consistency evaluation (FSCE) to leverage the transformer's benefits even on a small dataset. Tested on 208 scenes, the model achieved a mean absolute error (MAE) of 0.00372 mm, outperforming Unet (0.03458 mm) models, PDE (0.01063 mm) and PCTNet (0.00518 mm). It demonstrated precise measurement capabilities with deviations of ~90 μm for a 25.4 mm radius ball and ~6 μm for a 20 mm thick metal part. Additionally, TTFDNet showed excellent generalization and robustness in dynamic reconstruction and varied imaging conditions, making it appropriate for practical applications in manufacturing, automation and computer vision.

PMID:39066131 | DOI:10.3390/s24144733

Sensors (Basel). 2024 Jul 21;24(14):4730. doi: 10.3390/s24144730.

ABSTRACT

The food crisis has increased demand for agricultural resources due to various factors such as extreme weather, energy crises, and conflicts. A solar greenhouse enables counter-seasonal winter cultivation due to its thermal insulation, thus alleviating the food crisis. The root temperature is of critical importance, although the mechanism of soil thermal environment change remains uncertain. This paper presents a comprehensive study of the soil thermal environment of a solar greenhouse in Jinzhong City, Shanxi Province, employing a variety of analytical techniques, including theoretical, experimental, and numerical simulation, and deep learning modelling. The results of this study demonstrate the following: During the overwintering period, the thermal environment of the solar greenhouse floor was divided into a low-temperature zone, a constant-temperature zone, and a high-temperature zone; the distance between the low-temperature boundary and the southern foot was 2.6 m. The lowest temperature in the low-temperature zone was 11.06 °C and the highest was 19.05 °C. The floor in the low-temperature zone had to be heated; the lowest value of the constant-temperature zone was 18.29 °C, without heating. The minimum distance between the area of high temperature and the southern foot of the solar greenhouse was 8 m and the lowest temperature reading was 19.29 °C. The indoor soil temperature tended to stabilise at a depth of 45 cm, and the lowest temperature reading at a horizontal distance of 1400 mm from the south foot was 19.5 °C. The Fluent and LSTM models fitted well and the models can be used to help control soil temperature during overwintering in extreme climates. The research can provide theoretical and data support for the crop areas and the heating of pipelines in the solar greenhouse.

PMID:39066129 | DOI:10.3390/s24144730

Sensors (Basel). 2024 Jul 20;24(14):4715. doi: 10.3390/s24144715.

ABSTRACT

In air traffic control (ATC), speech communication with radio transmission is the primary way to exchange information between the controller and the pilot. As a result, the integration of automatic speech recognition (ASR) systems holds immense potential for reducing controllers' workload and plays a crucial role in various ATC scenarios, which is particularly significant for ATC research. This article provides a comprehensive review of ASR technology's applications in the ATC communication system. Firstly, it offers a comprehensive overview of current research, including ATC corpora, ASR models, evaluation measures and application scenarios. A more comprehensive and accurate evaluation methodology tailored for ATC is proposed, considering advancements in communication sensing systems and deep learning techniques. This methodology helps researchers in enhancing ASR systems and improving the overall performance of ATC systems. Finally, future research recommendations are identified based on the primary challenges and issues. The authors sincerely hope this work will serve as a clear technical roadmap for ASR endeavors within the ATC domain and make a valuable contribution to the research community.

PMID:39066111 | DOI:10.3390/s24144715

Sensors (Basel). 2024 Jul 19;24(14):4692. doi: 10.3390/s24144692.

ABSTRACT

The presence of fog in the background can prevent small and distant objects from being detected, let alone tracked. Under safety-critical conditions, multi-object tracking models require faster tracking speed while maintaining high object-tracking accuracy. The original DeepSORT algorithm used YOLOv4 for the detection phase and a simple neural network for the deep appearance descriptor. Consequently, the feature map generated loses relevant details about the track being matched with a given detection in fog. Targets with a high degree of appearance similarity on the detection frame are more likely to be mismatched, resulting in identity switches or track failures in heavy fog. We propose an improved multi-object tracking model based on the DeepSORT algorithm to improve tracking accuracy and speed under foggy weather conditions. First, we employed our camera-radar fusion network (CR-YOLOnet) in the detection phase for faster and more accurate object detection. We proposed an appearance feature network to replace the basic convolutional neural network. We incorporated GhostNet to take the place of the traditional convolutional layers to generate more features and reduce computational complexities and costs. We adopted a segmentation module and fed the semantic labels of the corresponding input frame to add rich semantic information to the low-level appearance feature maps. Our proposed method outperformed YOLOv5 + DeepSORT with a 35.15% increase in multi-object tracking accuracy, a 32.65% increase in multi-object tracking precision, a speed increase by 37.56%, and identity switches decreased by 46.81%.

PMID:39066088 | DOI:10.3390/s24144692

Sensors (Basel). 2024 Jul 19;24(14):4686. doi: 10.3390/s24144686.

ABSTRACT

Infrared images hold significant value in applications such as remote sensing and fire safety. However, infrared detectors often face the problem of high hardware costs, which limits their widespread use. Advancements in deep learning have spurred innovative approaches to image super-resolution (SR), but comparatively few efforts have been dedicated to the exploration of infrared images. To address this, we design the Residual Swin Transformer and Average Pooling Block (RSTAB) and propose the SwinAIR, which can effectively extract and fuse the diverse frequency features in infrared images and achieve superior SR reconstruction performance. By further integrating SwinAIR with U-Net, we propose the SwinAIR-GAN for real infrared image SR reconstruction. SwinAIR-GAN extends the degradation space to better simulate the degradation process of real infrared images. Additionally, it incorporates spectral normalization, dropout, and artifact discrimination loss to reduce the potential image artifacts. Qualitative and quantitative evaluations on various datasets confirm the effectiveness of our proposed method in reconstructing realistic textures and details of infrared images.

PMID:39066083 | DOI:10.3390/s24144686

Sensors (Basel). 2024 Jul 18;24(14):4668. doi: 10.3390/s24144668.

ABSTRACT

The advancement of medical imaging has profoundly impacted our understanding of the human body and various diseases. It has led to the continuous refinement of related technologies over many years. Despite these advancements, several challenges persist in the development of medical imaging, including data shortages characterized by low contrast, high noise levels, and limited image resolution. The U-Net architecture has significantly evolved to address these challenges, becoming a staple in medical imaging due to its effective performance and numerous updated versions. However, the emergence of Transformer-based models marks a new era in deep learning for medical imaging. These models and their variants promise substantial progress, necessitating a comparative analysis to comprehend recent advancements. This review begins by exploring the fundamental U-Net architecture and its variants, then examines the limitations encountered during its evolution. It then introduces the Transformer-based self-attention mechanism and investigates how modern models incorporate positional information. The review emphasizes the revolutionary potential of Transformer-based techniques, discusses their limitations, and outlines potential avenues for future research.

PMID:39066065 | DOI:10.3390/s24144668

Sensors (Basel). 2024 Jul 18;24(14):4662. doi: 10.3390/s24144662.

ABSTRACT

Marker-less hand-eye calibration permits the acquisition of an accurate transformation between an optical sensor and a robot in unstructured environments. Single monocular cameras, despite their low cost and modest computation requirements, present difficulties for this purpose due to their incomplete correspondence of projected coordinates. In this work, we introduce a hand-eye calibration procedure based on the rotation representations inferred by an augmented autoencoder neural network. Learning-based models that attempt to directly regress the spatial transform of objects such as the links of robotic manipulators perform poorly in the orientation domain, but this can be overcome through the analysis of the latent space vectors constructed in the autoencoding process. This technique is computationally inexpensive and can be run in real time in markedly varied lighting and occlusion conditions. To evaluate the procedure, we use a color-depth camera and perform a registration step between the predicted and the captured point clouds to measure translation and orientation errors and compare the results to a baseline based on traditional checkerboard markers.

PMID:39066062 | DOI:10.3390/s24144662

Medicine (Baltimore). 2024 Jul 26;103(30):e39052. doi: 10.1097/MD.0000000000039052.

ABSTRACT

The Gulf Cooperation Council (GCC) countries are more vulnerable to many transmissible diseases, including tuberculosis (TB). This study is to identify the scientific publications related to TB in the GCC countries using topic modeling and co-word analysis. A bibliometric analytic study. The R-package, VOSviewer software, IBM SPPS, and Scopus Analytics were used to analyze performance, hotspots, knowledge structure, thematic evolution, trend topics, and inter-gulf and international cooperation on TB in the past 30 years (1993-2022). A total of 1999 publications associated with research on GCC-TB were published. The annual growth rate of documents was 7.76%. Saudi Arabia is the most highly published, followed by the United Arab Emirates, Kuwait, Qatar, Oman, and Bahrain. The most-cited GC country is Kingdom Saudi Arabia, followed by Kuwait. One hundred sixty research institutions contributed to the dissemination of TB-related knowledge in the GCC, where the highest publishing organizations were King Saud University (Kingdom Saudi Arabia; n = 518). The number of publications related to TB is high in GCC Countries. The current tendencies indicated that GCC scholars are increasingly focused on deep learning, chest X-ray, molecular docking, comorbid covid-19, risk factors, and Mycobacterium bovis.

PMID:39058842 | DOI:10.1097/MD.0000000000039052

PLoS One. 2024 Jul 26;19(7):e0306627. doi: 10.1371/journal.pone.0306627. eCollection 2024.

ABSTRACT

Photon-counting detector (PCD)-based computed tomography (CT) offers several advantages over conventional energy-integrating detector-based CT. Among them, the ability to discriminate energy exhibits significant potential for clinical applications because it provides material-specific information. That is, material decomposition (MD) can be achieved through energy discrimination. In this study, deep learning-based material decomposition was performed using live animal data. We propose MD-Unet, which is a deep learning strategy for material decomposition based on an Unet architecture trained with data from three energy bins. To mitigate the data insufficiency, we developed a pretrained model incorporating various simulation data forms and augmentation strategies. Incorporating these approaches into model training results in enhanced precision in material decomposition, thereby enabling the identification of distinct materials at individual pixel locations. The trained network was applied to the acquired animal data to evaluate material decomposition results. Compared with conventional methods, the newly generated MD-Unet demonstrated more accurate material decomposition imaging. Moreover, the network demonstrated an improved material decomposition ability and significantly reduced noise. In addition, they can potentially offer an enhancement level similar to that of a typical contrast agent. This implies that it can acquire images of the same quality with fewer contrast agents administered to patients, thereby demonstrating its significant clinical value.

PMID:39058758 | DOI:10.1371/journal.pone.0306627

PLoS One. 2024 Jul 26;19(7):e0306789. doi: 10.1371/journal.pone.0306789. eCollection 2024.

ABSTRACT

Respiratory diseases significantly affect respiratory function, making them a considerable contributor to global mortality. The respiratory muscles play an important role in disease prognosis; as such, quantitative analysis of the respiratory muscles is crucial to assess the status of the respiratory system and the quality of life in patients. In this study, we aimed to develop an automated approach for the segmentation and classification of three types of respiratory muscles from computed tomography (CT) images using artificial intelligence. With a dataset of approximately 600,000 thoracic CT images from 3,200 individuals, we trained the model using the Attention U-Net architecture, optimized for detailed and focused segmentation. Subsequently, we calculated the volumes and densities from the muscle masks segmented by our model and performed correlation analysis with pulmonary function test (PFT) parameters. The segmentation models for muscle tissue and respiratory muscles obtained dice scores of 0.9823 and 0.9688, respectively. The classification model, achieving a generalized dice score of 0.9900, also demonstrated high accuracy in classifying thoracic region muscle types, as evidenced by its F1 scores: 0.9793 for the pectoralis muscle, 0.9975 for the erector spinae muscle, and 0.9839 for the intercostal muscle. In the correlation analysis, the volume of the respiratory muscles showed a strong correlation with PFT parameters, suggesting that respiratory muscle volume may serve as a potential novel biomarker for respiratory function. Although muscle density showed a weaker correlation with the PFT parameters, it has a potential significance in medical research.

PMID:39058719 | DOI:10.1371/journal.pone.0306789

PLoS One. 2024 Jul 26;19(7):e0307649. doi: 10.1371/journal.pone.0307649. eCollection 2024.

ABSTRACT

Cancer treatment has become one of the biggest challenges in the world today. Different treatments are used against cancer; drug-based treatments have shown better results. On the other hand, designing new drugs for cancer is costly and time-consuming. Some computational methods, such as machine learning and deep learning, have been suggested to solve these challenges using drug repurposing. Despite the promise of classical machine-learning methods in repurposing cancer drugs and predicting responses, deep-learning methods performed better. This study aims to develop a deep-learning model that predicts cancer drug response based on multi-omics data, drug descriptors, and drug fingerprints and facilitates the repurposing of drugs based on those responses. To reduce multi-omics data's dimensionality, we use autoencoders. As a multi-task learning model, autoencoders are connected to MLPs. We extensively tested our model using three primary datasets: GDSC, CTRP, and CCLE to determine its efficacy. In multiple experiments, our model consistently outperforms existing state-of-the-art methods. Compared to state-of-the-art models, our model achieves an impressive AUPRC of 0.99. Furthermore, in a cross-dataset evaluation, where the model is trained on GDSC and tested on CCLE, it surpasses the performance of three previous works, achieving an AUPRC of 0.72. In conclusion, we presented a deep learning model that outperforms the current state-of-the-art regarding generalization. Using this model, we could assess drug responses and explore drug repurposing, leading to the discovery of novel cancer drugs. Our study highlights the potential for advanced deep learning to advance cancer therapeutic precision.

PMID:39058696 | DOI:10.1371/journal.pone.0307649

IEEE Trans Pattern Anal Mach Intell. 2024 Jul 26;PP. doi: 10.1109/TPAMI.2024.3434417. Online ahead of print.

ABSTRACT

High-quality private machine learning (ML) data stored in local data centers becomes a key competitive factor for AI corporations. In this paper, we present a novel insider attack called Matryoshka to reveal the possibility of breaking the privacy of ML data even with no exposed interface. Our attack employs a scheduled-to-publish DNN model as a carrier model for covert transmission of secret models which memorize the information of private ML data that otherwise has no interface to the outsider. At the core of our attack, we present a novel parameter sharing approach which exploits the learning capacity of the carrier model for information hiding. Our approach simultaneously achieves: (i) High Capacity - With almost no utility loss of the carrier model, Matryoshka can transmit over 10,000 real-world data samples within a carrier model which has 220× less parameters than the total size of the stolen data, and simultaneously transmit multiple heterogeneous datasets or models within a single carrier model under a trivial distortion rate, neither of which can be done with existing steganography techniques; (ii) Decoding Efficiency - once downloading the published carrier model, an outside colluder can exclusively decode the hidden models from the carrier model with only several integer secrets and the knowledge of the hidden model architecture; (iii) Effectiveness - Moreover, almost all the recovered models either have similar performance as if it is trained independently on the private data, or can be further used to extract memorized raw training data with low error; (iv) Robustness - Information redundancy is naturally implemented to achieve resilience against common post-processing techniques on the carrier before its publishing; (v) Covertness - A model inspector with different levels of prior knowledge could hardly differentiate a carrier model from a normal model.

PMID:39058616 | DOI:10.1109/TPAMI.2024.3434417

IEEE Trans Pattern Anal Mach Intell. 2024 Jul 26;PP. doi: 10.1109/TPAMI.2024.3434483. Online ahead of print.

ABSTRACT

Graphs are the most ubiquitous data structures for representing relational datasets and performing inferences in them. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations. This drawback is mitigated by hypergraphs, in which an edge can connect an arbitrary number of nodes. Most hypergraph learning approaches convert the hypergraph structure to that of a graph and then deploy existing geometric deep learning methods. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power. We present HyperMSG, a novel hypergraph learning framework that uses a modular two-level neural message passing strategy to accurately and efficiently propagate information within each hyperedge and across the hyperedges. HyperMSG adapts to the data and task by learning an attention weight associated with each node's degree centrality. Such a mechanism quantifies both local and global importance of a node, capturing the structural properties of a hypergraph. HyperMSG is inductive, allowing inference on previously unseen nodes. Further, it is robust and outperforms state-of-the-art hypergraph learning methods on a wide range of tasks and datasets. Finally, we demonstrate the effectiveness of HyperMSG in learning multimodal relations through detailed experimentation on a challenging multimedia dataset.

PMID:39058615 | DOI:10.1109/TPAMI.2024.3434483