A Hybrid of Modified Capsule and Transformer Model for Sepsis Diagnosis
Article Sidebar
Main Article Content
Abstract
Sepsis is a critical and urgent medical condition that imposes a global health burden due to high mortality and the risk of long-term disability without prompt treatment. In this study, we propose a novel hybrid modified capsule and a transformer encoder (CaT) using a selected subset of biomarkers for the diagnosis of sepsis. The biomarkers are identified by a dual selection strategy that combines the differential expression analysis of immune-related genes with the Boruta algorithm using a random forest model. The modified capsule network consists of 4 parallel capsule layers, each implemented as a feedforward unit comprising a linear transformation followed by ReLU activation. On the validation set using Leave-One- Dataset-Out Cross-Validation, the CaT model shows better performance compared to other machine learning and deep learning models, with an accuracy of 96.8%, sensitivity of 98.0%, specificity of 87.9%, Mathews correlation coefficient of 85.6%, and area under curve of 98.0%. These findings highlight the robustness, generalization, and effectiveness of the proposed CaT model, demonstrating its potential as a reliable tool for the prediction of sepsis in clinical practice.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
N. Wang, H. Huang, Y. Tan and N. Zhang, “Research progress of biomarkers for sepsis and precision medicine,” Emergency Medicine International, vol. 2025, no. 1, p. 4585495, 2025.
R. Pan, J. Mao, Y. Zheng, W. Chen, J. Guo and L. Wang, “MiR-30a-5p alleviates LPS-induced HPMEC injury through regulation of autophagy via Beclin-1,” Biocell, vol. 48, no. 3, p. 431, 2024.
K. L. Kalantar et al., “Integrated host-microbe plasma metagenomics for sepsis diagnosis in a prospective cohort of critically ill adults,” Nature Microbiology, vol. 7, no. 11, pp. 1805–1816, 2022.
T. van der Poll, M. Shankar-Hari and W. J. Wiersinga, “The immunology of sepsis,” Immunity, vol. 54, no. 11, pp. 2450–2464, 2021.
K. L. Burnham et al., “Shared and distinct aspects of the sepsis transcriptomic response to fecal peritonitis and pneumonia,” American Journal of Respiratory and Critical Care Medicine, vol. 196, no. 3, pp. 328–339, 2017.
Q. Su, J. Huang, Y. Zhang, Z. Liu, Z. Lv, C. Zhang, C. Ling, H. Su, L. Zhan and Z. Zhang, “AI-driven discovery of minimal sepsis biomarkers for disease detection and progression: precision medicine across diverse populations,” Frontiers in Medicine, vol. 12, p. 1521827, 2025.
Z.H. Chen, W.Y. Zhang, H. Ye, Y.Q. Guo, K. Zhang and X.M. Fang, “A signature of immune-related genes correlating with clinical prognosis and immune microenvironment in sepsis,” BMC Bioinformatics, vol. 24, no. 1, p. 20, 2023, doi: 10.1186/s12859-023-05134-1.
M. Abbas and Y. El-Manzalawy, “Machine learning based refined differential gene expression analysis of pediatric sepsis,” BMC Medical Genomics, vol. 13, no. 1, p. 122, 2020.
J. Wang, J. Cai, L. Yue, X. Zhou, C. Hu and H. Zhu, “Identification of potential biomarkers of septic shock based on pathway and transcriptome analyses of immune-related genes,” Genetics Research, vol. 2023, no. 1, p. 9991613, 2023.
Y. Yang, Y. Zhang, S. Li, X. Zheng, M.H. Wong, K.S. Leung and L. Cheng, “A robust and generalizable immune-related signature for sepsis diagnostics,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 19, no. 6, pp. 3246–3254, 2021.
K. H. Goh, L. Wang, A. Y. K. Yeow, H. Poh, K. Li, J. J. L. Yeow and G. Y. H. Tan, “Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare,” Nature Communications, vol. 12, no. 1, p. 711, 2021.
W. Xiong, Y. Zhan, R. Xiao and F. Liu, “Advancing sepsis diagnosis and immunotherapy: machine learning-driven identification of stable molecular biomarkers and therapeutic targets,” Scientific Reports, vol. 15, no. 1, p. 8333, 2025.
H. Wang, L. Len, L. Hu and Y. Hu, “Combining machine learning and single-cell sequencing to identify key immune genes in sepsis,” Scientific Reports, vol. 15, no. 1, p. 1557, 2025.
W. Zhang, H. Shi and J. Peng, “A diagnostic model for sepsis using an integrated machine learning framework approach and its therapeutic drug discovery,” BMC Infectious Diseases, vol. 25, no. 1, p. 219, 2025.
Y. Fan, Q. Han, J. Li, G. Ye, X. Zhang, T. Xu and H. Li, “Revealing potential diagnostic gene biomarkers of septic shock based on machine learning analysis,” BMC Infectious Diseases, vol. 22, no. 1, p. 65, 2022.
Z. Chen, L. Zeng, G. Liu, Y. Ou, C. Lu, B. Yang and L. Zuo, “Construction of autophagy-related gene classifiers for early diagnosis, prognosis and predicting immune microenvironment features in sepsis by machine learning algorithms,” Journal of Inflammation Research, pp. 6165–6186, 2022.
Q. Zhao, N. Xu, H. Guo and J. Li, “Identification of the diagnostic signature of sepsis based on bioinformatic analysis of gene expression and machine learning,” Combinatorial Chemistry & High Throughput Screening, vol. 25, no. 1, pp. 21–28, 2022.
T. H. Zhang, M. M. Hasib, Y. C. Chiu, Z. F. Han, Y. F. Jin, M. Flores, Y. Chen and Y. Huang, “Transformer for gene expression modeling (T-GEM): an interpretable deep learning model for gene expression-based phenotype pre
A. Khan and B. Lee, “DeepGene Transformer: Transformer for the gene expression-based classification of cancer subtypes,” Expert Systems with Applications, vol. 226, p. 120047, 2023.
H. S. Shon, Y. Yi, K. O. Kim, E. J. Cha and K. A. Kim, “Classification of stomach cancer gene expression data using CNN algorithm of deep learning,” Journal of Biomedical and Translational Research, vol. 20, no. 1, pp. 15–20, 2019.
E. A. T. Strickler, J. Thomas, J. P. Thomas, B. Benjamin and R. Shamsuddin, “Exploring a global interpretation mechanism for deep learning networks when predicting sepsis,” Scientific Reports, vol. 13, no. 1, p. 3067, 2023.
B. Y. Al-Mualemi and L. Lu, “A deep learning-based sepsis estimation scheme,” IEEE Access, vol. 9, pp. 5442–5452, 2020.
Y. Tang, Y. Zhang, and J. Li, “A time series driven model for early sepsis prediction based on transformer module,” BMC Medical Research Methodology, vol. 24, no. 1, p. 23, 2024.
Z. Zhai, J. Peng, W. Zhong, J. Tao, Y. Ao, B. Niu and L. Zhu, “Identification of key genes and potential therapeutic targets in sepsis-associated acute kidney injury using transformer and machine learning approaches,” Bioengineering, vol. 12, no. 5, p. 536, 2025.
E. Clough and T. Barrett, “The Gene Expression Omnibus Database,” in Statistical Genomics: Methods and Protocols, C. E. Gondro, J. van der Werf, and B. Hayes, Eds. New York, NY: Humana Press, 2018, pp. 93–110.
U. Sarkans, A. F¨ullgrabe, A. Ali, A. Athar, E. Behrangi, N. Diaz, S. Fexova, N. George, H. Iqbal, S. Kurr and J. Munoz, “From ArrayExpress to BioStudies,” Nucleic Acids Research, vol. 49, no. D1, pp. D1502–D1506, 2021.
M. B. Kursa, A. Jankowski and W. R. Rudnicki, “Boruta–a system for feature selection,” Fundamenta Informaticae, vol. 101, no. 4, pp. 271–285, 2010.
X. Zheng et al., “scCaT: An explainable encapsulating architecture for sepsis diagnosis transferring from single-cell RNA sequencing,” PLOS Computational Biology, vol. 20, no. 10, p. e1012083, 2024.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, 2017.
L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. 155
S. Kiranyaz, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj and D. J. Inman, “1D convolutional neural networks and applications: A survey,” Mechanical Systems and Signal Processing, vol. 151, p. 107398, 2021.
M.-C. Popescu, V. E. Balas, L. Perescu-Popescu and N. Mastorakis, “Multilayer perceptron and neural networks,” WSEAS Transactions on Circuits and Systems, vol. 8, no. 7, pp. 579–588, 2009.
