A Knowledge-Graph-Enhanced Transformer Framework for Clinical NLP and Diagnostic Decision Support
Keywords:
Natural Language Processing, AI Edge Artificial Intelligence, Network Intrusion Detection, Sparse Neural Networks, Transformer Accelerator, Entropy-Guided ControlAbstract
Unstructured clinical text remains an everlasting problem for health organizations trying to exploit artificial intelligence in practice. In this paper, we introduce the framework KG-BioClinical, which leverages a domain-aware transformer encoder along with knowledge graph reasoning to improve multi-label ICD-10 diagnosis prediction from unstructured discharge summaries. Our core claim here is straightforward: structured relational knowledge embedded in biomedical ontologies can significantly boost implicit statistical knowledge learned by large-scale language models, especially for infrequent diagnosis groups in training data. Experiments show that the proposed model yields the macro-F1 score of 0.871 on the main testbed of MIMIC-IV dataset as well as a low-resource French corpus and eICU database, outperforming baselines by 1.4 macro-F1 points in general and by 2.3 points for rare diagnosis codes. Contribution of individual components, training convergence, and explanations through attention mechanisms are quantified via ablation studies. Finally, an exhaustive analysis of previous studies highlights five ongoing research problems, motivating further work in the field.
Downloads
References
[1] Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., … Mark, R. G. (2023). MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 10(1), 1. doi:10.1038/s41597-022-01899-x
[2] Sun, Z., Lin, M., Zhu, Q., Xie, Q., Wang, F., Lu, Z., & Peng, Y. (2023). A scoping review on multimodal deep learning in biomedical images and texts. Journal of Biomedical Informatics, 146(104482), 104482. doi:10.1016/j.jbi.2023.104482
[3] Wang, X., Gao, T., Zhu, Z., Zhang, Z., Liu, Z., Li, J., & Tang, J. (2019). KEPLER: A unified model for Knowledge Embedding and pre-trained LanguagE Representation. doi:10.48550/arXiv.1911.06136
[4] Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). Proceedings of the 2019 Conference of the North. Presented at the Proceedings of the 2019 Conference of the North, Minneapolis, Minnesota. doi:10.18653/v1/n19-1423
[5] Huang, C.-W., Tsai, S.-C., & Chen, Y.-N. (2022). PLM-ICD: Automatic ICD Coding with Pretrained Language Models. doi:10.48550/arXiv.2207.05289
[6] Jain, S., & Wallace, B. C. (2019). Proceedings of the 2019 Conference of the North. Presented at the Proceedings of the 2019 Conference of the North, Minneapolis, Minnesota. doi:10.18653/v1/n19-1357
[7] Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2019). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. doi:10.48550/arXiv.1901.08746
[8] Li, F., & Yu, H. (2020). ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 8180–8187. https://doi.org/10.1609/aaai.v34i05.6331
[9] Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., … Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180. doi:10.1038/s41586-023-06291-2
[10] Peng, B., Zhu, Y., Liu, Y., Bo, X., Shi, H., Hong, C., … Tang, S. (2026). Graph retrieval-Augmented Generation: A Survey. ACM Transactions on Information Systems, 44(2), 1–52. doi:10.1145/3777378
[11] Hu, S., Teng, F., Huang, L., Yan, J., & Zhang, H. (2021). An explainable CNN approach for medical codes prediction from clinical text. BMC Medical Informatics and Decision Making, 21(Suppl 9), 256. doi:10.1186/s12911-021-01615-6
[12] Naseem, U., Khushi, M., Khan, S. K., Shaukat, K., & Moni, M. A. (2021). A Comparative Analysis of Active Learning for Biomedical Text Mining. Applied System Innovation, 4(1), 23. https://doi.org/10.3390/asi4010023
[13] Pai, S., & Bader, G. D. (2018). Patient similarity networks for precision medicine. Journal of Molecular Biology, 430(18 Pt A), 2924–2938. doi:10.1016/j.jmb.2018.05.037
[14] Pollard, T. J., Johnson, A. E. W., Raffa, J. D., Celi, L. A., Mark, R. G., & Badawi, O. (2018). The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Scientific Data, 5(1), 180178. doi:10.1038/sdata.2018.178
[15] Ridnik, T., Ben-Baruch, E., Zamir, N., Noy, A., Friedman, I., Protter, M., & Zelnik-Manor, L. (2021, October). Asymmetric Loss For Multi-Label Classification. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Presented at the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada. doi:10.1109/iccv48922.2021.00015
[16] Hou, W.-H., Wang, X.-K., Wang, Y.-N., Wang, J.-Q., & Xiao, F. (2024). Modelling long medical documents and code associations for explainable automatic ICD coding. Expert Systems With Applications, 249(123519), 123519. doi:10.1016/j.eswa.2024.123519
[17] Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2018). Graph attention networks. doi:10.17863/CAM.48429
[18] Oniani, D., Wu, X., Visweswaran, S., Kapoor, S., Kooragayalu, S., Polanska, K., & Wang, Y. (2024). Enhancing Large Language Models for Clinical Decision Support by incorporating Clinical Practice Guidelines. doi:10.48550/arXiv.2401.11120
[19] Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., … Sun, M. (2020). Graph neural networks: A review of methods and applications. AI Open, 1, 57–81. doi:10.1016/j.aiopen.2021.01.001
[20] Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019). ERNIE: Enhanced language representation with informative entities. doi:10.48550/arXiv.1905.07129
[21] Xu, S., Chen, M., & Chen, S. (2024). Enhancing retrieval-augmented generation models with knowledge graphs: Innovative practices through a dual-pathway approach. In Lecture Notes in Computer Science. Lecture Notes in Computer Science (pp. 398–409). doi:10.1007/978-981-97-5678-0_34
[22] Zhao, Y., Yin, J., Zhang, L., Zhang, Y., & Chen, X. (2023). Drug-drug interaction prediction: databases, web servers and computational models. Briefings in Bioinformatics, 25(1), bbad445. doi:10.1093/bib/bbad445
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Hamzah Alkhazaleh, Moaza Almarri, Rami Howash, Saud Al Belyhed, Sultan Almheiri, Shadi Atalla (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
