LLM-Based Prompt Ensemble for Reliable Medical Entity Recognition from EHRs

Document Type

Conference Proceeding

Publication Date

2025

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Source Publication

2025 IEEE International Conference on Information Reuse and Integration and Data Science (IRI)

Source ISSN

979-8-3315-99447

Original Item ID

DOI: 10.1109/IRI66576.2025.00037

Abstract

Electronic Health Records (EHRs) are digital records of patient information, often containing unstructured clinical text. Named Entity Recognition (NER) is essential in EHRs for extracting key medical entities like problems, tests, and treatments to support downstream clinical applications. This paper explores prompt-based medical entity recognition using large language models (LLMs), specifically GPT-4o and DeepSeek-R1, guided by various prompt engineering techniques, including zero-shot, few-shot, and an ensemble approach. Among all strategies, GPT-4o with prompt ensemble achieved the highest classification performance with an F1-score of 0.95 and recall of 0.98, outperforming DeepSeek-R1 on the task. The ensemble method improved reliability by aggregating outputs through embedding-based similarity and majority voting.

Comments

Published as part of the proceedings of the 2025 IEEE International Conference on Information Reuse and Integration and Data Science (IRI). DOI.

Link to Full Text

Share

COinS