Medical Journal of The Islamic Republic of Iran (MJIRI)
Iran University of Medical Sciences
Wed, Jun 10, 2026
[Archive]
Volume 40, Issue 1 (1-2026)
Med J Islam Repub Iran 2026 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Hajiabbasi A, Jamshidi A R, Shoaee S, Shenavar Masooleh I, Ebrahimi H, Mehdipour Dalivand M et al . A Fully Expert Human-Based Retrieval Augmented Generation (FEH-RAG) Framework: A Proof of Concept Study in Labelling Patients with Sjögren Syndrome. Med J Islam Repub Iran 2026; 40 (1) :414-436
URL: http://mjiri.iums.ac.ir/article-1-9634-en.html
URL: http://mjiri.iums.ac.ir/article-1-9634-en.html
Asghar Hajiabbasi 
, Ahmad Reza Jamshidi , Shervan Shoaee , Irandokht Shenavar Masooleh , Hourvash Ebrahimi , Mahsa Mehdipour Dalivand , Pooneh Ghavidel Parsa
, Ahmad Reza Jamshidi , Shervan Shoaee , Irandokht Shenavar Masooleh , Hourvash Ebrahimi , Mahsa Mehdipour Dalivand , Pooneh Ghavidel Parsa
Guilan Rheumatology Research Center, Department of Rheumatology, Razi Hospital, School of Medicine, Guilan University of Medical Sciences, Rasht, Iran , grrc@gums.ac.ir
Abstract: (30 Views)
Background: Accurate application of reference standards (RSs) is essential for correct decision-making in areas governed by such standards. Yet in real-world practice, even fully trained users often apply RSs inconsistently, due to cognitive overload, stress, or other contextual factors, generating misleading evidence. This problem is exemplified by the fact that up to 80% of board-certified hematologists mislabel patients with Sjögren syndrome (SS), a connective tissue disorder (CTD) associated with the greatest risk of lymphoproliferative disorders compared to other CTDs. Embedding RS-based consultations with full objectivity into the decision-making layers of digitalized and non-digitalized settings is critical for improving both decision-making and the quality of resulting evidence. The aim of this proof-of-concept (POC) study is to apply a newly developed framework, the Fully Expert Human-based Retrieval Augmented Generation (FEH-RAG) framework, to provide such a foundation for augmenting SS case labeling in routine daily clinical practice.
Methods: In this POC study, using the nine steps of the FEH-RAG framework, seven expert end-users systematically selected the most widely used SS classification criteria (SSCC) and extracted their elements, including items, definitions, item weights, and inter-item relationships. Extracted items were profiled based on their usage in routine clinical practice. A pathway layout and decision tables were developed accordingly. Following pathway generation, the residual misalignment of the outputs with the SSCC was assessed in a cohort of patients at risk of SS. The experts predefined that the residual misalignment rate of the FEH-RAG outputs with the SSCC must be ≤2% (95% confidence, using the rule of three).
Results: The FEH-RAG framework objectively generated RS-based, transparent, and traceable outputs, including decision tables, an SS classification pathway, and a list of misinterpretations of the SSCC. These misinterpretations involved definitions of dry eye and dry mouth, application of secondary SS criteria, handling SS criteria-specific exclusion rules, and interpretation of serological and objective test results. This POC study achieved its expert-defined maximum misalignment threshold of ≤2% with 95% confidence (0 misalignment in 150 consecutive patients at risk of SS).
Conclusion: This POC study established the needed foundation for improving SS case labeling in daily clinical practice across both digital and non-digital settings. As shown here, publishing FEH-RAG outputs while highlighting potential RS misinterpretations offers a transparent and traceable basis for augmenting decision-making in domains governed by RSs.
Methods: In this POC study, using the nine steps of the FEH-RAG framework, seven expert end-users systematically selected the most widely used SS classification criteria (SSCC) and extracted their elements, including items, definitions, item weights, and inter-item relationships. Extracted items were profiled based on their usage in routine clinical practice. A pathway layout and decision tables were developed accordingly. Following pathway generation, the residual misalignment of the outputs with the SSCC was assessed in a cohort of patients at risk of SS. The experts predefined that the residual misalignment rate of the FEH-RAG outputs with the SSCC must be ≤2% (95% confidence, using the rule of three).
Results: The FEH-RAG framework objectively generated RS-based, transparent, and traceable outputs, including decision tables, an SS classification pathway, and a list of misinterpretations of the SSCC. These misinterpretations involved definitions of dry eye and dry mouth, application of secondary SS criteria, handling SS criteria-specific exclusion rules, and interpretation of serological and objective test results. This POC study achieved its expert-defined maximum misalignment threshold of ≤2% with 95% confidence (0 misalignment in 150 consecutive patients at risk of SS).
Conclusion: This POC study established the needed foundation for improving SS case labeling in daily clinical practice across both digital and non-digital settings. As shown here, publishing FEH-RAG outputs while highlighting potential RS misinterpretations offers a transparent and traceable basis for augmenting decision-making in domains governed by RSs.
Policy Review: Original Research |
Subject:
Internal Medicine
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
