SR-MKG: Self-Refining Medical Knowledge Graph via LLM-Based Topological Confidence Scoring for Radiological Reports of Lumbar Spinal Stenosis

Abstract

The automated creation of a medical knowledge graph (MKG) from unstructured clinical data is a significant outcome in healthcare decision support and retrieval-augmented generation (RAG) applications. However, large language models (LLMs) based on knowledge extraction inevitably introduce noise, hallucinations, and factual inconsistencies that compromise graph reliability and clinical utility. Existing refinement approaches require supervised labels or complex neural architectures, limiting their applicability in resource-constrained clinical environments. This study introduces Self-Refining Medical Knowledge Graph (SR-MKG), a novel two-stage framework that autonomously enhances LLM-generated medical knowledge graph (KG) quality through unsupervised topological analysis. The methodology comprises: (1) The first stage utilizes an LLM for high-recall triplet extraction from lumbar spine unstructured MRI radiological reports, and (2) an unsupervised self-refinement module implementing topological confidence scoring for each relationship based on graph structural support patterns. The SR-MKG framework evaluated with state-of-the-art LLMs Gemma3 and LLaVA, demonstrates substantial improvements in knowledge graph refinement across two thresholds (0.3 and 0.5). For Gemma3, the initial 1,524 triplets were refined to 843 and 372, reducing noise by 44.7% and 75.5%, and boosting precision from 44% to 92-94% and recall from 54% to 94-96%. LLaVA started with 1,381 triplets, refining to 720 and 270, with noise reductions of 47.9% and 80.4%, while improving precision from 46% to 76-90% and recall from 58% to 88-94%. This work establishes a foundation for automated medical knowledge extraction that bridges the gap between raw clinical text and structured, actionable medical insights. © 2025 IEEE.