Recent achievements of the team - A study on named entity recognition through the efficient integration of local models and large language models
Our team published the paper “NERLLM: Efficient integration of local NER models and large language models for named entity recognition” in Expert Systems With Applications (IF: 9.4, CAS Zone 1). The School of Computer Engineering and Science at Shanghai University is the first affiliation of this paper.
Local Named Entity Recognition (NER) models typically provide efficient inference and accurate in-domain recognition, but their performance may degrade when handling semantic ambiguity, emerging entities, and out-of-distribution text. In contrast, Large Language Models (LLMs) possess broader knowledge and stronger contextual reasoning capabilities, yet their direct application to NER often results in unstable entity boundaries and low precision. Existing hybrid methods primarily rely on entity confidence scores produced by local NER models for routing and generally handle suspected errors by directly deleting entities. This makes it difficult to prioritize genuine prediction errors under limited computational costs or effectively correct partially accurate entities.
To address these issues, this paper proposes NERLLM, a hybrid framework that balances recognition accuracy and invocation cost. By selectively rechecking low-confidence entities produced by local models, the framework efficiently verifies and corrects prediction results. NERLLM consists of two core modules: a Confidence Scorer and an LLM Rechecker. The Confidence Scorer learns span-level reliability from local model outputs and dataset statistics, prioritizing predictions that are more likely to be erroneous for LLM-based rechecking. The LLM Rechecker uses a multi-stage pipeline to correct labels, refine boundaries, and resolve conflicts. A secondary LLM evaluation is further applied to mitigate the impact of LLM hallucinations and improve correction reliability.
Experiments on the CleanCoNLL, WNUT’16, and WNUT’17 datasets show that NERLLM effectively improves Micro-F1 across different combinations of local models and LLMs, with more substantial gains on social media text containing ambiguous entities and emerging entities. In particular, Binder-large combined with DeepSeek-R1 achieved an F1 score of 61.0% on WNUT’16, an improvement of 5.8 percentage points over the local model. Meanwhile, the framework reduces unnecessary LLM invocations and achieves a favorable balance between recognition performance and computational cost, providing a new solution for efficient and accurate named entity recognition.
