Inferring organizational duties from Persian administrative and employment laws using Large Language Models (LLMs) and few-shot learning

Document Type : Original Article

Authors

Department of Computer Engineering Ferdowsi University of Mashhad

Abstract
Abstract—Extracting organizational duties from legal documents is a critical yet challenging task, particularly in low-resource languages like Persian. This paper presents an innovative approach that integrates state-of-the-art Named Entity Recognition (NER) with advanced segmentation techniques and Large Language Models (LLMs) to accurately identify and extract duties assigned to organizations from Persian legal texts. Leveraging the power of the BERT-based model for NER, we enhance the recognition of relevant entities and ensure precise linkage to target organizations. Our method involves segmenting documents into sentences with an enhanced POS-based tokenizer, followed by the retrieval of contextually relevant segments based on the detected entities. We then explore the effectiveness of different LLM configurations, including a hierarchical approach that leverages both small and large models. Our experiments demonstrate that the hierarchical approach, combining ’Llama-3.1-8B’ and ’gpt-4o’, achieves an F1-score of 0.7901, significantly outperforming single-model approaches. This research underscores the potential of LLMs in legal text analysis, paving the way for more advanced tools in Natural Language Processing. Future work will include testing on a broader range of organizations, refining prompt engineering techniques, and enhancing model interpretability.

Keywords

Subjects

Volume 2, Special Issue on AI 4 All- 2 - Serial Number 4
1st International Conference on Artificial Intelligence
January 2025
Pages 16-25

  • Receive Date 30 April 2025
  • Revise Date 05 May 2025
  • Accept Date 05 May 2025
  • First Publish Date 05 May 2025
  • Publish Date 01 January 2025