An Open-Source Plugin For E-Mail Automation Using Large Language Models – A Design and Specification Proposal

This work presents an open-source plugin for email classification, categoriza-tion, and automated response generation using a hybrid approach based on Large Language Models (LLMs). Existing commercial email assistants largely rely on proprietary, cloud-based LLMs, which raise concerns related to opera-tional costs, data privacy, and limited customization—particularly in public administration contexts. To address these limitations, the proposed plugin inte-grates keyword-based email categorization with Retrieval-Augmented Genera-tion (RAG) to produce context-aware responses using both local and proprietary LLMs. The plugin supports timeline tracking for each email, covering all stages from categorization to response generation and human validation. The main sci-entific contributions of this work include: (i) the design of an open-source, lo-cally deployable email automation plugin; (ii) the integration of RAG with local LLM inference; and (iii) a comparative evaluation of open-source and proprie-tary LLMs in a real-world public administration scenario. The proposed solu-tion was evaluated in a realistic municipal setting involving two public service departments. Email classification and categorization performance was assessed using accuracy metrics, while response generation quality was evaluated through human supervision based on correctness, relevance, and adequacy of the generated replies. Experimental results show classification and categoriza-tion accuracy rates of approximately 94%. For response generation, a local LLM (Ollama) was compared with proprietary models, namely ChatGPT and Gemini, achieving comparable accuracy while offering significant advantages in terms of cost efficiency and data sovereignty. To foster further research and reproducibility, the source code, pretrained models, and experimental bench-marks are publicly available in an open-source GitHub repository.