[2603.20313] Semantic Tool Discovery for Large Language Models: A Vector-Based Approach to MCP Tool Selection

Computer Science > Software Engineering arXiv:2603.20313 (cs) [Submitted on 19 Mar 2026] Title:Semantic Tool Discovery for Large Language Models: A Vector-Based Approach to MCP Tool Selection Authors:Sarat Mudunuri, Jian Wan, Ally Qin, Srinivasan Manoharan View a PDF of the paper titled Semantic Tool Discovery for Large Language Models: A Vector-Based Approach to MCP Tool Selection, by Sarat Mudunuri and 3 other authors View PDF HTML (experimental) Abstract:Large Language Models (LLMs) with tool-calling capabilities have demonstrated remarkable potential in executing complex tasks through external tool integration. The Model Context Protocol (MCP) has emerged as a standardized framework for connecting LLMs to diverse toolsets, with individual MCP servers potentially exposing dozens to hundreds of tools. However, current implementations face a critical scalability challenge: providing all available tools to the LLM context results in substantial token overhead, increased costs, reduced accuracy, and context window constraints. We present a semantic tool discovery architecture that addresses these challenges through vector-based retrieval. Our approach indexes MCP tools using dense embeddings that capture semantic relationships between tool capabilities and user intent, dynamically selecting only the most relevant tools (typically 3-5) rather than exposing the entire tool catalog (50-100+). Experimental results demonstrate a 99.6% reduction in tool-related token consumption ...