Optimizing LLM costs

Managing Large Language Model (LLM) inference costs is essential for developing sustainable AI applications. This guide outlines key strategies to optimize expenditure on the ElevenLabs platform by effectively utilizing its features. For detailed model capabilities and pricing, refer to our main LLM documentation.

LLM inference costs on our platform are primarily influenced by:

• Input tokens: The amount of data processed from your prompt, including user queries, system instructions, and any contextual data.
• Output tokens: The number of tokens generated by the LLM in its response.
• Model choice: Different LLMs have varying per-token pricing. More powerful models generally incur higher costs.

Monitoring your usage via the ElevenLabs dashboard or API is crucial for identifying areas for cost reduction. You can also estimate an agent’s expected LLM cost directly from Claude or another MCP client through the hosted MCP server, which is useful for comparing models before making a change.

Choosing the most appropriate LLM is a primary factor in cost efficiency.

• Right-sizing: Select the least complex (and typically less expensive) model that can reliably perform your specific task. Avoid using high-cost models for simple operations. For instance, models like Google’s gemini-2.0-flash offer highly competitive pricing for many common tasks. Always cross-reference with the full Supported LLMs list for the latest pricing and capabilities.
• Experimentation: Test various models for your tasks, comparing output quality against incurred costs. Consider language support, context window needs, and specialized skills.

Prompt engineering is a powerful technique for reducing token consumption and associated costs. By crafting clear, concise, and unambiguous system prompts, you can guide the model to produce more efficient responses. Eliminate redundant wording and unnecessary context that might inflate your token count. Consider explicitly instructing the model on your desired output length—for example, by adding phrases like “Limit your response to two sentences” or “Provide a brief summary.” These simple directives can significantly reduce the number of output tokens while maintaining the quality and relevance of the generated content.

Modular design: For complex conversational flows, leverage agent-agent transfer. This allows you to break down a single, large system prompt into multiple, smaller, and more specialized prompts, each handled by a different agent. This significantly reduces the token count per interaction by loading only the contextually relevant prompt for the current stage of the conversation, rather than a comprehensive prompt designed for all possibilities.

For applications requiring access to large information volumes, Retrieval Augmented Generation (RAG) and a well-maintained knowledge base are key.

• Efficient RAG:

  • RAG reduces input tokens by providing the LLM with only relevant snippets from your Knowledge Base, instead of including extensive data in the prompt.
  • Optimize the retriever to fetch only the most pertinent “chunks” of information.
  • Fine-tune chunk size and overlap for a balance between context and token count.
  • Learn more about implementing RAG.

• Context size:

  • Ensure your Knowledge Base contains accurate, up-to-date, and relevant information.
  • Well-structured content improves retrieval precision and reduces token usage from irrelevant context.

Using webhook tools allows LLMs to delegate tasks to external APIs or custom code, which can be more cost-effective.

• Task offloading: Identify deterministic tasks, those requiring real-time data, complex calculations, or API interactions (e.g., database lookups, external service calls).
• Orchestration: The LLM acts as an orchestrator, making structured tool calls. This is often far more token-efficient than attempting complex tasks via prompting alone.
• Tool descriptions: Provide clear, concise descriptions for each tool, enabling the LLM to use them efficiently and accurately.

Consider applying these techniques to reduce cost: