Skip to main content
ElevenLabs Documentation Docs

Search documentation

Type to search this documentation.

On this pageOverview

Together AI

Connect an agent to a custom LLM on Together AI.

Overview

Section titled “Overview”

Together AI provides an AI Acceleration Cloud, allowing you to train, fine-tune, and run inference on AI models blazing fast, at low cost, and at production scale.

Instantly run 200+ models including DeepSeek, Llama3, Mixtral, and Stable Diffusion, optimized for peak latency, throughput, and context length.

Choosing a model

Section titled “Choosing a model”

To make use of the full power of ElevenLabs Agents you need to use a model that supports tool use and structured outputs. Together AI supports function calling for these models:

  • meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
  • meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo
  • meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
  • meta-llama/Llama-3.3-70B-Instruct-Turbo
  • mistralai/Mixtral-8x7B-Instruct-v0.1
  • mistralai/Mistral-7B-Instruct-v0.1

With this in mind, it’s recommended to use at least meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo for your ElevenLabs Agents agent.

Set up Llama 3.1 on Together AI

Section titled “Set up Llama 3.1 on Together AI”

  1. Navigate to api.together.xyz/settings/api-keys and create a new API key.

    Add Secret

  2. Once you have your API key, you can test it by running the following curl command:

    Bash
    curl https://api.together.xyz/v1/chat/completions -s \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <API_KEY>" \
-d '{
"model": "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
"messages": [{
    "role": "user",
    "content": "Hello, how are you?"
}]
}'

  3. Navigate to your AI Agent, scroll down to the “Secrets” section and select “Add Secret”. After adding the secret, make sure to hit “Save” to make the secret available to your agent.

    Add Secret

  4. Choose “Custom LLM” from the dropdown menu.

    Choose custom llm

  5. For the Server URL, specify Together AI’s OpenAI-compatible API endpoint: https://api.together.xyz/v1. For the Model ID, specify meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo as discussed above, and select your API key from the dropdown menu.

    Enter url

  6. Now you can go ahead and click “Test AI Agent” to chat with your custom Llama 3.1 model.

Suggest an edit

Propose a replacement for this page. The site team reviews it before applying any changes.

Export
Documentation menu