LiteLLM

Overview

LiteLLM fits AI Sonar in two common ways:

use AI Sonar as an OpenAI-compatible endpoint behind LiteLLM
put LiteLLM in front of AI Sonar when your team wants team-managed virtual keys, model-selection policy, or centralized observability

For AI Sonar, the cleanest default is LiteLLM’s custom OpenAI / OpenAI-compatible path pointed at https://api.aisonar.dev/v1.

If you specifically need Claude-native or Gemini-native request shapes, prefer AI Sonar’s dedicated native integrations instead of forcing those workflows through LiteLLM’s OpenAI-compatible abstraction.

Type: Framework or PlatformPrimary Path: OpenAI-compatible endpointSupport Confidence: Supported path

Install

pip install 'litellm[proxy]'

Proxy Configuration

Create a litellm-config.yaml like this:

model_list:
  - model_name: aisonar-gpt-5.4
    litellm_params:
      model: custom_openai/gpt-5.4
      api_base: https://api.aisonar.dev/v1
      api_key: os.environ/OPENAI_API_KEY

  - model_name: aisonar-claude-sonnet
    litellm_params:
      model: custom_openai/claude-sonnet-4-6
      api_base: https://api.aisonar.dev/v1
      api_key: os.environ/OPENAI_API_KEY

Start the proxy:

export OPENAI_API_KEY="sk-your-api-key"
litellm --config litellm-config.yaml --port 4000

Call LiteLLM Through OpenAI SDK

from openai import OpenAI

client = OpenAI(
    api_key="anything",
    base_url="http://127.0.0.1:4000"
)

response = client.chat.completions.create(
    model="aisonar-gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

Direct Python Usage

If you are using LiteLLM as a Python library instead of the proxy, keep the same AI Sonar base URL:

import litellm

response = litellm.completion(
    model="custom_openai/gpt-5.4",
    api_base="https://api.aisonar.dev/v1",
    api_key="sk-your-api-key",
    messages=[{"role": "user", "content": "Summarize this repo."}]
)

Best Practices

Prefer custom_openai for AI Sonar

Treat AI Sonar as an OpenAI-compatible endpoint unless you have a very specific reason to build a more complex provider mapping.

Use LiteLLM when you need one more gateway layer

LiteLLM makes sense when your own platform wants virtual keys, extra model-selection policy, or centralized logs in front of AI Sonar.

Keep native-provider expectations realistic

OpenAI-compatible translation layers are great for broad compatibility, but they are not the right place to promise every provider-native feature.

Troubleshooting

Connection errors

Verify api_base is exactly https://api.aisonar.dev/v1
Make sure LiteLLM can reach AI Sonar over the public internet
If you run the proxy locally, verify the OpenAI client points to your LiteLLM port instead of AI Sonar directly

Authentication errors

Check that LiteLLM is reading the right OPENAI_API_KEY
Confirm the AI Sonar key starts with sk-
Confirm the key is active in AI Sonar dashboard

Model not found

Verify the AI Sonar model name in custom_openai/<model>
Keep your LiteLLM model_name alias separate from the real AI Sonar model id

Compatibility Matrix

Coding Tools

Native SDKs

Frameworks & Platforms

Chat Apps

Overview

Install

Proxy Configuration

Call LiteLLM Through OpenAI SDK

Direct Python Usage

Best Practices

Troubleshooting

​Overview

​Install

​Proxy Configuration

​Call LiteLLM Through OpenAI SDK

​Direct Python Usage

​Best Practices

​Troubleshooting

Overview

Install

Proxy Configuration

Call LiteLLM Through OpenAI SDK

Direct Python Usage

Best Practices

Troubleshooting