Ragas

Overview

Ragas can evaluate AI Sonar-backed applications by passing an OpenAI-compatible AsyncOpenAI client into llm_factory.

Type: Evaluation frameworkPrimary Path: OpenAI-compatible Chat CompletionsSupport Confidence: Supported OpenAI-compatible path

Environment

export API_KEY="sk-your-api-key"

Example Evaluator

import os

from openai import AsyncOpenAI
from ragas.llms import llm_factory
from ragas.metrics import DiscreteMetric

client = AsyncOpenAI(
    api_key=os.environ["API_KEY"],
    base_url="https://api.aisonar.dev/v1",
)
llm = llm_factory("claude-sonnet-5", client=client)

metric = DiscreteMetric(
    name="summary_accuracy",
    allowed_values=["accurate", "inaccurate"],
    prompt="Evaluate whether the response is accurate. Answer only accurate or inaccurate.\n\nResponse: {response}",
)

Use llm in Ragas metrics and testsets the same way you would use an OpenAI SDK-backed model.

Endpoint Notes

Ragas uses the OpenAI SDK client path here. Native AI Sonar Responses, Anthropic Messages, and Gemini routes are best used through evaluation runners that support those request shapes directly.

VoltAgent Guardrails

Compatibility Matrix

Coding Tools

Native SDKs

Frameworks & Platforms

Chat Apps

Overview

Environment

Example Evaluator

Endpoint Notes

​Overview

​Environment

​Example Evaluator

​Endpoint Notes

Overview

Environment

Example Evaluator

Endpoint Notes