PromptEvaluatorsClient

adaline.prompts.evaluators manages the evaluators attached to a prompt — LLM-as-a-judge graders, JavaScript checks, text matchers, cost, latency, and response-length guards. Evaluators are always scoped to a prompt; there is no workspace-level evaluators collection. Every method is async.

Access

from adaline.main import Adaline

adaline = Adaline()
evaluators = adaline.prompts.evaluators  # PromptEvaluatorsClient

The class is also exported directly:

from adaline.clients import PromptEvaluatorsClient

Types from adaline_api:

from adaline_api.models.evaluator import Evaluator
from adaline_api.models.create_evaluator_request import CreateEvaluatorRequest
from adaline_api.models.update_evaluator_request import UpdateEvaluatorRequest
from adaline_api.models.list_evaluators_response import ListEvaluatorsResponse

Evaluator types at a glance:

`type`	What it measures
`llm-as-a-judge`	Qualitative grading via an LLM rubric
`javascript`	Arbitrary JS/TS sandboxed check
`text-matcher`	String contains / regex / equality
`cost`	Cost threshold per row
`latency`	Response time threshold
`response-length`	Token / character bounds

list()

List evaluators attached to a prompt (paginated).

async def list(
    *,
    prompt_id: str,
    limit: Optional[int] = None,
    cursor: Optional[str] = None,
    sort: Optional[SortOrderInput] = None,
    created_after: Optional[int] = None,
    created_before: Optional[int] = None,
) -> ListEvaluatorsResponse

Example

response = await adaline.prompts.evaluators.list(
    prompt_id="prompt_abc123",
    limit=50,
)

create()

Attach a new evaluator to a prompt.

async def create(
    *,
    prompt_id: str,
    evaluator: CreateEvaluatorRequest,
) -> Evaluator

Example — LLM-as-a-judge

from adaline_api.models.create_evaluator_request import CreateEvaluatorRequest

evaluator = await adaline.prompts.evaluators.create(
    prompt_id="prompt_abc123",
    evaluator=CreateEvaluatorRequest(
        type="llm-as-a-judge",
        title="Factuality",
        settings={
            "model": "gpt-4o",
            "rubric": "Rate 1-5 for factual accuracy against the reference answer.",
            "threshold": 4,
        },
    )
)

get()

Fetch a single evaluator by ID.

async def get(*, prompt_id: str, evaluator_id: str) -> Evaluator

update()

Update an evaluator’s title, settings, or threshold.

async def update(
    *,
    prompt_id: str,
    evaluator_id: str,
    evaluator: UpdateEvaluatorRequest,
) -> Evaluator

Example

from adaline_api.models.update_evaluator_request import UpdateEvaluatorRequest

await adaline.prompts.evaluators.update(
    prompt_id="prompt_abc123",
    evaluator_id="evaluator_abc123",
    evaluator=UpdateEvaluatorRequest(settings={"threshold": 3}),
)

delete()

Permanently delete an evaluator.

async def delete(*, prompt_id: str, evaluator_id: str) -> None

Get started

API

SDKs

Proxy

Prompt evaluators

PromptEvaluatorsClient

Access

list()

Example

create()

Example — LLM-as-a-judge

get()

update()

Example

delete()

See Also

​PromptEvaluatorsClient

​Access

​list()

​Example

​create()

​Example — LLM-as-a-judge

​get()

​update()

​Example

​delete()

​See Also

PromptEvaluatorsClient

Access

list()

Example

create()

Example — LLM-as-a-judge

get()

update()

Example

delete()

See Also