List evaluators

GET /api/evaluators/

List Evaluators

cURL

curl --request GET \
  --url https://api.keywordsai.co/api/evaluators/ \
  --header 'Authorization: Bearer <token>'

Returns a paginated list of evaluators for your organization.

Authentication

All endpoints require API key authentication:

Authorization: Bearer YOUR_API_KEY

Query Parameters

Parameter	Type	Description
page	integer	Page number for pagination (default: 1)
page_size	integer	Number of items per page (default: 20)
search	string	Search evaluators by name or slug
type	string	Filter by evaluator type: llm, human, or code
score_value_type	string	Filter by score type: numerical, boolean, categorical, or comment
starred	boolean	Filter by starred status
tags	string	Filter by tags (comma-separated)

Examples

Basic List Request

Python

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

response = requests.get(url, headers=headers)
print(response.json())

With Filters and Pagination

Python

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

# Filter by type and score value type
params = {
    "type": "llm",
    "score_value_type": "numerical",
    "page": 1,
    "page_size": 10
}

response = requests.get(url, headers=headers, params=params)
print(response.json())

Search Evaluators

Python

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

# Search by name or slug
params = {
    "search": "quality",
    "starred": True
}

response = requests.get(url, headers=headers, params=params)
print(response.json())

Response

Status: 200 OK

{
  "results": [
    {
      "id": "0f4325f9-55ef-4c20-8abe-376694419947",
      "name": "Response Quality Evaluator",
      "evaluator_slug": "response_quality_v1",
      "type": "llm",
      "score_value_type": "numerical",
      "eval_class": "",
      "description": "Evaluates response quality on a 1-5 scale",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "[e-mail]"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "[e-mail]"
      },
      "created_at": "2025-09-11T09:43:55.858321Z",
      "updated_at": "2025-09-11T09:43:55.858331Z",
      "custom_required_fields": [],
      "categorical_choices": null,
      "starred": false,
      "organization": 2,
      "tags": []
    },
    {
      "id": "cat-eval-123",
      "name": "Content Quality Assessment",
      "evaluator_slug": "content_quality_categorical",
      "type": "human",
      "score_value_type": "categorical",
      "eval_class": "",
      "description": "Human assessment of content quality with predefined categories",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "[e-mail]"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "[e-mail]"
      },
      "created_at": "2025-09-11T09:44:00.000000Z",
      "updated_at": "2025-09-11T09:44:00.000000Z",
      "custom_required_fields": [],
      "categorical_choices": [
        { "name": "Excellent", "value": 5 },
        { "name": "Good", "value": 4 },
        { "name": "Average", "value": 3 },
        { "name": "Poor", "value": 2 },
        { "name": "Very Poor", "value": 1 }
      ],
      "starred": false,
      "organization": 2,
      "tags": []
    },
    {
      "id": "bool-eval-456",
      "name": "Response Length Checker",
      "evaluator_slug": "length_checker_boolean",
      "type": "code",
      "score_value_type": "boolean",
      "eval_class": "",
      "description": "Checks if response meets minimum length requirement",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "[e-mail]"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "[e-mail]"
      },
      "created_at": "2025-09-11T09:45:00.000000Z",
      "updated_at": "2025-09-11T09:45:00.000000Z",
      "custom_required_fields": [],
      "categorical_choices": [],
      "starred": false,
      "organization": 2,
      "tags": ["automation", "validation"]
    }
  ],
  "count": 3,
  "previous": null,
  "next": null,
  "current_filters": {
    "type": null,
    "score_value_type": null,
    "search": null,
    "starred": null,
    "tags": null
  }
}

Response Fields

Evaluator Object

Field	Type	Description
id	string	Unique evaluator identifier
name	string	Display name of the evaluator
evaluator_slug	string	URL-friendly identifier
type	string	Evaluator type: llm, human, or code
score_value_type	string	Score format: numerical, boolean, categorical, or comment
eval_class	string	Pre-built template class (if used)
description	string	Description of the evaluator
created_by	object	User who created the evaluator
updated_by	object	User who last updated the evaluator
created_at	string	ISO timestamp of creation
updated_at	string	ISO timestamp of last update
custom_required_fields	array	Additional required fields
categorical_choices	array	Choices for categorical evaluators
starred	boolean	Whether the evaluator is starred
organization	integer	Organization ID
tags	array	Tags associated with the evaluator

Pagination Object

Field	Type	Description
count	integer	Total number of evaluators
previous	string	URL for previous page (null if first page)
next	string	URL for next page (null if last page)
current_filters	object	Currently applied filters

Error Responses

401 Unauthorized

{
  "detail": "Your API key is invalid or expired, please check your API key at https://platform.keywordsai.co/platform/api/api-keys"
}

400 Bad Request

{
  "detail": "Invalid filter parameter: type must be one of 'llm', 'human', 'code'"
}