Rate limit for users - Docs

This feature is only available for AI gateway users.

The user-level rate limit helps you control the LLM usage of each user. You can set a rate limit of requests per minute for each user and we will block the API calls that exceed the limit.

Why user-level rate limit?

Prevent random users from abusing your system
Control the cost of your LLM usage

How to set user-level rate limit

"customer_params": {
        "customer_identifier": "xxxx", // The user you want to set the rate limit for
        "rate_limit": 100 // The rate limit of the user, requests per minute
    },

Detailed example

AI gateway
OpenAI SDK
Other SDKs

import requests
def demo_call(input, 
              model="gpt-4o-mini",
              token="YOUR_KEYWORDS_AI_API_KEY"
              ):
    headers = {
        'Content-Type': 'application/json',
        'Authorization': f'Bearer {token}',
    }

    data = {
        'model': model,
        'messages': [{'role': 'user', 'content': input}],
        "customer_params": {"customer_identifier": "xxxx", "rate_limit": 100}
    }

    response = requests.post('https://api.keywordsai.co/api/chat/completions', headers=headers, json=data)
    return response

messages = "Say 'Hello World'"
print(demo_call(messages).json())

fetch('https://api.keywordsai.co/api/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer YOUR_KEYWORDS_AI_API_KEY'
  },
    body: JSON.stringify({
        model: 'gpt-4o-mini',
        messages: [{role: 'user', content: "Say 'Hello World'"}],
        "customer_params": {"customer_identifier": "xxxx", "rate_limit": 100}
    })
})
.then(response => response.json())
.then(data => console.log(data));

from openai import OpenAI

client = OpenAI(
    base_url="https://api.keywordsai.co/api/",
    api_key=YOUR_KEYWORDSAI_API_KEY,
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role":"user", "content":"Tell me a long story"}],
    extra_body={"customer_params": {"customer_identifier": "xxxx", "rate_limit": 100}}
)

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://api.keywordsai.co/api",
  apiKey: process.env.KEYWORDS_AI_API_KEY,
});

const response = await client.chat.completions
  .create({
    messages: [{ role: "user", content: "Say this is a test" }],
    model: "gpt-4o-mini",
    // @ts-expect-error
    customer_params: { customer_identifier: "xxxx", rate_limit: 100 },
  })
  .asResponse();

console.log(await response.json());