What are Token Limits?
Token limits define the maximum number of tokens a language model can process per request, or the maximum tokens a user can consume within a billing period. Tokens are the fundamental unit of LLM processing -- roughly 3/4 of an English word. Token limits exist at two levels: model-level limits (the context window) and platform-level limits (usage caps for cost control).
Model-level token limits are set by the model architecture. Claude has a 200K token context window. GPT-4o supports 128K. These are hard limits -- you cannot send more tokens than the model supports. Platform-level limits are soft business constraints: a SaaS might limit free-tier users to 100K tokens per day or standard users to 1M tokens per week.
Understanding token limits is essential for both cost management and application design. Every token costs money on LLM APIs, so runaway usage can quickly become expensive. For AI agents that make many sequential LLM calls within a single task, token consumption can add up rapidly.
How Token Limits Work
- Input tokens -- The tokens in your prompt, including system prompt, conversation history, and any retrieved documents
- Output tokens -- The tokens in the model's response
- Per-request limits -- Maximum input and output tokens for a single API call, set by the model
- Rate limits -- Maximum tokens per minute or per day, set by the API provider
- Usage caps -- Maximum tokens per billing period, set by the platform for cost control
Why Token Limits Matter
Token limits directly affect cost and capability. More tokens means higher API bills. But too-restrictive limits mean the agent cannot handle complex tasks that require long contexts, multiple tool calls, or processing large documents. The balance between capability and cost is a core product decision for any AI platform.
For managed platforms, token limits also protect against abuse and ensure fair usage among tenants. Without caps, a single user running an expensive loop could consume the entire API budget.
How KiwiClaw Manages Token Limits
KiwiClaw enforces weekly usage caps through its LLM proxy using atomic Redis operations. Standard tier users get a generous weekly allowance with both Auto (cost-efficient) and MAX (premium) models. Usage is tracked in real-time, visible in the dashboard, and soft-limited -- the agent finishes its current response even if it crosses the cap threshold. Users can purchase additional credits if they need more capacity.
Related Terms
Frequently Asked Questions
What are token limits in AI?
Token limits define the maximum number of tokens an LLM can process per request (context window) or that a user can consume within a billing period (usage caps). They exist for both technical and cost-control reasons.
How are tokens counted?
Tokens are text fragments -- roughly 3/4 of an English word. Both input tokens (your prompt and context) and output tokens (the model response) count toward limits. Each API call tracks input and output tokens separately.
What happens when you hit KiwiClaw token limits?
KiwiClaw uses soft limits -- the agent finishes its current response even if it crosses the cap threshold. Usage resets weekly. Users can purchase additional credits or upgrade their plan for higher limits.