Token counting, conversation state and 429 errors in the OpenAI Responses API

Every application built on a large language model ends up solving the same three problems. It needs to know how many tokens a request will use before sending it, to stay inside the context window and the budget. It needs to carry a conversation across turns. And it needs to survive 429 errors without hammering the API or retrying a request that will never succeed.

The OpenAI API has changed shape several times, and older code samples for all three are still in wide circulation. This article covers how each one works with the current Responses API, with examples in Python, raw HTTP and C#.

How the API has changed

Three shifts explain why so much older sample code no longer works.

1. The Python SDK v1 break. openai 1.0.0 shipped in November 2023 and replaced module-level calls like openai.ChatCompletion.create(...) with a client object: client = OpenAI() and client.chat.completions.create(...). That change alone explains the You tried to access openai.ChatCompletion error. Pinning openai==0.28 is not a fix. It's a time bomb.
2. Chat Completions to Responses. OpenAI's migration guide says Chat Completions remains supported, but Responses is recommended for all new projects. Responses uses items instead of messages, takes instructions for system-level guidance, and can store state on the server.
3. Assistants API retirement. OpenAI's deprecations page lists the Assistants API for removal on August 26, 2026, with the Responses and Conversations APIs as the replacement. If you have code that creates threads and runs, it's on borrowed time, or already past it.

Counting tokens before you send

The traditional approach is to tokenize locally with tiktoken and add a hand-tuned number of tokens per message for role markers. That worked for plain text on a known model. It falls apart with tools, JSON schemas, images and files, all of which add tokens you can't see. Each new model family also brought a new magic constant.

Today there's an endpoint for this. The token counting guide says it accepts the same input format as the Responses API, and the count includes formatting tokens for roles and message boundaries that a local tokenizer won't see.

from openai import OpenAI

client = OpenAI()

payload = {
    "model": "your-model",
    "instructions": "Answer briefly.",
    "input": [
        {"role": "user", "content": "What is WAL mode in SQLite?"},
    ],
}

count = client.responses.input_tokens.count(**payload)
print(count.input_tokens)

if count.input_tokens < 50_000:
    response = client.responses.create(**payload)
    print(response.output_text)

The raw HTTP call is handy for scripts and for languages without an SDK:

curl https://api.openai.com/v1/responses/input_tokens \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model",
    "input": "What is WAL mode in SQLite?"
  }'

Keep in mind that the endpoint counts input only. It can't tell you how many tokens the model will generate in its reply, so budget for output separately, for example with max_output_tokens.

The endpoint is still a network round trip, so don't call it on every keystroke. For a rough "will this paragraph fit" check in a UI, tiktoken is fine:

import tiktoken

enc = tiktoken.get_encoding("o200k_base")
approx = len(enc.encode(user_text))   # plain text only: no tools, images or message overhead

Use the endpoint when the number drives a real decision: truncating history, picking a model, or rejecting a request. Use tiktoken when you only need a ballpark figure.

Keeping conversation state

The model itself is stateless, so the traditional approach is to send the whole message list every time. That's still one valid option, but now there are three.

Option 1: replay history yourself

You own the history, you can trim or summarize it, and with store=False nothing persists on OpenAI's side for later retrieval. The conversation state guide suggests appending the full output array back into the input, not just the text, so that reasoning items survive between turns.

history = [{"role": "user", "content": "Explain WAL mode in two sentences."}]

r1 = client.responses.create(model="your-model", input=history, store=False)
history += r1.output                                   # keep every output item

history.append({"role": "user", "content": "And what does synchronous=NORMAL risk?"})
r2 = client.responses.create(model="your-model", input=history, store=False)
print(r2.output_text)

Option 2: chain with previous_response_id

Responses are stored by default (store defaults to true, and the API reference says stored response data is kept for at least 30 days). That lets you send only the new turn:

r1 = client.responses.create(
    model="your-model",
    instructions="You are a concise database tutor.",
    input="Explain WAL mode in two sentences.",
)

r2 = client.responses.create(
    model="your-model",
    previous_response_id=r1.id,
    instructions="You are a concise database tutor.",   # NOT inherited, send it again
    input="And what does synchronous=NORMAL risk?",
)

Two catches that bite people. First, the API reference says that with previous_response_id, instructions from the previous response are not carried over. Forget to resend them and your system prompt silently disappears from turn two. Second, you send less, but you don't pay less: the guide notes that all previous input tokens in the chain are billed as input tokens. Chaining is built on stored responses, so if you opt out with store=False, plan on manual replay instead.

Option 3: the Conversations API

For a chat that has to survive across sessions or devices, create a conversation object and attach responses to it. Items from the conversation are prepended to each request, and new input and output are added automatically. According to the guide, conversation items aren't subject to the 30-day limit. You can't combine conversation with previous_response_id in the same request.

conv = client.conversations.create()

client.responses.create(model="your-model", conversation=conv.id,
                        input="Explain WAL mode in two sentences.")
r = client.responses.create(model="your-model", conversation=conv.id,
                            input="And what does synchronous=NORMAL risk?")
print(r.output_text)

My recommendation: manual replay when there's personal data or I need tight control over context, previous_response_id for short tool loops, and Conversations when the product really is a long-lived chat.

The same thing in C#

The official OpenAI .NET library exposes Responses through ResponsesClient. In the 2.x line (2.14.0 at the time of writing), these types are marked experimental, so the compiler stops you with OPENAI001 until you opt in. That's worth knowing before you put it in a shared library.

#pragma warning disable OPENAI001   // Responses types are [Experimental] in OpenAI 2.x
using OpenAI.Responses;

var client = new ResponsesClient(Environment.GetEnvironmentVariable("OPENAI_API_KEY")!);
const string Instructions = "You are a concise database tutor.";

var first = new CreateResponseOptions { Model = "your-model", Instructions = Instructions };
first.InputItems.Add(ResponseItem.CreateUserMessageItem("Explain WAL mode in two sentences."));
ResponseResult r1 = await client.CreateResponseAsync(first);
Console.WriteLine(r1.GetOutputText());

var second = new CreateResponseOptions
{
    Model = "your-model",
    PreviousResponseId = r1.Id,
    Instructions = Instructions,          // resend: not carried over
};
second.InputItems.Add(ResponseItem.CreateUserMessageItem("And what does synchronous=NORMAL risk?"));
ResponseResult r2 = await client.CreateResponseAsync(second);
Console.WriteLine(r2.GetOutputText());

StoredOutputEnabled = false is the .NET name for store: false.

429 errors: quota versus rate limit

A 429 can mean two very different problems that share one status code. The current error codes guide splits 429 into several cases. Credit balance exhausted, organization or project spend limit reached, and usage limit reached are billing problems. The guide is blunt that retrying billing, spend or quota errors won't restore access. "Rate limit reached" and slow_down are rate problems. For those, follow the Retry-After header when it's present and back off.

Both SDKs already retry 429s with exponential backoff: the Python library twice by default (max_retries), the .NET library up to three more times. So your job is mostly to tell the two cases apart and fail fast on billing errors:

import openai
from openai import OpenAI

client = OpenAI(max_retries=4)   # SDK handles backoff and Retry-After

BILLING = {"insufficient_quota", "credit_balance_exhausted",
           "organization_spend_limit_exceeded", "project_spend_limit_exceeded",
           "organization_usage_limit_exceeded"}

try:
    r = client.responses.create(model="your-model", input="ping")
except openai.RateLimitError as e:
    if e.code in BILLING or e.type == "insufficient_quota":
        raise RuntimeError("Billing or quota problem, retrying won't help") from e
    raise   # genuine rate limit after all SDK retries: shed load or queue

Checklist

• No openai.ChatCompletion or openai==0.28 pins left in the codebase.
• New code on Responses. Nothing left on the Assistants API.
• Token counts that drive decisions come from /v1/responses/input_tokens. tiktoken only for estimates.
• instructions resent on every turn when using previous_response_id.
• A conscious choice about store: the default keeps responses, so set store=False where data shouldn't persist.
• 429 handling that separates billing errors from rate limits.
• Model names in config, not hard-coded. They change faster than any of the above.

Most of the friction with this API comes from code written for an earlier version of it. Before copying a sample, check which API and SDK version it targets.