Limit AI spend per user

Say each user on your free plan gets $2 of model calls a month. You don’t know what a call costs until it’s done, because it depends on the tokens used. So you hold an estimate before the call and settle the real cost after it.

1. Create the budget

Count in cents. $2 is 200:

curl -X POST https://api.limitry.com/v1/limits \
  -H "Authorization: Bearer $LIMITRY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Free model budget","subjectKind":"user","action":"llm",
       "amount":200,"unit":"cents","window":"month","failMode":"closed"}'

The fail mode is closed: if Limitry can’t be reached, refuse the call rather than spend money nobody is counting. Choose open if you’d rather keep answering users.

2. Hold, call, settle

Estimate on the high side. Whatever the call doesn’t use, Limitry gives back straight away.

import { createClient, createLimits, LimitExceeded } from "@limitry/sdk";

const limits = createLimits(
  createClient({ token: process.env.LIMITRY_API_KEY! }),
);

// Your prices, in cents per million tokens.
const priceCents = (usage: { inputTokens: number; outputTokens: number }) =>
  Math.ceil((usage.inputTokens * 300 + usage.outputTokens * 1500) / 1_000_000);

export async function answer(userId: string, prompt: string) {
  try {
    return await limits.hold(
      { subject: { kind: "user", id: userId }, action: "llm", estimate: 5 },
      async (held) => {
        const res = await llm(prompt);
        held.cost(priceCents(res.usage));
        return res.text;
      },
    );
  } catch (err) {
    if (err instanceof LimitExceeded) {
      return "You've used this month's AI allowance. It resets on the 1st.";
    }
    throw err;
  }
}

If the model call throws, hold releases the estimate, so a failed call costs the user nothing.

3. Different budgets per plan

Give paying users more by putting a second limit on a different subject kind, and sending that kind for them. For example, a pro-user limit of 2,000 cents a month, and { kind: "pro-user", id } as the subject for Pro users.

4. Know when someone runs out

Add a webhook endpoint under Settings → Webhooks. The first time a user hits the budget in a month, you receive limit.exceeded, which you can use to show an upgrade offer.