Slow API endpoint fix: reply first, fan out later

A slow API endpoint is often slow because it does work the caller is not waiting for. The request saves the thing the user asked for, then keeps going: notifications, counters, emails, webhooks, all before the response goes out. This post is about one such endpoint on Buffdudes, the fitness platform I run the backend for. Creating a post took about 19 seconds for the biggest accounts and ran past the app's timeout. The fix was to reply as soon as the post was saved and do the follower fan-out afterwards, in one SQL statement. Here is how I found it, what I changed, and when I'd reach for a queue instead.

Why this slow API endpoint took 19 seconds

Creating a post on Buffdudes notifies the author's followers. The original handler did that inline. It saved the post and its photo, then wrote one notification row per follower, and only then sent the response.

For most users that is invisible. A few hundred followers means a few hundred inserts, and the request still feels quick. The cost grows with the follower count, though, and the biggest accounts have about 72k followers. For them, creating a post took about 19 seconds on production. The app's request timeout is 20 seconds, and these requests went past it.

How a timeout turned into a confusing error

The user-facing symptom of this API timeout was worse than a slow spinner.

When the app gave up waiting, it showed "No Internet Connection". But the server had not given up. The post was saved and the notifications were still being written. So the user saw a failure for a post that had gone through.

The natural next step was to try again. The retry reached the server, which correctly saw the workout was already posted and answered "This Workout Is Already Posted". From the user's side, the app first said the post failed and then said it already existed.

This pattern is worth recognising. When a client times out on a request that the server completes, every retry runs into the first request's result. The server's duplicate check was right. The problem was that the first response never arrived.

Reply first: send the response once the user's work is saved

The question to ask of any slow handler is: what does the caller need before it can move on? Here, the app needs the saved post, with its photo. It does not need the followers' notifications to exist yet. Nobody is looking at the author's screen waiting for 72k other people to be notified.

So both create handlers on Buffdudes now reply as soon as the post and its photo are saved, and write the notifications after that. In Express the shape is simple. This is simplified, with illustrative function names:

router.post("/posts", async (req, res, next) => {
  try {
    const post = await createPostWithPhoto(req.user.id, req.body);
    res.status(201).json(post);

    // After the response: a fan-out failure must not surface as a failed post.
    notifyFollowers(post.authorId, post.id).catch((err) =>
      console.error("follower fan-out failed", post.id, err),
    );
  } catch (err) {
    next(err);
  }
});

Two details matter. The post save stays inside the request, so a failure there still returns an error the app can show. And the fan-out has its own error handling, because once the response is sent, an unhandled rejection has nowhere useful to go.

Fan out later, in one INSERT ... SELECT

Moving the work after the response fixed the timeout. Writing it as one statement is the other half.

Building notification rows in application code means loading every follower into Node and sending the rows back to the database. The database already has all the information it needs, so it can build the rows itself:

INSERT INTO notifications (user_id, actor_id, post_id, type, delivery_status)
SELECT f.follower_id, $1, $2, 'new_post', 'pending'
FROM follows f
JOIN users u ON u.id = f.follower_id
WHERE f.following_id = $1
  AND f.status = 'approved'
  AND u.deleted_at IS NULL;

The names are simplified, but the rules are the same as the old loop: approved followers only, only accounts that are still live, and every row starts with delivery pending. That was the requirement I held the new version to. Same rows, written in a single statement. The INSERT documentation covers the INSERT ... SELECT form.

If you make this change, compare the rows the old and new versions produce for a few real authors before shipping. A dropped condition in the WHERE clause means notifications to blocked or deleted accounts, which is a much harder bug to spot than a slow request.

When to move fan-out to a queue instead

Replying first and running the work in the same process is the smallest change that fixes the timeout, and for a single insert statement it is enough. It has limits, and I'd move to a proper background queue when any of these apply:

• The work must survive a restart. In-process work after the response is lost if the server is deployed or crashes mid-way. A queued job is retried.
• Each recipient needs its own external call. Sending push notifications through a provider, one request per device, belongs in workers with retries and rate limits.
• The work keeps growing. If one statement starts holding locks long enough to slow other writes to the table, batching it in jobs gives you control over that.
• You need to see it. A job table shows what is pending, what failed and why. Work fired after a response only shows up in logs.

I described the job table and worker setup I use for this in running slow jobs in the background with queues. The same structure works for fan-out as well as AI work.

Make retries safe either way

Whatever runs the fan-out, the create endpoint should be safe to retry. Buffdudes already refused a second post for the same workout, which is why the retry produced "already posted" instead of a duplicate. For endpoints without a natural key like that, my default is a client-generated idempotency key sent with the request and stored with the result, so a retry returns the original response. The IETF Idempotency-Key header draft describes the common shape, and Stripe's engineering blog has a good write-up on how idempotency keys make retries safe.

Notification tables grow fast

Fan-out has a second-order cost: rows. One post from a large account writes tens of thousands of notifications, and those add up quickly across a week. Plan for cleanup at the same time as the fan-out. On Buffdudes that became its own piece of work, which I wrote up in deleting millions of old rows in Postgres without locking the app.

What should an API do if a request takes longer than the client timeout?

Ideally it never gets there: reply as soon as the caller's own work is saved, and push everything else to after the response or onto a queue. Keep the client timeout above the slowest expected response, and make the endpoint idempotent so a retry after a timeout returns the original result instead of an error.

The short version

For a slow API endpoint, list what the caller needs before it can continue and send the response as soon as that is saved. Write fan-out as one set-based statement where you can, and move it to a queue when it needs retries, external calls or visibility. More notes from the same backend are in how I scaled Buffdudes to 60k users, and if your API has endpoints like this, it is the kind of work I do through my services.