This article is an adaption of this article I wrote back in 2021 but I don’t really use javascript anymore so I thought I’d rewrite it in Typescript and I have a few more tricks up my sleeve now.

Rate limits

Rate limits are a common problem when working with APIs. They are used to prevent abuse and ensure that the API is available to all users. When you hit a rate limit, the API will return a 429 status code, which means that you have made too many requests in a given time period.

There are a few ways to deal with rate limits. One common approach is to use exponential backoff, which means that you wait a certain amount of time before retrying the request. Another approach is to use a queue to manage the requests and ensure that you don’t exceed the rate limit.

In this article, we will look at how to manage rate limits in Typescript using the Bottleneck library.

Bottleneck

”Bottleneck is a lightweight and zero-dependency Task Scheduler and Rate Limiter for Node.js and the browser.” its been around for a while (it’s written in CoffeeScript!!).

Every now and then you might need to send a large number of requests to a REST API that is rate-limited or, in some cases, unable to handle more than a certain number of requests a second. Often when you send a request to an API for a specific set of data you recieve a list of ids back. You then need to send a request for each id to get the data you need. This can be a problem if you have a large number of ids to get data for.

The Problem

Imagine we were writing a script to download a list of ingredients from an external api to save in our own database. When we call the ‘/ingredients’ endpoint we get back a list of ids:

type Ingredient = {
  id: number;
  name: string;
  description: string;
  price: number;
}

async function getIngredients(): Promise<Array<Ingredient['id']>> => {
  const response = await fetch('https://api.example.com/ingredients');
  const data = await response.json();
  return data;
}

const ingredients = await getIngredients();
console.log(ingredients, 'ingredients'); // [1, 2, 3, 4, 5, ...1500]

The response doesn’t have the required detail for our needs. There are a few ways you might consider handling this. You could either loop over the ingredient ids using async/await to get each detail one after another (Method 1). Or you you might try using Promise.all() or Promise.allSettled (Method 2). These both have their advantages and drawbacks.

Method 1: Looping over the ids

type Result<T, E> = {
  result: T; 
  error: null;
} | {
  result: null;
  error: E;
}

class IngredientApiError extends Error {
  constructor(public statusCode: number, public response: unknown, message: string) {
    super(message);
    this.name = 'IngredientApiError';
  }
}

async function getIngredientDetails(ingredientIds: Array<Ingredient['id']>): Promise<
  Array<
    Result<Ingredient, IngredientApiError>
  >
>{
  const results: Result<Ingredient, IngredientApiError> = [];
  for (const id of ingredientIds) {
      const response = await fetch(`https://someapi.com/api/ingredients/${id}`);
      if (!response.ok) {
        const text = await response.text() // read text because the error response might not be json
        const error = new IngredientApiError(response.status, text, `Failed to get ingredient with id ${id}`);
        results.push({result: null, error});
      }
      const ingredient = await response.json(); 
      ingredients.push(ingredient);
  }
  return results; 
}

This is nice, readable, and maintainable. But it is not very performant at all as we only send a new request when the last one has resolved. Also there is a chance, albeit slim, that our requests would be rate-limited by external api and we would end up with a lot of 429 errors or 5xx (for badly written apis) when sending large numbers of requests.

Method 2: Using Promise.all()

async function getIngredientById(id: number): Promise<Result<Ingredient>> {
  const response = await fetch(`https://someapi.com/api/ingredients/${id}`);
  if (!response.ok) {
    const text = await response.text() // read text because the error response might not be json
    return {
      error: new IngredientApiError(response.status, text, `Failed to get ingredient with id ${id}`),
      result: null
    }
  }
  return {
    error: null,
    result: await response.json()
  };
}

async function getIngredientDetails(ingredientIds: Array<Ingredient['id']>): Promise<
  Array<
    Result<Ingredient, IngredientApiError>
  >
>{
  // if your function has a second argument do [1, 2, 3].map(id => getIngredientById(id, secondArg)) instead
  return Promise.all(ingridientIds.map(getIngredientById));
}

This method is great. It is simple and clean and fast, however, it has one major drawback, the requests are going to be executed extremely quickly so requests will definitely fail due to rate-limiting on the external api or you will get ECONNRESET from the external api server.

Alternatively, we could also have used Promise.allSettled() but because we are using the Result pattern we are already handling errors gracefully.

We can expect a lot of 429 errors or 5xx errors when sending large numbers of requests. This is where Bottleneck comes in.

Using Bottleneck

We can use a bottleneck ‘limiter’ to schedule requests in a queue and we can tweak how many we allow per minute amongst many other things. I encrouage you to read the bottleneck docs to get an understanding of how powerful the package is. Anyway, this how our code should look once we add a limiter.

import Bottleneck from 'bottleneck';

const limiter = new Bottleneck({
    minTime: 250, //minimum time between requests
    maxConcurrent: 40, //maximum concurrent requests
});

async function getIngredientDetails(ingredientIds: Array<Ingredient['id']>): Promise<
  Array<
    Result<Ingredient, IngredientApiError>
  >
>{
  return Promise.all(ingridientIds.map(id => limiter.schedule(() => getIngredientById(id)));
}

Now this is epic, we can dispatch requests much faster than simply using Async/Await, we handle and detail all errors, and we can tweak the limiter to get the best performance possible and ensuring that we retrieve all the data we need.

Also since we are using the result pattern we can later on decide to retry failed requests or write them to a log.