How to handle rate limits, concurrency, and bulk letter queues in production.

3/6/2026

In backend systems architecture, scaling from dispatching ten messages a day to dispatching ten thousand messages an hour is where fragile software architectures crumble.When an application sends ten letters a week, any basic implementation works: a synchronous HTTP call inside a web request handler, basic string concatenation, and zero queue management.However, as your business scales—as thousands of month-end invoice statements need to be generated simultaneously, or as an urgent regulatory compliance notice must be dispatched to fifty thousand policyholders across the United Kingdom—the engineering reality completely transforms.If your application naively attempts to dispatch 10,000 letters by firing 10,000 concurrent HTTP requests in a raw `Promise.all()` or unconstrained thread pool, your infrastructure will experience catastrophic collapse.Your application server runs out of memory as thousands of PDF buffers flood RAM. Outgoing network sockets are exhausted. The print API's rate limiters trigger a wall of HTTP 429 Too Many Requests responses. Unhandled timeouts cause worker processes to crash, database connection pools are depleted, and critical customer communications are dropped or duplicated.The internal operational stress on engineering teams is overwhelming. Developers are woken up by PagerDuty alerts at 2:00 AM on the first of the month as billing jobs fail. Executive leadership demands to know why invoices haven't been delivered, while DevOps scrambles to restart crashed container clusters.Philosophically, high-volume postal operations should never threaten your core application stability. A robust physical mail architecture must be decoupled, resilient, and rate-regulated, allowing your platform to absorb massive volume spikes with zero server downtime.At LetterAPI, our cloud infrastructure is engineered to process massive enterprise bulk mailings effortlessly. In this deep architectural guide, we walk you through the engineering patterns required to handle rate limits, manage worker concurrency, build resilient job queues with BullMQ and Celery, and dispatch thousands of physical letters per hour with mathematical precision.

The mechanics of API rate limiting: Understanding the Token Bucket algorithm

To build client applications that gracefully navigate API limits, developers must understand how LetterAPI enforces traffic regulation.LetterAPI employs a distributed Token Bucket algorithm implemented across our global Redis infrastructure:Each API account is assigned a virtual token bucket with a maximum capacity (e.g. 100 tokens for standard tiers) that continuously refills at a steady rate (e.g. 100 tokens per second).Every incoming HTTP request consumes one token from your account's bucket. If tokens are available, the request executes instantly with zero delay.If your application bursts traffic faster than the refill rate—attempting to execute 500 requests in 50 milliseconds—the token bucket is exhausted. LetterAPI rejects excess requests with an HTTP 429 Too Many Requests status and returns three critical headers:`Retry-After: <seconds>`: The exact number of seconds to sleep before tokens are available.`X-RateLimit-Limit`: Your account's maximum requests per second threshold.`X-RateLimit-Remaining`: The remaining tokens currently available in your bucket.Client architectures that inspect these headers and regulate outgoing throughput never drop requests or fail billing runs.

The 3-step plan: Designing resilient bulk letter queues

Architecting high-scale bulk postal infrastructure requires a disciplined, decoupled approach. Follow our 3-step plan:Step 1: Decouple document generation from dispatch. Never generate PDFs and execute HTTP calls in the same synchronous worker. Separate the workflow into two stages: Stage 1 compiles data payloads and enqueues lightweight dispatch jobs; Stage 2 consumes jobs, compiles documents, and calls the API.Step 2: Regulate concurrency with client-side rate limiters. Implement client-side rate limiting (using libraries like Bottleneck in Node.js or Celery rate limits in Python) to ensure outgoing traffic never exceeds your LetterAPI tier threshold.Step 3: Implement automated exponential backoff and dead-letter queues. Wrap API requests with automated retry logic that respects `Retry-After` headers, and route persistently failing jobs (such as unresolvable addresses) to a Dead-Letter Queue (DLQ) for administrative review.

Technical blueprint: High-throughput Node.js BullMQ queue with Bottleneck throttling

Here is a complete, production-ready TypeScript implementation combining BullMQ with Bottleneck to dispatch high-volume letter batches with automatic rate-limit throttling and safe retries:

src/queues/bulk-mail-queue.ts — typescript
import { Queue, Worker, Job } from 'bullmq';
import Bottleneck from 'bottleneck';
import axios, { AxiosError } from 'axios';
interface BulkMailJobData {
  batchId: string;
  accountNumber: string;
  recipient: {
    name: string;
    company?: string;
    address_line1: string;
    city: string;
    postcode: string;
  };
  documentHtml: string;
  postageClass: 'first_class' | 'second_class';
}
// 1. Configure BullMQ Queue
export const bulkMailQueue = new Queue<BulkMailJobData>('bulk-mail-dispatch', {
  connection: { host: 'localhost', port: 6379 }
});
// 2. Configure Bottleneck Rate Limiter (Max 50 requests per second, min 20ms between requests)
const limiter = new Bottleneck({
  minTime: 20,
  maxConcurrent: 10,
  reservoir: 50,
  reservoirRefreshAmount: 50,
  reservoirRefreshInterval: 1000
});
// 3. Worker Process Consuming Jobs with Rate Limiting
export const bulkMailWorker = new Worker<BulkMailJobData>('bulk-mail-dispatch', async (job: Job<BulkMailJobData>) => {
  const { batchId, accountNumber, recipient, documentHtml, postageClass } = job.data;
  const idempotencyKey = `bulk_${batchId}_${accountNumber}`;
  return limiter.schedule(async () => {
    try {
      const response = await axios.post(
        'https://api.letterapi.uk/v1/letters',
        {
          recipient: {
            name: recipient.name,
            company: recipient.company || '',
            address_line1: recipient.address_line1,
            city: recipient.city,
            postcode: recipient.postcode,
            country: 'GB'
          },
          template_html: documentHtml,
          postage_class: postageClass,
          color: false,
          double_sided: false,
          metadata: { batchId, accountNumber }
        },
        {
          headers: {
            'Authorization': `Bearer ${process.env.LETTERAPI_API_KEY}`,
            'Content-Type': 'application/json',
            'Idempotency-Key': idempotencyKey
          },
          timeout: 15000
        }
      );
      return { letterId: response.data.id, status: 'dispatched' };
    } catch (error: any) {
      const axiosErr = error as AxiosError;
      
      // Handle 429 Rate Limiting Gracefully
      if (axiosErr.response?.status === 429) {
        const retryAfter = parseInt(axiosErr.response.headers['retry-after'] || '2', 10);
        console.warn(`[Rate Limit] 429 encountered. Backing off for ${retryAfter} seconds...`);
        limiter.adjustReservoir(-50); // Temporarily exhaust reservoir
        throw new Error(`Rate limited. Retry after ${retryAfter}s`);
      }
      throw error; // Let BullMQ retry with exponential backoff
    }
  });
}, {
  connection: { host: 'localhost', port: 6379 },
  concurrency: 15,
  limiter: {
    max: 50,
    duration: 1000
  }
});

Memory management and streaming during 50,000-letter billing runs

A common failure mode during massive end-of-month billing batches is application memory exhaustion.If a script loads 50,000 customer records from PostgreSQL into a single in-memory JavaScript array (`SELECT * FROM invoices`), Node.js easily exceeds its default 1.4GB heap limit and crashes with `FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory`.To process massive volumes safely, backend engineers must implement database cursor streaming and paginated chunking:1. Cursor-Based Querying: Use PostgreSQL cursors or Prisma pagination (`take: 500, skip: offset`) to read records in small, manageable chunks.2. Queue Pipelining: As each 500-record batch is read from the database, enqueue lightweight job descriptors into Redis and release the database records from memory immediately.3. Decoupled Processing: Worker fleets pick up jobs from Redis one by one, keeping Node.js heap memory under 150MB regardless of whether you are dispatching 100 letters or 500,000 letters.This streaming architecture ensures that your systems process massive enterprise workloads with rock-solid memory stability.

The Dead-Letter Queue (DLQ) pattern: Isolating invalid addresses

In any bulk mailing involving thousands of recipients, a small percentage of addresses will inevitably contain bad data: non-existent postcodes, missing street thoroughfares, or corrupted characters.If your background queue is not configured properly, an invalid address will fail PAF validation (HTTP 422), trigger infinite retries, clog worker threads, and delay the entire billing run.Enterprise queue design isolates these failures using Dead-Letter Queues (DLQ):Configure BullMQ or Celery to retry jobs up to 3 times with exponential backoff. If a job fails due to an unrecoverable 422 validation error, intercept the exception, bypass further retries, and route the failed payload to a dedicated `mail-dlq` queue.Your administration dashboard displays DLQ items to operations staff, highlighting the specific customer account and PAF validation diagnostics.Staff can correct the customer's address in the CRM and re-queue the letter with a single click, allowing the remaining 99% of your batch to dispatch at full speed without disruption.

The failure stakes: What happens when concurrency is unconstrained

What happens when engineering teams attempt bulk mailing without proper queue architecture?Month-end billing runs trigger massive memory spikes that crash production application servers. Web users experience HTTP 504 Gateway Timeouts, and customer transactions fail.Unconstrained API bursts trigger walls of HTTP 429 rate limit rejections. Background workers crash and restart in chaotic loops, causing duplicate letters to be printed and posted to customers.Billing cycles are delayed by days, customer support is overwhelmed by confusion, and engineering teams spend entire weekends manually patching databases and reconciling postal logs.

The success transformation: Effortless, high-volume postal scaling

Now envision your backend infrastructure powered by LetterAPI's decoupled bulk queue architecture.At the end of the quarter, your systems queue 50,000 individualized statements in minutes. Background workers process the queue with smooth, rate-regulated precision.Memory usage remains flat, database connection pools remain healthy, and web user traffic experiences zero latency impact.Every letter is validated against PAF, rendered with 1200 DPI vector clarity, and injected into the Royal Mail network on schedule.Your developers sleep soundly through month-end billing, confident in an enterprise postal engine that scales effortlessly from ten letters to ten million letters.Master high-scale bulk postal automation. Build resilient, decoupled queues with LetterAPI today.

The circuit breaker pattern: Preventing cascading failures in postal integrations

In microservice architectures, when a downstream third-party service experiences latency spikes or network partitions, client services can quickly exhaust their thread pools and memory attempting to retry requests.This phenomenon—known as cascading failure—can bring down your entire application.Implementing Michael Nygard's classic Circuit Breaker pattern protects your core business services from downstream postal API disruptions:Closed State (Normal Operation): All letter dispatch requests are forwarded directly to LetterAPI's HTTP endpoints. The circuit breaker monitors the rolling ratio of successful responses to errors (timeouts, 5xx server errors).Open State (Trip Triggered): If the error rate exceeds a pre-configured threshold (e.g. 5 consecutive timeouts or 25% failure over a 60-second window), the circuit breaker immediately trips 'Open'. Subsequent dispatch attempts bypass the network entirely, instantly routing letter jobs into a local persistent queue (Redis, RabbitMQ, or PostgreSQL) without blocking client threads.Half-Open State (Canary Testing): After a configurable sleep window (e.g. 30 seconds), the circuit breaker enters 'Half-Open'. It allows a single trial request through to LetterAPI. If the trial succeeds, the breaker resets to 'Closed'. If it fails, the breaker re-opens for another sleep duration.By wrapping your LetterAPI client in a circuit breaker, your core customer checkout, billing, and user registration flows remain lightning-fast and 100% available even during catastrophic upstream network blips.

Multi-region fallback and queue persistence strategies: Redis vs RabbitMQ vs SQS

Where should pending letter dispatch jobs be buffered before transmission across the public internet?Evaluating queue technologies against the specific requirements of physical mail automation reveals distinct architectural tradeoffs:1. Redis (with BullMQ): Outstanding throughput (100,000+ operations/second) and native delay scheduling. Redis is ideal for SaaS platforms dispatching up to 50,000 letters per day. However, because Redis is an in-memory data store, you must configure Append-Only File (AOF) persistence with `fsync everysec` to prevent data loss during sudden server power cycles.2. AWS Simple Queue Service (SQS) FIFO: Provides virtually unlimited scalability, automatic multi-AZ redundancy, and built-in at-least-once delivery guarantees with deduplication. AWS SQS FIFO queues are the recommended architecture for cloud-native enterprise deployments requiring zero server management and bulletproof durability.3. RabbitMQ (AMQP): Offers sophisticated message routing topologies and durable disk-backed exchanges. RabbitMQ excels in complex enterprise environments where letters must be routed to different queues based on postal class, tenant tier, or regulatory jurisdiction.

Automated reconciliation pipelines: Detecting and healing dropped requests

In high-volume distributed systems, relying solely on real-time HTTP responses and webhook listeners is insufficient for 100% data integrity.Ephemeral network partitions, server restarts, or unhandled exceptions can occasionally cause a webhook event to be dropped or a dispatch response to be lost in transit.To achieve zero-downtime reliability, production architectures must run an automated reconciliation job (the 'Reconciliation Cron'):Every hour, a background reconciliation task queries your local database for any letter records created more than 90 minutes ago that remain in `pending` or `queued` status.For each stalled record, the reconciliation worker calls LetterAPI's read endpoint (`GET /v1/letters/:id` or queries `GET /v1/letters?idempotency_key=...`).If the letter exists on LetterAPI's platform, the worker updates the local database status and synchronizes the carrier tracking ID. If the letter does NOT exist on LetterAPI (meaning the original request never arrived), the worker re-submits the letter using the original deterministic idempotency key.This automated self-healing loop guarantees that no letter is ever dropped, delayed, or duplicated, maintaining 100% financial and operational consistency.

Graceful degradation: Prioritizing critical statutory letters over marketing campaigns

Under severe infrastructure strain or strict daily postal dispatch cutoff deadlines (e.g. Royal Mail 17:00 intake), not all letters are created equal.A statutory Notice of Eviction or an urgent court claim warning must take precedence over a monthly promotional newsletter.Production print pipelines should implement weighted priority queuing:High-Priority Queue (Priority 1): Statutory legal notices, court documents, time-sensitive security verification PINs, and overdue dunning warnings. High-priority workers process these jobs immediately with zero batching delay.Standard Queue (Priority 2): Routine monthly invoices, billing statements, and transactional contract updates.Bulk Economy Queue (Priority 3): Marketing campaigns, shareholder circulars, and informational announcements. Bulk jobs are throttled and dispatched during off-peak hours (between 22:00 and 04:00), ensuring that high-priority transactional mail always has 100% dedicated bandwidth and zero queue congestion.

The disaster recovery (DR) runbook: What to do during cloud provider outages

Even Tier 1 cloud providers (AWS, Google Cloud, Microsoft Azure) experience regional availability zone outages.Having a documented, tested Disaster Recovery (DR) runbook ensures your engineering team can respond calmly during major internet disruptions:Scenario A: Primary Application Database Failover: If your primary Postgres cluster fails, your application triggers automated failover to the hot read-replica. Because your letter worker queries idempotency keys from the replica, no duplicate print requests are issued.Scenario B: Webhook Receiver Endpoint Failure: If your API servers crash and fail to receive incoming webhook events from LetterAPI, our webhook delivery engine automatically enters exponential retry mode, buffering all events securely for up to 72 hours. Once your servers recover, LetterAPI delivers all missed webhooks in sequential chronological order.Scenario C: Physical Mail Center Disruption: In the rare event of extreme weather or mechanical breakdown at a regional print hub, LetterAPI's automated load balancer instantly re-routes print jobs to our secondary mirrored UK print facility in the Midlands, ensuring continuous next-day Royal Mail injection with zero disruption to customer SLAs.

Automated canary testing and staged rollout of postal template changes

Modifying the HTML/CSS template of physical letters in production carries substantial real-world risk.Unlike a web page where a minor CSS layout glitch can be patched with a quick Git push, once a letter is printed and sealed into an envelope, CSS errors cannot be rolled back.A layout shift that pushes the recipient address 15mm downward will obscure the postcode behind the solid paper of the DL envelope, rendering thousands of physical letters undeliverable by Royal Mail optical sorters.To mitigate this risk, engineering teams must implement automated canary testing for template changes:Stage 1: Automated Visual Regression Testing: In your CI/CD pipeline, run headless Puppeteer to generate high-resolution PNG snapshots of the rendered letter. Use pixel-diff comparison tools (e.g. Percy or Pixelmatch) to verify that the 20mm left margin, 40mm top margin, and address window aperture remain pixel-perfect against the baseline.Stage 2: Synthetic Sandbox Proofing: Dispatch test requests to LetterAPI's sandbox environment, asserting that `paf_verified` returns `true` and downloading the virtual proof PDF to verify vector font rendering.Stage 3: Canary Physical Dispatch: Before rolling out the new template to your entire user base, dispatch a 1% canary sample to internal corporate addresses. Only after physical inspection confirms optical alignment is the template promoted to 100% production traffic.

Handling burst traffic and rate limiting during bulk end-of-month billing runs

Many software applications experience extreme burstiness in outbound postal volume.A property management platform or billing engine might send 20 letters on a typical Tuesday, but suddenly dispatch 15,000 monthly statements and service charge notices on the final business day of the month.Unmanaged burst traffic can overwhelm internal application databases, trigger API rate limit errors (HTTP 429 Too Many Requests), or exhaust connection pools.Architecting for Burst Resilience:Token Bucket Rate Limiting: Implement a token bucket rate limiter in your background worker fleet (e.g. limiting outbound API requests to 50 requests per second per worker). This smooths out spiky traffic, ensuring steady throughput without triggering rate limits.Bulk Chunking and Pagination: When querying your database for eligible billing accounts, process records in chunked batches (e.g. 500 accounts per batch) using cursor-based pagination rather than loading 15,000 records into memory simultaneously.Decoupled Worker Scaling: Autoscale your background worker containers (e.g. AWS ECS or Kubernetes Horizontal Pod Autoscaler) based on the depth of the Redis/SQS queue, ensuring that end-of-month billing runs complete in under 30 minutes with zero degradation of core application responsiveness.Zero-Drop Queue Guarantees: By configuring dead-letter queues (DLQs) with alerting triggers, any payload rejected due to structural validation errors is captured without stopping the broader batch, guaranteeing that 100% of valid transactions reach the print stream smoothly.

LetterAPIPrint and post letters in the UK, from one letter to thousands.