Why Database Traffic Isn't the Same Problem as API Traffic

Load at the backend and API layer is measured in requests per second, concurrent connections, and bandwidth. Load at the database layer is a different set of measurements entirely, and the two are genuinely decoupled: a backend can serve light API traffic while its database is being crushed by one enormously expensive query, or serve heavy API traffic that's almost entirely absorbed by an in-memory cache, barely touching the database at all. Understanding database load requires its own vocabulary, because it doesn't reduce to "how many requests is the API handling."

The four independent dimensions of database load

Concurrent connections are a hard, fixed resource, not a soft one. PostgreSQL's own documentation states plainly that it "sizes certain resources based directly on the value of max_connections. Increasing its value leads to higher allocation of those resources, including shared memory." The default is around 100, and it's set only at server start. This is a ceiling independent of how busy the database feels: 100 nearly idle connections hit it just as easily as 100 genuinely busy ones.

Query throughput is a separate axis from connection count entirely. A database can have very few connections, each running an expensive query (high load, low connection count), or hundreds of idle connections doing essentially nothing (low load, high connection count, but still consuming the fixed resource above). Connection count alone is a poor proxy for how loaded a database actually is.

Lock contention is independent again. PostgreSQL uses MVCC, so reads never block writes and writes never block reads, but writers can still block other writers. The documentation is explicit: if a transaction takes a write lock on a row, no other transaction can update, delete, or acquire a write or read lock on that same row until the holding transaction ends. A system with low connection count and low query throughput can still be crippled by a handful of transactions serializing on the same hot rows.

Replication lag is load-adjacent rather than load itself: a resource consumed by load, in multi-node setups. Postgres tracks it via write-ahead-log positions on the primary versus what a standby has received and replayed, and the documented tunable max_standby_streaming_delay governs how long a standby tolerates lag before it starts canceling conflicting queries. High write load on the primary directly inflates lag on every replica downstream of it, a dimension with no real analog at a stateless API tier.

Why database connections are expensive in a way HTTP requests aren't

PostgreSQL is explicitly process-per-connection, not thread-per-connection, and this is a deliberate architectural choice, not an implementation detail. Its own documentation: "To achieve this it starts ('forks') a new process for each connection... client and associated server processes come and go" as connections open and close. Each connection is a full OS process with its own memory allocation. MySQL differs in mechanism (it's thread-per-connection) but shares the cost profile: MySQL's own engineering blog gives a planning guideline of roughly 10MB of memory per connection on average.

This is the real reason "just open more connections" is a reasonable scaling move for a stateless API server but a bad one for a database specifically. An API request is typically handled by a lightweight thread, async task, or event-loop callback with minimal fixed overhead, and API servers scale by adding more of those, cheaply. A database connection is a heavyweight, stateful, OS-level resource, a full process in Postgres, a dedicated thread carrying real session state in MySQL, that cannot be created or destroyed cheaply and is capped by both a hard configuration ceiling and real available memory. This is precisely why connection pooling exists as close to mandatory infrastructure past small scale, rather than an optional optimization: it decouples the number of logical connections an application wants from the number of expensive backend connections actually held open against the database. (The general idea behind pooling, and the specific tools that implement it, are covered in their own post.)

The honest finding: database rate limiting mostly doesn't exist

API-layer rate limiting is a named, purpose-built pattern with dedicated algorithms and tooling. The honest finding at the database layer is that nothing directly equivalent exists as a core feature in the most widely used databases.

PostgreSQL's max_connections is admission control, a hard ceiling with no smoothing or backoff semantics, not a rate limiter: once it's hit, new connections are simply rejected outright. statement_timeout caps how long a single query is allowed to run, which bounds the cost of one query but says nothing about how many queries can arrive per second. MySQL comes closer with Resource Groups (MySQL 8.0), which let administrators bind threads to CPU affinity and priority levels, but this governs CPU scheduling priority, not request rate, and MySQL's own documentation describes it in exactly those terms rather than as throttling.

What actually functions as database-level throttling, in practice, is indirect: connection pool sizing becomes the de facto admission-control point, since it caps how many operations can be in flight against the database at once, and query timeouts bound the cost of any single operation that does get through. There is no direct database analog to a token bucket or sliding window counter guarding query arrival rate, the way there is at the API layer. This asymmetry is worth stating directly rather than assuming the two layers work the same way just because both involve "traffic."

Applying it

• Monitor connection count, query throughput, lock contention, and replication lag as separate metrics, not as one combined "database load" number. A database can be in trouble on any one of these while looking fine on the others.
• Never scale a database's effective connection count the way a stateless API server scales instances. Use a connection pool to cap real backend connections regardless of how many application instances are running.
• Don't look for a database-native rate limiter that mirrors API rate limiting; it largely doesn't exist. Build the equivalent protection through connection pool sizing and query timeouts instead.
• When a backend feels fine but something downstream is slow, check the database's own metrics directly rather than assuming API-layer request volume is a reliable proxy for database load. It isn't.