How to Protect Your API from IP Exhaustion Attacks: Defensive Design

How a Server Runs Out of Space Without Receiving Any Real Data

Most developers think about denial-of-service attacks in terms of bandwidth: an attacker floods your server with so much data that the network pipe fills up. But one of the most effective and technically elegant attack categories works at a completely different level — it exhausts the server's connection state table without sending large volumes of data, and without the attacker ever completing a single legitimate connection.

This is the IP exhaustion attack, most commonly realized as a SYN flood. Understanding the mechanics of how TCP connections are established, why incomplete connections consume server resources, and what specific defenses eliminate this attack surface is essential knowledge for any engineer building or operating a public-facing API or web service.

The TCP Three-Way Handshake and Why It Creates Vulnerability

Every TCP connection begins with a three-step process:

1. The client sends a SYN (synchronize) packet to the server, initiating a connection request.
2. The server responds with a SYN-ACK (synchronize-acknowledge), allocating a small amount of memory for the pending connection state and storing the client's sequence number.
3. The client sends an ACK (acknowledge), and the connection is fully established.

The vulnerability exists in step 2. When the server sends its SYN-ACK, it allocates an entry in its TCP connection table — a kernel data structure that tracks all pending and established connections. This entry remains allocated while the server waits for the client's final ACK.

In a normal interaction, the ACK arrives within a few milliseconds. But in a SYN flood attack, the attacker sends thousands of SYN packets per second using spoofed source IP addresses — fake IPs that either do not exist or belong to other machines that will never send the completing ACK. The server dutifully allocates a connection table entry for each one, sends SYN-ACK to a non-existent destination, and waits. The table fills up with half-open connections. When it is full, the server rejects all new connection attempts — including those from legitimate users — returning a "Connection refused" or simply dropping all new SYN packets.

The attacker has crashed your API without ever completing a connection, without sending any application-layer data, and often using very little bandwidth. A low-rate SYN flood using a 1 Mbps upstream connection can exhaust a server with default TCP timeout settings in under a minute.

The Anatomy of a Connection Table Exhaustion Attack

The default TCP SYN-RECEIVED timeout on most Linux kernels is 60 seconds. Without mitigations, a server can maintain roughly 8 million half-open connections per GB of kernel memory allocated to the connection table. An attacker sending 1,000 spoofed SYN packets per second would fill a typical connection table allocation in roughly 8,000 seconds — but with spoofed source IPs and no rate limiting, realistic attack tools can sustain 100,000–1,000,000 SYNs per second, which exhausts resources in seconds to minutes.

Modern attacks often combine SYN floods with application-layer techniques. An attacker might complete the TCP handshake (to bypass basic SYN flood detection) but then hold the HTTP connection open without sending a complete request — similar to a Slowloris attack. This occupies a connection-level resource slot for the duration of the idle timeout, which for HTTP keep-alive can be 30–120 seconds.

Defensive Layer 1: SYN Cookies

SYN cookies are the primary kernel-level defense against SYN floods and have been standard in Linux since the 2.6 kernel. When the connection table nears capacity, the kernel enables SYN cookie mode.

Instead of allocating a connection table entry when it receives a SYN, the server encodes all the connection state information (source IP, destination IP, ports, timestamp, and a secret seed) into the initial sequence number (ISN) of the SYN-ACK response. No memory is allocated. The server sends the SYN-ACK and forgets about it entirely.

When a legitimate client receives the SYN-ACK and sends its ACK, the server can reconstruct the connection state by decoding the ISN in the ACK packet using the same secret seed. Only then is memory allocated for the connection. Since spoofed-IP SYNs never result in an ACK (the SYN-ACK went to a non-existent address), no memory is ever consumed for spoofed connections.

Enable SYN cookies on Linux with: sysctl -w net.ipv4.tcp_syncookies=1. Persist it by adding net.ipv4.tcp_syncookies = 1 to /etc/sysctl.conf. This single setting provides substantial protection against basic SYN floods at no cost.

Defensive Layer 2: Timeout Tuning

The default TCP connection table parameters in many Linux distributions are tuned for general use, not for high-volume public services under attack. Adjusting these parameters reduces the window during which incomplete connections consume resources:

• net.ipv4.tcp_syn_retries: Number of times the server retransmits a SYN-ACK before giving up. Default is 5 (which keeps a half-open connection for approximately 3 minutes). Set to 2 for internet-facing servers to reduce this to roughly 45 seconds.
• net.ipv4.tcp_synack_retries: Similar to above but specifically for SYN-ACK retransmissions in response to inbound SYNs. Set to 2–3 for public-facing APIs.
• net.ipv4.tcp_max_syn_backlog: The maximum number of queued SYN requests. Increasing this value (e.g., to 4096 or 8192) gives the server more headroom before it starts dropping legitimate SYNs. Combine with SYN cookies for best effect.

Defensive Layer 3: Rate Limiting and IP Filtering

Kernel-level defenses protect the server from the inside out, but absorbing attack traffic at the perimeter is more efficient. Several layers of rate limiting and filtering can intercept attack traffic before it reaches your application:

• Cloud-Level DDoS Mitigation: Services like AWS Shield Standard (free, automatic), Cloudflare Magic Transit, and Akamai Prolexic are positioned in front of your infrastructure and can absorb volumetric SYN floods at their network edge — often tens to hundreds of Gbps — before the traffic ever reaches your server. For any production API, these services should be considered essential infrastructure, not optional extras.
• iptables / nftables Rate Limiting: At the Linux firewall level, you can rate-limit inbound SYN packets to a maximum per source IP per second. The iptables hashlimit or connlimit modules allow per-source-IP connection rate limits without requiring a hardware device.
• WAF (Web Application Firewall): A WAF like AWS WAF, Cloudflare WAF, or ModSecurity can detect and block attack patterns at the application layer, including slow HTTP attacks that complete the TCP handshake but then hold connections open without completing HTTP requests.

Application listen queues and SYNPROXY

Even with SYN cookies, applications can stall if the accept() queue is too small or threads cannot dequeue fast enough. Linux tcp_abort_on_overflow and application-level queue metrics belong in the same runbook. Some deployments front servers with SYNPROXY or hardware SYN cookies so the kernel never allocates full sockets for illegitimate handshakes.

Defensive Layer 4: Application-Level Protections

Beyond the network stack, the application itself should be hardened against resource exhaustion:

• Connection timeouts in your web server/API framework: Configure aggressive idle timeouts for HTTP keep-alive connections. In nginx, keepalive_timeout 5; closes idle connections after 5 seconds. In application frameworks, configure socket read timeouts to close connections that do not complete a full request within a short window.
• Request rate limiting per IP at the application gateway: API gateways (NGINX, Kong, AWS API Gateway, Envoy) support per-IP request rate limiting. Limit inbound requests from a single source IP to a rate that a legitimate user could reasonably generate. Flag and block IPs that exceed the threshold.
• Resource pools for connection-heavy operations: Use connection pool limits on database connections, backend service connections, and thread pool allocations. Ensure each resource has a maximum queue depth and a timeout, so that a flood of slow connections does not cause unbounded memory growth in the application layer.

Common Misconceptions

Misconception 1: SYN Floods Are Only a Problem for Small Servers

Connection table exhaustion scales with the size of the attack, not the server. A large server with more memory has a proportionally larger connection table, but a proportionally larger attack can still fill it. The solution is architectural (SYN cookies, perimeter filtering) rather than simply provisioning more RAM. Even large cloud instances can be brought down by a well-resourced SYN flood if SYN cookies and perimeter filtering are not in place.

Misconception 2: HTTPS Prevents SYN Flood Attacks

SYN floods attack at the TCP layer, before any TLS or HTTP processing occurs. The presence of HTTPS does not help because the attack never reaches the TLS handshake phase. TLS adds a second resource-consuming handshake on top of TCP, which actually creates an additional attack surface: TLS handshake floods that complete the TCP handshake but then repeatedly initiate and abandon TLS negotiations, consuming CPU for asymmetric cryptography operations.

Misconception 3: A CDN Automatically Protects Against All DoS Attacks

A CDN protects against attacks targeting cached content by absorbing requests at edge nodes. But if your API serves dynamic, non-cacheable responses, the CDN must forward each request to your origin server. An attacker who can generate requests that bypass CDN caching can still reach your origin with significant load. Origin server hardening is always necessary in addition to CDN deployment.

Misconception 4: IP Exhaustion Is the Same as Bandwidth Saturation

These are distinct attack categories. Bandwidth saturation floods your network pipe with raw data until the link is full — the attack success depends on having more upstream bandwidth than your connection. IP exhaustion targets the server's memory and connection state table — a few hundred kilobits per second of carefully crafted SYN packets can be sufficient. Defending against them requires different countermeasures: bandwidth saturation requires network-level volumetric scrubbing, while IP exhaustion requires protocol-level defenses like SYN cookies and timeout tuning.

Pro Tips

• Enable SYN cookies immediately on any Linux server: sysctl -w net.ipv4.tcp_syncookies=1 takes effect instantly and provides the single most cost-effective protection against SYN floods. There is no reason not to enable this on every public-facing server.
• Use ss -s to monitor connection table health: The ss -s command on Linux shows connection state counts including the number of SYN-RECEIVED connections. A rising count of SYN-RECEIVED entries that does not decrease is an early indicator of a SYN flood in progress.
• Set up an alert on connection table utilization: Export connection state metrics (via node_exporter or custom scripts reading /proc/net/netstat) to your monitoring system and alert when SYN-RECEIVED connections exceed a threshold. Early detection allows you to activate additional filtering before the table fills completely.
• Test your defenses with a controlled SYN flood: Use hping3 or scapy from a test machine to simulate a SYN flood against your staging environment. Verify that SYN cookies activate, legitimate connections still succeed during the flood, and monitoring alerts fire correctly before you are in a real incident.
• Configure nginx with a tight limit_conn directive: The nginx limit_conn and limit_req directives enforce per-IP connection and request limits at the application gateway. Combine these with a short client_header_timeout (e.g., 10 seconds) to close connections that do not complete an HTTP request header within that window.
• Put non-public API endpoints behind network-level access control: Internal APIs, admin endpoints, and management interfaces should only be accessible from known IP ranges (corporate office, VPN, specific cloud subnets). This removes them from the attack surface entirely — you cannot exhaust resources on an endpoint that drops all traffic before the IP reaches the TCP stack.

IP exhaustion attacks are precision tools that exploit the way the TCP protocol allocates memory for connection state. The defenses — SYN cookies, timeout tuning, perimeter filtering, and application-level rate limiting — are well-understood and available at no cost in standard Linux kernel and web server configurations. There is no excuse for a modern production API to be vulnerable to basic SYN floods. Check your current IP and connection status here.