HTTP Deep Dive

HTTP Version Evolution #

HTTP has evolved significantly since its creation, and each version introduces changes that directly affect how you test web applications. Understanding these differences is essential for diagnosing performance issues and writing accurate test assertions.

HTTP/1.1: The Workhorse #

HTTP/1.1 has powered the web since 1997. Its key features include persistent connections (reusing a TCP connection for multiple requests) and pipelining (sending multiple requests without waiting for responses). However, HTTP/1.1 suffers from head-of-line blocking — if one request is slow, all subsequent requests on the same connection must wait.

Testing implications:

• Browsers open 6-8 parallel connections per domain to work around head-of-line blocking
• Domain sharding (serving assets from multiple subdomains) was a common optimization — test that it works correctly
• Each request/response includes full text-based headers, adding overhead

HTTP/2: Multiplexing Revolution #

HTTP/2, standardized in 2015, introduced several breakthrough features:

• Multiplexing: Multiple requests and responses share a single TCP connection simultaneously. No more head-of-line blocking at the HTTP level.
• Header compression (HPACK): Reduces header overhead by 85-90% using a compression table.
• Server push: The server can proactively send resources before the client requests them.
• Binary framing: HTTP/2 uses binary format instead of text, making parsing faster and less error-prone.

Testing implications:

• Domain sharding actually hurts HTTP/2 performance — test with a single domain
• Server push can preload critical resources — verify it works and does not push stale content
• HTTP/2 still runs over TCP, so TCP-level head-of-line blocking remains

HTTP/3: QUIC and UDP #

HTTP/3, built on the QUIC protocol over UDP, solves TCP’s remaining head-of-line blocking problem. Key features:

• QUIC transport: UDP-based with built-in encryption (TLS 1.3 is mandatory)
• 0-RTT connection establishment: Repeat connections can send data immediately without a handshake
• Independent streams: Loss on one stream does not block others (unlike TCP where a single lost packet blocks everything)

Testing implications:

• HTTP/3 uses UDP port 443 — firewalls may block it, causing fallback to HTTP/2
• 0-RTT has replay attack risks — test that your application handles this securely
• Not all clients/servers support HTTP/3 yet — test fallback behavior

Advanced HTTP Headers #

HTTP headers carry metadata that controls caching, security, content negotiation, and cross-origin behavior. QA engineers must verify these headers on every endpoint.

Caching Headers #

What to test: Verify that dynamic API responses include Cache-Control: no-store and static assets have appropriate max-age. Test conditional requests (If-None-Match, If-Modified-Since) return 304 correctly.

CORS Headers #

What to test: Verify CORS headers are present on API responses, test that preflight OPTIONS requests return correct headers, and check that wildcard (*) is not used when credentials are required.

Security Headers #

HTTP Debugging Techniques #

Browser DevTools Network Tab #

The Network tab is the most powerful HTTP debugging tool available to QA engineers. Key features:

• Timing breakdown: DNS lookup, TCP connection, TLS handshake, TTFB, content download
• Request/response headers: Full header inspection for every request
• Filtering: By type (XHR, JS, CSS), status code, or text search
• Throttling: Simulate slow network conditions
• Preserve log: Keep network history across page navigations

curl for HTTP Debugging #

# Verbose output showing all headers and timing
curl -v https://api.example.com/users

# Timing breakdown
curl -w "DNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTLS: %{time_appconnect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n" -o /dev/null -s https://api.example.com

# Force HTTP/2
curl --http2 -v https://api.example.com

# Test specific server IP without DNS
curl --resolve api.example.com:443:10.0.0.1 https://api.example.com

HTTP Caching Verification #

# First request — should be a cache MISS
curl -I https://cdn.example.com/style.css

# Second request — should be a cache HIT
curl -I https://cdn.example.com/style.css

# Conditional request with ETag
curl -H "If-None-Match: \"abc123\"" -I https://api.example.com/data
# Should return 304 if ETag matches

Advanced HTTP Testing #

HTTP/2 Server Push Testing #

Server push allows the server to send resources before the client requests them. Testing requires verifying:

1. Pushed resources match what the page actually needs
2. Pushed resources are not stale (cache validation)
3. Push does not waste bandwidth on resources the client already has
4. Push promises use correct headers

# Check for HTTP/2 push promises using nghttp
nghttp -v https://example.com | grep "push promise"

Cookie Security Attributes #

Modern cookies should include security attributes that QA must verify:

HSTS Testing #

# Check HSTS header
curl -sI https://example.com | grep -i strict-transport

# Verify HSTS preload eligibility
# Header must include: max-age=31536000; includeSubDomains; preload

Connection Coalescing in HTTP/2 #

HTTP/2 allows reusing a connection for multiple domains if they share the same IP address and TLS certificate. This can cause unexpected behavior in testing:

• Requests to api.example.com and cdn.example.com may share a connection
• If one domain has different CORS or auth requirements, sharing causes problems
• Test that coalescing does not bypass security boundaries

Hands-On Exercise #

Audit a website’s HTTP configuration using curl and browser DevTools:

1. Check HTTP version: What version does the server support? Does it negotiate correctly?
2. Analyze caching headers: Are static assets cached? Are API responses uncached?
3. Verify CORS policy: Does the API return correct CORS headers? Test from a different origin.
4. Test security headers: Is HSTS configured? Is CSP present? Check all security headers.
5. Measure timing: What is the DNS, TLS, and TTFB breakdown? Where are bottlenecks?

# 1. Check HTTP version
curl -sI https://example.com | head -1
curl --http2 -sI https://example.com | head -1

# 2. Caching headers
curl -sI https://example.com/style.css | grep -i cache
curl -sI https://api.example.com/data | grep -i cache

# 3. CORS (test from different origin)
curl -H "Origin: https://evil.com" -sI https://api.example.com/data | grep -i access-control

# 4. Security headers
curl -sI https://example.com | grep -iE "strict-transport|content-security|x-frame|x-content-type"

# 5. Timing
curl -w "DNS:%{time_namelookup} TLS:%{time_appconnect} TTFB:%{time_starttransfer} Total:%{time_total}\n" -o /dev/null -s https://example.com

Pro Tips #

• Always check the HTTP version in use — HTTP/2 behavior differs significantly from HTTP/1.1, especially around connection management and header handling
• Test CORS from the browser, not just curl — browsers enforce CORS while curl does not, leading to false confidence when testing from the command line only
• Verify caching headers on every endpoint — incorrect caching is one of the most common causes of stale data bugs in production
• Use curl --resolve to test against specific server IPs without DNS changes — invaluable for testing deployments before DNS cutover
• Check for HSTS preload on security-critical applications — without preload, the first visit is still vulnerable to downgrade attacks

Key Takeaways #

1. HTTP version differences (1.1, 2, 3) have significant implications for performance testing and test infrastructure setup
2. Caching, CORS, and security headers are critical test verification points on every HTTP endpoint
3. Browser DevTools Network tab is the single most valuable HTTP debugging tool for QA engineers
4. Always test CORS in real browsers — API testing tools do not enforce browser security policies