Proxy Latency vs Throughput: What to Benchmark

Proxy latency vs throughput is not a contest in which one metric replaces the other. Latency describes how long an operation takes. Throughput describes how much useful work or data completes over time. A proxy can return the first byte quickly but transfer a large file slowly, or show higher per-request latency while completing more requests through connection reuse and concurrency.

The correct benchmark starts with the workload. Interactive checks care about response time. Bulk collection may care about records per minute and bytes per second. Session workflows care about consistent tail latency and completion, not only a fast average.

Break Latency Into Stages

End-to-end response time contains several waits:

• Local queue or connection-pool acquisition.

• DNS resolution for the gateway or destination.

• TCP connection to the proxy gateway.

• Proxy protocol negotiation and authentication.

• Exit connection to the destination.

• TLS handshake for encrypted destinations.

• Time to first byte from the destination.

• Response transfer and application processing.

Measure as many stages as the client exposes. If total time rises, stage data shows whether the problem is local queueing, gateway reachability, route distance, destination generation, or payload transfer.

Time to first byte is useful but not a pure proxy metric. It includes destination processing and network travel. Compare equivalent destinations and requests.

Understand Throughput

Throughput can be expressed as completed requests per second, valid records per minute, pages per hour, or payload bytes per second. Choose useful work rather than raw attempts. A system that sends more requests but produces the same number of valid records has higher traffic, not higher business throughput.

Bandwidth is the capacity or transfer rate available on a path. Throughput is what the workload actually achieves after protocol overhead, latency, errors, retries, and application bottlenecks. Advertised network capacity does not guarantee a single client will reach it.

For large responses, report both compressed bytes on the wire and uncompressed application size when possible. Compression can reduce traffic while adding CPU work.

Why Averages Are Not Enough

Latency distributions are often skewed. Most requests can be fast while a small group takes many seconds. Report median, p90, p95, and p99 along with error rate. Tail latency strongly affects multi-request tasks because a workflow waits for its slowest components.

Throughput should also be measured over intervals. A high five-second burst may be followed by a long stall. Show sustained performance, queue growth, and completion by minute or hour.

Always include failed and timed-out work. A benchmark that measures only successful responses can make an unstable system look fast by discarding the slowest attempts.

Concurrency Connects the Metrics

Concurrency allows work to overlap. If one request takes one second, ten parallel requests can theoretically complete more work per second than one sequential worker. Real systems encounter limits in the client, proxy account, exit network, destination, bandwidth, and downstream parser.

As concurrency rises, throughput may improve until queueing and errors increase. Latency usually rises near saturation. The useful operating point is before that knee, with headroom for normal variation.

Measure the same workload at gradually increasing concurrency. Plot completed valid tasks, first-attempt success, p95 latency, queue time, bandwidth, and cost. Stop increasing when extra workers produce little useful throughput or materially worsen reliability.

Design a Fair Benchmark

Use representative destinations

Include a controlled diagnostic endpoint and one or more authorized business destinations. A tiny static file cannot predict a dynamic page or larger response.

Keep comparison variables equal

Use the same client, protocol, location, session policy, request headers, payload, concurrency, and time window. Comparing a nearby static IP with a distant rotating residential exit does not isolate the provider or protocol.

Test cold and warm paths

Cold tests include DNS, new TCP connections, proxy negotiation, and TLS. Warm tests reuse connections and caches. Production may contain both, so report both.

Use enough repetitions

One speed test is a snapshot. Run a representative sample over time and retain raw stage measurements. Avoid aggressive tests against systems that have not authorized the load.

Validate response content

Confirm that each response is the expected page or record. A fast error page is not useful throughput.

Record location and exit

Route distance affects latency. For rotating pools, record observed exits and requested regions so a change in distribution is not mistaken for an infrastructure improvement.

A proxy latency test can provide a quick comparison, while a production benchmark should use the actual client, payload, destination, session behavior, and concurrency.

Test Variability Across Time

Network routes, residential exit availability, destination load, and local infrastructure change during the day. A single benchmark window can favor one region or route by chance. Run the same proxy latency vs throughput test at several planned times, then compare distributions rather than selecting the fastest run.

Keep the workload small enough to remain responsible and statistically interpretable. If the system changes during the test, record deployment, endpoint, routing, and configuration versions. Do not merge measurements from different protocols or locations without labeling them.

Use a fixed control request to identify broad network changes, but do not let the control replace the business workload. If the control remains fast while production pages slow down, the destination or payload may be responsible. If both change together, investigate the shared route, gateway, and client environment.

Retain the raw observations long enough to reproduce every published summary, chart, recommendation, and comparison accurately.

Metrics for Different Workloads

Small API-style requests

Focus on connect time, TLS time, time to first byte, total latency, success, and requests per second. Connection reuse can make a large difference.

Large files or pages

Measure first byte separately from transfer duration, bytes per second, compression, total bytes, incomplete transfers, and total deadline.

Browser journeys

Measure navigation milestones, required-resource completion, page validity, session stability, and whole-journey time. One fast HTML response does not mean the page is usable.

Data collection pipelines

Report valid records per minute, bytes per valid record, first-attempt success, retry amplification, parser time, queue delay, and end-to-end freshness.

Location-sensitive monitoring

Add location accuracy, content correctness, and result variance by region. A faster route that returns the wrong market is not an improvement.

Diagnose Common Benchmark Results

Low latency, low throughput

The client may be running sequentially, waiting on a small connection pool, or spending time in parsing and storage. Inspect queue and downstream stages.

High latency, acceptable throughput

Concurrency may hide slow individual requests. This can be acceptable for background jobs but poor for interactive or session-dependent work. Watch resource use and tail behavior.

Throughput rises, then success falls

The system has crossed a capacity or policy threshold. Reduce concurrency, add queues, and identify whether the client, gateway, exit, destination, or storage is saturated.

Fast first byte, slow completion

Transfer size, bandwidth, compression, or read behavior dominates. Test a small and large response and record bytes over time.

Wide variance between exits

Segment by region, ASN, exit, and time. A global average can hide a constrained pool. Use enough samples before changing weights or quarantining an address.

Improve Performance Responsibly

Reuse connections when the client and destination support it. Place workloads near the proxy gateway when appropriate. Select the closest acceptable exit region rather than a distant location with no business purpose. Reduce unnecessary payloads, cache unchanged content, and avoid duplicate requests.

Use bounded concurrency, per-destination rate controls, timeouts, and conservative retries. Follow destination instructions and permissions. Do not increase rotation or parallelism to evade access controls.

Optimize the slowest meaningful stage. If destination generation takes two seconds, saving ten milliseconds in local DNS will not change the user outcome. If workers wait in a queue, changing proxy type may not help.

Reporting Checklist

• Workload, destinations, regions, and observation window.

• Client, protocol, session policy, and connection reuse.

• Request and response sizes.

• Cold and warm sample counts.

• p50, p90, p95, and p99 stage and total latency.

• First-attempt and eventual success.

• Valid tasks or records per time interval.

• Bytes and cost per valid task.

• Concurrency, queue time, and local resource use.

• Location and content validation results.

Frequently Asked Questions

Is lower latency always better?

Only if reliability, location, content, and cost remain acceptable. A very fast error or wrong regional page has little value.

Is bandwidth the same as throughput?

No. Bandwidth describes path capacity or transfer rate, while useful throughput reflects completed work after latency, errors, overhead, and application limits.

Why does adding concurrency increase latency?

Requests compete for client resources, gateway capacity, exits, destination capacity, and downstream processing. Queueing grows as the system approaches saturation.

What is a good proxy response time?

It depends on route distance, destination, payload, protocol, and workload. Establish a representative baseline and service objective using percentiles, not a universal threshold.

Conclusion

Proxy latency vs throughput becomes useful when the metrics are tied to the job. Break latency into stages, measure valid work rather than raw traffic, include tail behavior and failures, and test concurrency gradually. Quick checks can reveal obvious problems, but production decisions need representative destinations, payloads, sessions, and regions. Optimize for reliable completion and data quality, not for the smallest isolated number on a speed-test screen.