Making API Performance Tests More Realistic: From Endpoint Metrics to Role-Based Journeys

Daniel Balcarek's article "API Performance Testing: How to Design Realistic Tests" highlights the importance of designing load tests that reflect how users interact with a system. The author's existing test suite measured various metrics, such as p50, p95, p99 latency, throughput, error rates, warm-up phases, stepped load, and repeatable reports.

However, it did not capture the real usage patterns of users, which are typically more complex than calling the same endpoint repeatedly. To address this, the author extended the test suite with role-based journeys, sessions, think time, read and write paths, and explicit cleanup rules. The new journeys exposed issues related to authentication, API contract, cleanup, and rate limits, demonstrating that a valid journey is essential before assessing latency.

The author's existing test suite performed well in measuring p50, p95, and p99 latency, throughput, error rates, warm-up phases, stepped load, and repeatable reports. These metrics helped detect regressions, compare changes, and apply pressure to individual endpoints. However, they did not provide insights into how people actually use the system.

Users rarely call the same endpoint continuously; instead, they perform various actions across different contexts. Different roles interact with the system differently, seeing different data and performing different actions. This distinction between endpoint traffic and user behavior became the starting point for enhancing the test suite.

The author identified five extension areas for the test suite: role-based user journeys, warm-up and load stages, think time and staggered session starts, ramp-up only with no warm-up, and a repeatable mix of read and write paths. Each extension area was implemented separately, with technical thresholds derived from business requirements or production telemetry.

The final low-load comparison used a 5:3:1 mix of regular-user, agent, and administrator sessions, with each session signing in separately and waiting between actions during a ramp-up period. The runner extended an existing PowerShell performance suite, utilizing a single .NET HttpClient and cookie handler per session to record request duration after reading the full response body.

Percentiles were calculated using the nearest-rank method, keeping the browser-style cookie flow close to existing automation. The mix described sessions, not people, with one credential pair per role, resulting in low-load session comparisons rather than simulations of independent identities. The author also separated read and write tests, with the write journey creating, updating, and deleting issues while requiring stronger safety controls due to its state-changing nature.

Before interpreting latency, throughput, and error rates, the author used a staged process, first ensuring the journey was functionally and technically valid and stable when run in parallel. This distinction between validating a representative performance journey and investigating causes through system and resource metrics highlighted the importance of separating testing phases.