'Standard' vapor website drops 1.5% of requests, even at concurrency of 100!

It's certainly the subject of some debate. Personally I don't think it's typically an issue, as long as your benchmark driver (wrk in this case) doesn't use a lot of hardware resources or otherwise interfere too much with the workload under test.

But it's also a question of what exactly do you want to test? If you want to see your actual, production capabilities then of course you should test on your real production hardware and networks. Because your network probably is the bottleneck, with non-pathological test cases and production server code that's not grossly inefficient.

(and then you get into the question of where your test driver should run - is it "okay" if it still runs within your production network or do you need to stick it outside, on the public internet? And how far away? Is it okay if it runs from the same metro area, or does it need to be from a different continent? Maybe run it over cellular since that's how most of your real users access it? etc - very quickly you end up accidentally testing e.g. Qualcomm 5G modem upload latencies, rather than anything you actually control)

I think for @axello's purposes running it all on one machine is fine, although it might be marginally better to use separate machines connected by a sufficiently fast link - you don't want to be network-bottlenecked when investigating server behaviours, but you do want to test the actual network stack… although if it's all on the same machine & kernel anyway, probably there won't be any difference since from userspace a localhost TCP connection looks the same as one from another host… unless your benchmark is that latency-sensitive, of course.

1 Like