Http2 with fastify

lewis · December 2, 2021, 5:17am

I recently started looking into some perf optimizations for my app, and found out that fastify has support for http2 over plaintext: HTTP2

I was hoping enabling this would give me some quick perf wins for relatively little effort, but I still haven’t managed to get this to work.

Reading Public Networking, I went ahead and removed the http handler from port 443, but when I tested out the app by making a request from my browser, it starts a file download for some strange reason…

I extracted out a minimal test app deployed here to test out: https://fastify-h2c-test.fly.dev/
The code behind it is in this repo: GitHub - lewisl9029/fastify-h2c-test

Am I missing some additional configuration? Or if there’s some incompatibility between fly and fastify’s http2 implementation?

Thanks!

charsleysa · December 2, 2021, 10:06pm

Hi @lewis

AFAIK fly doesn’t support h2c.

kurt · December 2, 2021, 10:15pm

That is correct! We handle http2 for you, and when you remove the HTTP handler we don’t send the H2 ALPN bits.

You may not need it, if you change set type="requests" under concurrency we pool connections between our proxy and your VM. It will probably perform better than TCP + h2c even when we do support specifying ALPN extensions.

lewis · December 3, 2021, 2:38am

Ah understood, though in that case this part of the docs might be a bit misleading:

If your application stack has good HTTP/2 support (like Go), you will get better performance accepting TCP connections directly, and using the TLS handler to terminate SSL. Your application does need to understand h2c for this to work, however.

Maybe it’s out of date?

In any case, setting type=requests did seem to improve performance of highly concurrent requests by a noticeable amount, but there still seems to be a rather significant discrepancy (anywhere between 100ms - 300ms) between the response time reported by fastify’s logging vs the response time reported by browser devtools.

Here’s a short excerpt from devtools:

(note the average response time of 250-300ms, which gets worse for later requests)

Here’s a short excerpt from my logs:

2021-12-03T02:19:52.001 app[aa648dc0] sea [info] {"level":30,"time":1638497992001,"pid":535,"hostname":"aa648dc0","duration":32.06759100034833,"msg":"s3 fetch"}
2021-12-03T02:19:52.002 app[aa648dc0] sea [info] {"level":30,"time":1638497992002,"pid":535,"hostname":"aa648dc0","reqId":"req-o1","res":{"statusCode":200},"responseTime":33.224192999769,"msg":"request completed"}
2021-12-03T02:19:52.003 app[aa648dc0] sea [info] {"level":30,"time":1638497992003,"pid":535,"hostname":"aa648dc0","duration":40.0952660003677,"msg":"s3 fetch"}
2021-12-03T02:19:52.008 app[aa648dc0] sea [info] {"level":30,"time":1638497992008,"pid":535,"hostname":"aa648dc0","duration":36.054190000053495,"msg":"s3 fetch"}
2021-12-03T02:19:52.009 app[aa648dc0] sea [info] {"level":30,"time":1638497992009,"pid":535,"hostname":"aa648dc0","reqId":"req-o4","res":{"statusCode":200},"responseTime":37.24390200013295,"msg":"request completed"}
2021-12-03T02:19:52.010 app[aa648dc0] sea [info] {"level":30,"time":1638497992009,"pid":535,"hostname":"aa648dc0","duration":41.673618000000715,"msg":"s3 fetch"}
2021-12-03T02:19:52.010 app[aa648dc0] sea [info] {"level":30,"time":1638497992010,"pid":535,"hostname":"aa648dc0","reqId":"req-o0","res":{"statusCode":200},"responseTime":42.42418200010434,"msg":"request completed"}
2021-12-03T02:19:52.012 app[aa648dc0] sea [info] {"level":30,"time":1638497992011,"pid":535,"hostname":"aa648dc0","duration":55.17292800033465,"msg":"s3 fetch"}
2021-12-03T02:19:52.012 app[aa648dc0] sea [info] {"level":30,"time":1638497992012,"pid":535,"hostname":"aa648dc0","reqId":"req-nt","res":{"statusCode":200},"responseTime":56.0772079997696,"msg":"request completed"}
2021-12-03T02:19:52.013 app[aa648dc0] sea [info] {"level":30,"time":1638497992012,"pid":535,"hostname":"aa648dc0","duration":37.6782299997285,"msg":"s3 fetch"}
2021-12-03T02:19:52.014 app[aa648dc0] sea [info] {"level":30,"time":1638497992013,"pid":535,"hostname":"aa648dc0","reqId":"req-o6","res":{"statusCode":200},"responseTime":38.42921799980104,"msg":"request completed"}
2021-12-03T02:19:52.015 app[aa648dc0] sea [info] {"level":30,"time":1638497992015,"pid":535,"hostname":"aa648dc0","duration":50.18699200032279,"msg":"s3 fetch"}
2021-12-03T02:19:52.016 app[aa648dc0] sea [info] {"level":30,"time":1638497992015,"pid":535,"hostname":"aa648dc0","reqId":"req-ny","res":{"statusCode":200},"responseTime":50.88845200045034,"msg":"request completed"}
2021-12-03T02:19:52.016 app[aa648dc0] sea [info] {"level":30,"time":1638497992016,"pid":535,"hostname":"aa648dc0","duration":56.91121100028977,"msg":"s3 fetch"}
2021-12-03T02:19:52.017 app[aa648dc0] sea [info] {"level":30,"time":1638497992016,"pid":535,"hostname":"aa648dc0","reqId":"req-nu","res":{"statusCode":200},"responseTime":58.055178000126034,"msg":"request completed"}
2021-12-03T02:19:52.017 app[aa648dc0] sea [info] {"level":30,"time":1638497992017,"pid":535,"hostname":"aa648dc0","duration":55.574535000137985,"msg":"s3 fetch"}
2021-12-03T02:19:52.018 app[aa648dc0] sea [info] {"level":30,"time":1638497992018,"pid":535,"hostname":"aa648dc0","reqId":"req-nv","res":{"statusCode":200},"responseTime":56.77734400006011,"msg":"request completed"}
2021-12-03T02:19:52.019 app[aa648dc0] sea [info] {"level":30,"time":1638497992018,"pid":535,"hostname":"aa648dc0","duration":48.80187999969348,"msg":"s3 fetch"}
2021-12-03T02:19:52.020 app[aa648dc0] sea [info] {"level":30,"time":1638497992019,"pid":535,"hostname":"aa648dc0","reqId":"req-o2","res":{"statusCode":200},"responseTime":49.560678999871016,"msg":"request completed"}
2021-12-03T02:19:52.020 app[aa648dc0] sea [info] {"level":30,"time":1638497992019,"pid":535,"hostname":"aa648dc0","duration":45.569367999676615,"msg":"s3 fetch"}
2021-12-03T02:19:52.020 app[aa648dc0] sea [info] {"level":30,"time":1638497992020,"pid":535,"hostname":"aa648dc0","reqId":"req-o5","res":{"statusCode":200},"responseTime":47.42502700025216,"msg":"request completed"}
2021-12-03T02:19:52.021 app[aa648dc0] sea [info] {"level":30,"time":1638497992021,"pid":535,"hostname":"aa648dc0","reqId":"req-nw","res":{"statusCode":200},"responseTime":58.51471500005573,"msg":"request completed"}
2021-12-03T02:19:52.026 app[aa648dc0] sea [info] {"level":30,"time":1638497992025,"pid":535,"hostname":"aa648dc0","reqId":"req-nr","res":{"statusCode":200},"responseTime":135.28756800014526,"msg":"request completed"}
2021-12-03T02:19:52.032 app[aa648dc0] sea [info] {"level":30,"time":1638497992031,"pid":535,"hostname":"aa648dc0","duration":60.824439000338316,"msg":"s3 fetch"}
2021-12-03T02:19:52.033 app[aa648dc0] sea [info] {"level":30,"time":1638497992032,"pid":535,"hostname":"aa648dc0","reqId":"req-o3","res":{"statusCode":200},"responseTime":61.83361900039017,"msg":"request completed"}
2021-12-03T02:19:52.033 app[aa648dc0] sea [info] {"level":30,"time":1638497992033,"pid":535,"hostname":"aa648dc0","duration":56.91171899996698,"msg":"s3 fetch"}
2021-12-03T02:19:52.034 app[aa648dc0] sea [info] {"level":30,"time":1638497992033,"pid":535,"hostname":"aa648dc0","reqId":"req-o7","res":{"statusCode":200},"responseTime":57.73594400007278,"msg":"request completed"}
2021-12-03T02:19:52.039 app[aa648dc0] sea [info] {"level":30,"time":1638497992038,"pid":535,"hostname":"aa648dc0","duration":74.5585859999992,"msg":"s3 fetch"}
2021-12-03T02:19:52.039 app[aa648dc0] sea [info] {"level":30,"time":1638497992039,"pid":535,"hostname":"aa648dc0","reqId":"req-nx","res":{"statusCode":200},"responseTime":75.31736400024965,"msg":"request completed"}

(note the average response time of <100ms)

This discrepancy becomes clearer when I trace a specific request from the beginning to end, for instance, for themeToggle_.tsx, devtools shows a response time of 307ms (last line in the screenshot), but here are the corresponding log lines:

2021-12-03T02:19:51.959 app[aa648dc0] sea [info] {"level":30,"time":1638497991958,"pid":535,"hostname":"aa648dc0","reqId":"req-nu","req":{"method":"GET","url":"/style/themeToggle_.tsx.map?@reflame_contentKey=application%2Fjson_zb2rhkX7eJF5E3LmkpqtPqN1Gf8XYgs8faYowP3ZFTFKBfaKr","hostname":"reflame-cdn.fly.dev","remoteAddress":"::ffff:86.109.12.167","remotePort":59204},"msg":"incoming request"}

2021-12-03T02:19:52.017 app[aa648dc0] sea [info] {"level":30,"time":1638497992016,"pid":535,"hostname":"aa648dc0","reqId":"req-nu","res":{"statusCode":200},"responseTime":58.055178000126034,"msg":"request completed"}

So something appears to be adding almost 250ms of latency!

It should be possible to repro from your side by heading to https://reflame-cdn.fly.dev/ and then looking at the logs. Would really appreciate any suggestions you might have to minimize the apparent discrepancy in latency here.

lewis · January 19, 2022, 6:21am

@kurt So I’ve had some time to look into perf again and updated the repo with your type=requests recommendation to test out perf characteristics with various levels of concurrency: GitHub - lewisl9029/fastify-h2c-test

Live demo to play around with: https://fastify-h2c-test.fly.dev/

When you click run, it’ll fire of a number of simultaneous requests to /noop based on the number in the input box (defaults to 100).

Here’s what I’m seeing in devtools with the default 100 requests:

First handful of requests complete quickly enough, but things get progressively worse for later requests (last few are all in the 160ms ballpark).

Here’s what the last few fastify logs look like:

2022-01-19T05:58:55.228 app[203d7f0a] sjc [info]{"level":30,"time":1642571935228,"pid":536,"hostname":"203d7f0a","reqId":"req-84","res":{"statusCode":200},"responseTime":0.12462399998912588,"msg":"request completed"}
2022-01-19T05:58:55.228 app[203d7f0a] sjc [info]{"level":30,"time":1642571935228,"pid":536,"hostname":"203d7f0a","reqId":"req-85","req":{"method":"GET","url":"/noop?nonce=0.3834155671076118","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45046},"msg":"incoming request"}
2022-01-19T05:58:55.229 app[203d7f0a] sjc [info]{"level":30,"time":1642571935228,"pid":536,"hostname":"203d7f0a","reqId":"req-85","res":{"statusCode":200},"responseTime":0.12235900000086986,"msg":"request completed"}
2022-01-19T05:58:55.229 app[203d7f0a] sjc [info]{"level":30,"time":1642571935228,"pid":536,"hostname":"203d7f0a","reqId":"req-86","req":{"method":"GET","url":"/noop?nonce=0.9519116195985358","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45048},"msg":"incoming request"}
2022-01-19T05:58:55.229 app[203d7f0a] sjc [info]{"level":30,"time":1642571935228,"pid":536,"hostname":"203d7f0a","reqId":"req-86","res":{"statusCode":200},"responseTime":0.1224789999978384,"msg":"request completed"}
2022-01-19T05:58:55.229 app[203d7f0a] sjc [info]{"level":30,"time":1642571935229,"pid":536,"hostname":"203d7f0a","reqId":"req-87","req":{"method":"GET","url":"/noop?nonce=0.9931775853763385","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45082},"msg":"incoming request"}
2022-01-19T05:58:55.229 app[203d7f0a] sjc [info]{"level":30,"time":1642571935229,"pid":536,"hostname":"203d7f0a","reqId":"req-87","res":{"statusCode":200},"responseTime":0.1472759999887785,"msg":"request completed"}
2022-01-19T05:58:55.229 app[203d7f0a] sjc [info]{"level":30,"time":1642571935229,"pid":536,"hostname":"203d7f0a","reqId":"req-88","req":{"method":"GET","url":"/noop?nonce=0.49444808002841656","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45092},"msg":"incoming request"}
2022-01-19T05:58:55.230 app[203d7f0a] sjc [info]{"level":30,"time":1642571935229,"pid":536,"hostname":"203d7f0a","reqId":"req-88","res":{"statusCode":200},"responseTime":0.1578460000018822,"msg":"request completed"}
2022-01-19T05:58:55.230 app[203d7f0a] sjc [info]{"level":30,"time":1642571935229,"pid":536,"hostname":"203d7f0a","reqId":"req-89","req":{"method":"GET","url":"/noop?nonce=0.6833850924027194","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45094},"msg":"incoming request"}
2022-01-19T05:58:55.230 app[203d7f0a] sjc [info]{"level":30,"time":1642571935230,"pid":536,"hostname":"203d7f0a","reqId":"req-89","res":{"statusCode":200},"responseTime":0.6187099999951897,"msg":"request completed"}
2022-01-19T05:58:55.231 app[203d7f0a] sjc [info]{"level":30,"time":1642571935230,"pid":536,"hostname":"203d7f0a","reqId":"req-8a","req":{"method":"GET","url":"/noop?nonce=0.31162930185295434","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45098},"msg":"incoming request"}
2022-01-19T05:58:55.231 app[203d7f0a] sjc [info]{"level":30,"time":1642571935230,"pid":536,"hostname":"203d7f0a","reqId":"req-8a","res":{"statusCode":200},"responseTime":0.14397000000462867,"msg":"request completed"}
2022-01-19T05:58:55.231 app[203d7f0a] sjc [info]{"level":30,"time":1642571935231,"pid":536,"hostname":"203d7f0a","reqId":"req-8b","req":{"method":"GET","url":"/noop?nonce=0.8833881052338448","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45100},"msg":"incoming request"}
2022-01-19T05:58:55.231 app[203d7f0a] sjc [info]{"level":30,"time":1642571935231,"pid":536,"hostname":"203d7f0a","reqId":"req-8b","res":{"statusCode":200},"responseTime":0.14501199999358505,"msg":"request completed"}
2022-01-19T05:58:55.231 app[203d7f0a] sjc [info]{"level":30,"time":1642571935231,"pid":536,"hostname":"203d7f0a","reqId":"req-8c","req":{"method":"GET","url":"/noop?nonce=0.7686686872794564","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45102},"msg":"incoming request"}
2022-01-19T05:58:55.232 app[203d7f0a] sjc [info]{"level":30,"time":1642571935231,"pid":536,"hostname":"203d7f0a","reqId":"req-8c","res":{"statusCode":200},"responseTime":0.14159600000130013,"msg":"request completed"}
2022-01-19T05:58:55.232 app[203d7f0a] sjc [info]{"level":30,"time":1642571935231,"pid":536,"hostname":"203d7f0a","reqId":"req-8d","req":{"method":"GET","url":"/noop?nonce=0.14933176971709217","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45104},"msg":"incoming request"}
2022-01-19T05:58:55.232 app[203d7f0a] sjc [info]{"level":30,"time":1642571935232,"pid":536,"hostname":"203d7f0a","reqId":"req-8d","res":{"statusCode":200},"responseTime":0.1459939999913331,"msg":"request completed"}
2022-01-19T05:58:55.232 app[203d7f0a] sjc [info]{"level":30,"time":1642571935232,"pid":536,"hostname":"203d7f0a","reqId":"req-8e","req":{"method":"GET","url":"/noop?nonce=0.22092260004765896","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45106},"msg":"incoming request"}
2022-01-19T05:58:55.232 app[203d7f0a] sjc [info]{"level":30,"time":1642571935232,"pid":536,"hostname":"203d7f0a","reqId":"req-8e","res":{"statusCode":200},"responseTime":0.13948199999867938,"msg":"request completed"}
2022-01-19T05:58:55.232 app[203d7f0a] sjc [info]{"level":30,"time":1642571935232,"pid":536,"hostname":"203d7f0a","reqId":"req-8f","req":{"method":"GET","url":"/noop?nonce=0.41222002151315107","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45108},"msg":"incoming request"}
2022-01-19T05:58:55.233 app[203d7f0a] sjc [info]{"level":30,"time":1642571935232,"pid":536,"hostname":"203d7f0a","reqId":"req-8f","res":{"statusCode":200},"responseTime":0.14433999999891967,"msg":"request completed"}
2022-01-19T05:58:55.233 app[203d7f0a] sjc [info]{"level":30,"time":1642571935232,"pid":536,"hostname":"203d7f0a","reqId":"req-8g","req":{"method":"GET","url":"/noop?nonce=0.03771367953304372","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45110},"msg":"incoming request"}
2022-01-19T05:58:55.233 app[203d7f0a] sjc [info]{"level":30,"time":1642571935233,"pid":536,"hostname":"203d7f0a","reqId":"req-8g","res":{"statusCode":200},"responseTime":0.13906000000133645,"msg":"request completed"}
2022-01-19T05:58:55.233 app[203d7f0a] sjc [info]{"level":30,"time":1642571935233,"pid":536,"hostname":"203d7f0a","reqId":"req-8h","req":{"method":"GET","url":"/noop?nonce=0.37951247220321815","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45112},"msg":"incoming request"}
2022-01-19T05:58:55.233 app[203d7f0a] sjc [info]{"level":30,"time":1642571935233,"pid":536,"hostname":"203d7f0a","reqId":"req-8h","res":{"statusCode":200},"responseTime":0.14473200000065845,"msg":"request completed"}
2022-01-19T05:58:55.234 app[203d7f0a] sjc [info]{"level":30,"time":1642571935233,"pid":536,"hostname":"203d7f0a","reqId":"req-8i","req":{"method":"GET","url":"/noop?nonce=0.1082447794859942","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45114},"msg":"incoming request"}
2022-01-19T05:58:55.234 app[203d7f0a] sjc [info]{"level":30,"time":1642571935233,"pid":536,"hostname":"203d7f0a","reqId":"req-8i","res":{"statusCode":200},"responseTime":0.14171599999826867,"msg":"request completed"}
2022-01-19T05:58:55.234 app[203d7f0a] sjc [info]{"level":30,"time":1642571935234,"pid":536,"hostname":"203d7f0a","reqId":"req-8j","req":{"method":"GET","url":"/noop?nonce=0.15171719714828713","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45116},"msg":"incoming request"}
2022-01-19T05:58:55.234 app[203d7f0a] sjc [info]{"level":30,"time":1642571935234,"pid":536,"hostname":"203d7f0a","reqId":"req-8j","res":{"statusCode":200},"responseTime":0.14508200000273064,"msg":"request completed"}
2022-01-19T05:58:55.234 app[203d7f0a] sjc [info]{"level":30,"time":1642571935234,"pid":536,"hostname":"203d7f0a","reqId":"req-8k","req":{"method":"GET","url":"/noop?nonce=0.8671808951758786","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45118},"msg":"incoming request"}
2022-01-19T05:58:55.235 app[203d7f0a] sjc [info]{"level":30,"time":1642571935234,"pid":536,"hostname":"203d7f0a","reqId":"req-8k","res":{"statusCode":200},"responseTime":0.14196599999559112,"msg":"request completed"}
2022-01-19T05:58:55.235 app[203d7f0a] sjc [info]{"level":30,"time":1642571935234,"pid":536,"hostname":"203d7f0a","reqId":"req-8l","req":{"method":"GET","url":"/noop?nonce=0.9052876966643939","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45120},"msg":"incoming request"}
2022-01-19T05:58:55.235 app[203d7f0a] sjc [info]{"level":30,"time":1642571935234,"pid":536,"hostname":"203d7f0a","reqId":"req-8l","res":{"statusCode":200},"responseTime":0.14612399999168701,"msg":"request completed"}
2022-01-19T05:58:55.235 app[203d7f0a] sjc [info]{"level":30,"time":1642571935235,"pid":536,"hostname":"203d7f0a","reqId":"req-8m","req":{"method":"GET","url":"/noop?nonce=0.519005828261158","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45122},"msg":"incoming request"}
2022-01-19T05:58:55.235 app[203d7f0a] sjc [info]{"level":30,"time":1642571935235,"pid":536,"hostname":"203d7f0a","reqId":"req-8m","res":{"statusCode":200},"responseTime":0.1408439999941038,"msg":"request completed"}
2022-01-19T05:58:55.235 app[203d7f0a] sjc [info]{"level":30,"time":1642571935235,"pid":536,"hostname":"203d7f0a","reqId":"req-8n","req":{"method":"GET","url":"/noop?nonce=0.9476355703528119","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45124},"msg":"incoming request"}
2022-01-19T05:58:55.236 app[203d7f0a] sjc [info]{"level":30,"time":1642571935235,"pid":536,"hostname":"203d7f0a","reqId":"req-8n","res":{"statusCode":200},"responseTime":0.1441110000014305,"msg":"request completed"}
2022-01-19T05:58:55.236 app[203d7f0a] sjc [info]{"level":30,"time":1642571935235,"pid":536,"hostname":"203d7f0a","reqId":"req-8o","req":{"method":"GET","url":"/noop?nonce=0.5970482323336712","hostname":"fastify-h2c-test.fly.dev","remoteAddress":"::ffff:139.178.89.141","remotePort":45126},"msg":"incoming request"}
2022-01-19T05:58:55.236 app[203d7f0a] sjc [info]{"level":30,"time":1642571935235,"pid":536,"hostname":"203d7f0a","reqId":"req-8o","res":{"statusCode":200},"responseTime":0.14083400000527035,"msg":"request completed"}

All of them complete in less than a millisecond as you’d expect, since the /noop handler only sets a header and returns a hard coded string.

This behavior matches what I’m seeing with my real app at https://app.reflame.dev/ where fly’s proxy seems to add anywhere between 20-100+ ms of latency to the response time logged by fastify when highly concurrent requests are involved.

Any ideas what could be causing this latency and how it can be improved? Could node/fastify be the bottleneck somehow? (maybe it’s lying about the response time in the logs?)

zhengyutay · April 3, 2022, 3:00pm

I use Fastify to server requests as well and I found that there is a huge disparity between client response time and fly response time. I also wondering what’s the bottleneck

kurt · April 3, 2022, 8:18pm

How are you measuring client and server response times?

We shouldn’t be adding much overhead to responses. We might report times differently because we report how long the whole response takes to get to the client (vs just server generation time).

If you’re sending a whole bunch of requests at once, you might be hitting the hard_limit in the config file. That defaults to 25 concurrent requests, and will queue additional requests. You can try raising that limit to see if it helps.