Auto-suspend results in JWT error: "nbf" claim timestamp check failed

,
1

Node.js app using @shopify/shopify-app-remix 2.8.2.


app = 'zzz'
primary_region = 'iad'

[build]

[env]
  PORT = '3000'

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = "stop"
  # auto_stop_machines = "suspend"
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[[vm]]
  memory = '512mb'
  cpu_kind = 'shared'
  cpus = 1

Changed config from stop to auto_stop_machines = “suspend”. Was excited. App start time is fast now but noticed when app is restored from suspended state logging in gives “nbf” claim timestamp check failed error. This typically means there is timestamp skew between client and server. Switching back to stop eliminates this.

How to use suspend and avoid this error?

2

Hi… The auto-suspend feature is exciting, :bluegreen_artist:, but it’s also considered a little experimental (last I heard). The problems it causes with clock skew is one of the caveats that was mentioned when it was first released:

When resumed, your Machine may take a few seconds to update its clock, so for the first few seconds it will think that it’s in the past.

Work-arounds were discussed in the forum a couple months ago—but personally I would stick with stop in security-sensitive contexts for now.

Hope this helps!

4

Thanks for letting me know! Didn’t see this in the docs. Will checkout the workarounds.

1 Like
5

Hi @jackoffer, we implemented a fix for the issue with temporary timestamp skew upon resuming a suspended Machine, the system clock is now synchronized immediately on resume. You may need to re-deploy your Machine in order for the updated code to take effect. Let me know if this resolves your issue!

1 Like
6

Hi, thanks for the update. Happy to hear. Will give it a try and if problems come back and post.

7

Hm… My own tests are still showing unbounded clock skew, even with freshly cloned Machines. Maybe this hasn’t yet rolled out to all the fleet’s hosts?

$ while true; do curl 'http://icicle.flycast/' && sleep 1; done
-   5040s = 1751587221s - 1751592261s
-   5040s = 1751587222s - 1751592262s
-   5040s = 1751587223s - 1751592263s
-   5040s = 1751587224s - 1751592265s
-   5040s = 1751587225s - 1751592266s
-   5040s = 1751587226s - 1751592267s
-   5040s = 1751587227s - 1751592268s
-   5040s = 1751587228s - 1751592269s
+      0s = 1751592270s - 1751592270s
-      0s = 1751592271s - 1751592271s
-      0s = 1751592272s - 1751592272s
-      0s = 1751592273s - 1751592273s
-      0s = 1751592274s - 1751592274s
+      0s = 1751592275s - 1751592275s
-      0s = 1751592276s - 1751592276s
-      0s = 1751592277s - 1751592277s

This is the difference between the system time within the Machine (the timestamp to the left of the minus sign) and the time reported by the PTP device (which comes from the host’s own clock, ultimately).

(Times are rounded to the nearest second, for display porpoises—which is why you can see both +0 and -0.)

As a sanity check, PTP coincides nicely with the time on the client:

$ curl 'http://icicle.flycast/' && date +%s
-     80s = 1751592378s - 1751592458s
1751592457

icicle is on a shared-1x Machine with 512MB RAM, running Debian Bookworm.

1 Like
8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.