From CPU Quotas Update fly is using 1/16th the CPU for shared CPU. Fly does appear to have a burst system. That means if you haven’t been encoding video for a little bit, fly will think you’re just bursting to a higher usage temporarily, and allow it before eventually pulling the performance back down.
You’ll have more stable performance by switching to the performance plans. Switching from shared-8x to performance-1x looks to be about a 2x cost, but you should have 2x the performance.
Additionally, while I understand video processing can be multi threaded, I am not sure all encoders/decoders use multiple cores well. With that in mind, a faster single core server (performance-1x) will get better performance for those circumstances than shared-8x will.