On monitor refresh rate variability, how it interacts with deterministic results in a game, application, or simulation, and designing to fit this reality.
Few months ago I have decided to make a demo of a game with stutter-free experience because of numerous articles about stuttering I saw online, in the end I decided that delta-times are a no-go, noticing how for monitor refresh cycle and simulation cycle are completely different in timing and there is no way to solve them cleanly with a "traditional" game loop. And since I don't respect almost any programming traditions, I decided to do a new approach which I haven't seen done anywhere else, and it is to decouple those cycles into different threads. The approach worked great, giving me smooth interpolation on any monitor. The higher refresh rate - the smoother the game looks; no tricks. Thought you have made some inventions as well, but turns out you did it exactly as I did! This was a waste of $5, wasn't it :D
But at least a good programmer like you has now confirmed that this *is* a working approach, and it finally solves the ancient game programming problem of coupling simulation with frame-rate. I do wish you expounded more on the problems you faced, and solutions you have found. I found it tricky to set up communication between code cycles when one is going too fast or too slow, it can introduce deadlocks if you don't structure the communication system properly. There are also performance concerns depending on how you set up this data-sharing. etc.
I think provides a slightly better intuition for the underlying maths (though it is largely the same as the one Ryan linked to).
The x axis is the number of frames, the y axis is real world time. The transition is from 0 to 1, t is the time that the transition should take, a is the rate, f is the refresh rate, and m is how close to 1 that the `current` value needs to be by frame f*t.
I think it also helps explains where the `1 - Pow(2, (-50.f * dt))` comes from: it is approximating (m/t)^(1/t) with 2^(-50). Assuming that m is intended to be set to something very small, we can expect (m/t)^(1/t) to also be small. We also expect (m/t)^(1/t) to get smaller as m gets smaller. So using a tiny constant does the job, and is more practical.
I cannot speak for Ryan, but I found the same formula using generating functions (it's just a linear recurrence). You then find the formula (when current starts at 0 and target is 1): current = 1 - (1-r)^n with r the rate and n the number of frames that have run. Then you can just write n = t/dt and solve for r with t being the time at the halfway point (in this case that is 1/50 seconds) and current = 1/2.
I'm not clear on how splitting the main loop into multiple threads would solve the stuttering problem, I think you should still be getting some and it will be more noticeable at lower simulation rates, it should be obvious simulating at 30 and rendering at something like 75 or 50. I don't have a better solution that doesn't need interpolation or variable simulation rates either, are you sure it's drawing smoothly without any of that?. I'm also afraid this might introduce occasional stuttering even when you render and simulate at the same rate.
This is one of those things I've never seen solved in a reliable way, when I play a game with a fixed simulation rate like Sekiro or through an emulator I just set the monitor refresh rate to what it expects(usually 60) to get a stutter free experience.
> I'm not clear on how splitting the main loop into multiple threads would solve the stuttering problem
This stuttering problem in particular is caused by fixing simulation and rendering rate to a rate which does not cleanly divide the refresh rate. I explain this in the article, and it matches your examples of simulation at 30 Hz and rendering at 50 or 75 Hz. This solves it by doing both things you want — simulation at a fixed rate, rendering interpolated state at the monitor refresh rate. The reason the problem is eliminated is that rendering always uses fresh data—it samples from two valid states and blends between them using a timestamp. Both top level loops run at their desired rate. When a new simulation step is complete, it submits new state, which the render thread will not render immediately—it will blend between that and the next most recent state it has.
> are you sure it's drawing smoothly without any of that?
Maybe there has been some misunderstanding? Interpolation still occurs, but the separation into two top level threads is to avoid other issues. I go over all this in the article.
> This is one of those things I've never seen solved in a reliable way
I don’t think this is true, but to get to the bottom of this, where do you think the stuttering problem comes from, and why do you think my solution doesn’t solve it? You made assertions like “you should still get stuttering”... why do you suppose this is the case?
Sorry, I didn't see any mention of interpolation in the second part of the article and you mentioned "Writing both simulation and drawing code is nearly as simple as within a non-interpolated single-threaded context", so I was a bit confused.
I presumed there will be stuttering without interpolation because I imagine that with an arbitrary simulation rate, the instant the rendering thread started a new frame, the game thread could've possibly been just about to finish a simulation and you'd draw the same state twice, so it seemed possible to me that there could be a significant time gap between that and the next frame resulting in stuttering.
You also didn't mention if you locked the simulation thread during the rendering, so I assuming you didn't and just locked to copy the state, I imagine the threads would be more likely to be running out of phase if that's the case, which to my understanding makes stuttering happen more if you don't interpolate anything.(EDIT: I'm not feeling confident about what I said here, but I think it would also depend on how the tick timer is implemented)
I think there just weren't enough details for me to accept for a fact there wouldn't be any stuttering (and I wrongly assumed you weren't interpolating) and I haven't tried this approach yet, but I want to give it a try. I also associate different threads with different timings with stuttering due to my experiences with other software in the past, namely Godot Engine.
Few months ago I have decided to make a demo of a game with stutter-free experience because of numerous articles about stuttering I saw online, in the end I decided that delta-times are a no-go, noticing how for monitor refresh cycle and simulation cycle are completely different in timing and there is no way to solve them cleanly with a "traditional" game loop. And since I don't respect almost any programming traditions, I decided to do a new approach which I haven't seen done anywhere else, and it is to decouple those cycles into different threads. The approach worked great, giving me smooth interpolation on any monitor. The higher refresh rate - the smoother the game looks; no tricks. Thought you have made some inventions as well, but turns out you did it exactly as I did! This was a waste of $5, wasn't it :D
But at least a good programmer like you has now confirmed that this *is* a working approach, and it finally solves the ancient game programming problem of coupling simulation with frame-rate. I do wish you expounded more on the problems you faced, and solutions you have found. I found it tricky to set up communication between code cycles when one is going too fast or too slow, it can introduce deadlocks if you don't structure the communication system properly. There are also performance concerns depending on how you set up this data-sharing. etc.
I assume audio would go on a separate thread as well? What about netcode?
Great post!
In regard to the `F32 rate = 1 - Pow(2, (-50.f * dt));` part:
For anyone else feeling similarly obsessive, I suggest taking a look at this modified version of the desmos graph: https://www.desmos.com/calculator/ai4wlumpo1
I think provides a slightly better intuition for the underlying maths (though it is largely the same as the one Ryan linked to).
The x axis is the number of frames, the y axis is real world time. The transition is from 0 to 1, t is the time that the transition should take, a is the rate, f is the refresh rate, and m is how close to 1 that the `current` value needs to be by frame f*t.
I think it also helps explains where the `1 - Pow(2, (-50.f * dt))` comes from: it is approximating (m/t)^(1/t) with 2^(-50). Assuming that m is intended to be set to something very small, we can expect (m/t)^(1/t) to also be small. We also expect (m/t)^(1/t) to get smaller as m gets smaller. So using a tiny constant does the job, and is more practical.
I cannot speak for Ryan, but I found the same formula using generating functions (it's just a linear recurrence). You then find the formula (when current starts at 0 and target is 1): current = 1 - (1-r)^n with r the rate and n the number of frames that have run. Then you can just write n = t/dt and solve for r with t being the time at the halfway point (in this case that is 1/50 seconds) and current = 1/2.
I'm not clear on how splitting the main loop into multiple threads would solve the stuttering problem, I think you should still be getting some and it will be more noticeable at lower simulation rates, it should be obvious simulating at 30 and rendering at something like 75 or 50. I don't have a better solution that doesn't need interpolation or variable simulation rates either, are you sure it's drawing smoothly without any of that?. I'm also afraid this might introduce occasional stuttering even when you render and simulate at the same rate.
This is one of those things I've never seen solved in a reliable way, when I play a game with a fixed simulation rate like Sekiro or through an emulator I just set the monitor refresh rate to what it expects(usually 60) to get a stutter free experience.
> I'm not clear on how splitting the main loop into multiple threads would solve the stuttering problem
This stuttering problem in particular is caused by fixing simulation and rendering rate to a rate which does not cleanly divide the refresh rate. I explain this in the article, and it matches your examples of simulation at 30 Hz and rendering at 50 or 75 Hz. This solves it by doing both things you want — simulation at a fixed rate, rendering interpolated state at the monitor refresh rate. The reason the problem is eliminated is that rendering always uses fresh data—it samples from two valid states and blends between them using a timestamp. Both top level loops run at their desired rate. When a new simulation step is complete, it submits new state, which the render thread will not render immediately—it will blend between that and the next most recent state it has.
> are you sure it's drawing smoothly without any of that?
Maybe there has been some misunderstanding? Interpolation still occurs, but the separation into two top level threads is to avoid other issues. I go over all this in the article.
> This is one of those things I've never seen solved in a reliable way
I don’t think this is true, but to get to the bottom of this, where do you think the stuttering problem comes from, and why do you think my solution doesn’t solve it? You made assertions like “you should still get stuttering”... why do you suppose this is the case?
Sorry, I didn't see any mention of interpolation in the second part of the article and you mentioned "Writing both simulation and drawing code is nearly as simple as within a non-interpolated single-threaded context", so I was a bit confused.
I presumed there will be stuttering without interpolation because I imagine that with an arbitrary simulation rate, the instant the rendering thread started a new frame, the game thread could've possibly been just about to finish a simulation and you'd draw the same state twice, so it seemed possible to me that there could be a significant time gap between that and the next frame resulting in stuttering.
You also didn't mention if you locked the simulation thread during the rendering, so I assuming you didn't and just locked to copy the state, I imagine the threads would be more likely to be running out of phase if that's the case, which to my understanding makes stuttering happen more if you don't interpolate anything.(EDIT: I'm not feeling confident about what I said here, but I think it would also depend on how the tick timer is implemented)
I think there just weren't enough details for me to accept for a fact there wouldn't be any stuttering (and I wrongly assumed you weren't interpolating) and I haven't tried this approach yet, but I want to give it a try. I also associate different threads with different timings with stuttering due to my experiences with other software in the past, namely Godot Engine.