FAQ! *Q: What about lerp(a,b,F*Δt)?* A: That's an approximation, and it happens to be a pretty good approximation! Especially if you are running at a high framerate, but it can falter pretty violently if you get lag spikes, or if you use large values of F. Even if you clamp it to not overshoot a value of 1, you can get disparate outcomes depending on framerate. Now, this whole talk is about half of what my (upcoming) blog post will cover! In the blog post, I spend a *lot* of time talking about approximations, including this one, and exactly where and how it fails c: *Q: Can't you just use a fixed timestep loop, separate from the rendering update loop?* A: You totally can! But if your fixed timestep is lower than your render framerate (which it usually is - unity defaults to 50fps in FixedUpdate for example), then you now have a new problem. Your motion will look stuttery because the position isn't updated as often as your rendering is, and so now you need to make it interpolate across those gaps before the next FixedUpdate hits. Now, in unity specifically, and specifically for rigidbodies only, there is support for interpolation, if enabled. However, keep in mind that this is an additional hack/patch, on top of your initial hack, of not using a framerate independent smoothing method. Personally I prefer just to use the framerate independent variant at the render framerate, unless it has to be a part of some existing timestep ecosystem, such as when you want it for rigidbodies *Q: How did you make these slides/animations?* A: It's a long story! Usually I use unity together with my animation tool + shapes, my vector graphics library. But this time around I used html/js/css, since this started as a (yet to be released) blog post on the same topic. I use html5 canvas to draw 2D graphics, and I use KaTeX to convert LaTeX into html elements. Pretty much everything else is custom made, from the slide nav to the interaction to the animations etc.! I have actually considered making a video about my descent into web dev in making this, because I have OPINIONS about javascript, lordie christ ;-; *Q: Will you publish the slides?* A: I already have! They're available for all my patreon supporters right here! www.patreon.com/posts/105228270 *Q: Will you publish something about the spring physics you showed near the end?* A: Eventually yes! likely on my blog as well, maybe as part of the lerp smoothing blog post, or as its own separate thing afterwards since my lerp smoothing post is already enormous
Q: What about the y′′+2ζωy′+ω2y=0 ? It looks interesting, but even chatgpt dont know that and only 2 pages on google, not really mentioning a code! Show me a pseudocode for that pretty please :). Or confirm K1 K2 K3 K4 solution by chatgpt.
PLEASE do make that video about web dev. I am trying my hand at explanation-blogging and it's not easy, I want to see how you do it with your kind of quality.
> because I have OPINIONS about javascript, lordie christ ;-; I am putting this post on my Christmas wish list. I recently fell back into wanting to do some web dev, and... yes. Lots of opinions, lots of feelings :')
That's super cool, I'd love to see your thoughts on web dev, especially how you're doing the canvas stuff. Right now I'm working on my first browser game (it's a picross/nonogram clone) and I'm only using the canvas, so it would be neat to hear about how you achieved certain things like smooth animations and making it run efficiently.
literally though?? I don't think it's visible in the video, but there were *so* many people, including a crowd gathered, *standing* behind all the (filled) seats. after my talk I spent about 3 hours nonstop talking to people who had questions! it's weird, but also really fun and flattering ❤
I'm going into the video thinking "oh yeah, I know that" and in the middle of the talk you go deeper into a topic than I ever did and learn something new. It's amazing! That's why I always watch your videos!
haha, good! I generally try and capture as many people as possible in the beginning, so that the more advanced stuff is built on a foundation I've communicated, rather than assuming everyone knows that stuff already. sometimes it's useful to get reminders of things you already know too, so I think it's worth it! even if some people might find it basic or boring for a bit
@@acegikmo so I was reading the newly released Tom7 paper (called Badness 0, it's on HN frontpage) and he was talking about how some people explore subjects "depth first" and he gave an example of Donald Knuth. I was just thinking how your videos have so much depth too
Cool connection at 16:30: It takes 53 iterations of halving. The mantissa of a 64bit float only has 53 (effective) bits of precision. This is not a coincidence! Specifically, a float is stored as 1.mantissa x 2^exponent, where the mantissa is a binary number. The number closest (but not equal) to 2 is hence 1.111...1b x 2^0 with 53 1s, equal to 1.99999999999999977795539507497 in decimal. Up to rounding, this is f(52) shown in the video! On the next iteration, there is no nearer representable number -- it simply rounds to 2! And indeed, it takes 52 iterations to get to this point, because at the nth iteration, a is 2^-n away from 2. The difference between 2 and 1.111...1b? 0.000...1b = 2^-52!
@@F_A_F123 people don't respect the need for precision nearly as much as they should. why shouldn't we just round pi to 3.14? what could possibly go wrong?
If you just don't mention it, they'll put the bug back in the code and get free job security. So that depends on what you think about keeping that coworker around!
@@catmacopter8545if the fix was an accident and they aren’t sure why it is fixed, it can be vexing to not know the origin such that you could prevent it from recurring.
Hi Freya, you probably don't remember me in particular, but I hope you remember my class! Me (and hopefully everyone) from Futuregames 2022 Game Programmer just wants to say "thank you!" for all you did for us! I was saddened that you got sick for our other course with you and I hope you feel much better these days! Every time I think of Lerp, I think of you! With loving regards, an alumni from FutureGames!
hmm, I think for me the cringiest part is that I didn't get a haircut before this event, my style works much better when my wavy inverted bob cut is properly showing, but now that it's grown out it just kinda looks awkward-medium length wavy
@@acegikmoyo I know nothing about gamedev or smth you doing, but I think you are not awkward and kinda cute really. goals tbh, maybe in 1-2 years all be a programmer/gamedeveloper wearing cat ears too ❤ you make it sound less confusing so thanks
6:22 oooh dimensional analysis is great! I love it, glad you learned about it. It doesn't solve everything, but every once in a while it lets you skip some hard work
It's also a check that your calculations make sense to begin with, that the units at the end have the dimensions you actually want, so you don't waste a bunch of time, say, finding a neat, closed form solution to a nasty, nested integral that is actually fundamentally wrong 🙃
Great discussion. How to convert from a per frame implementation to a time based implementation should be game dev 101, but we've seen plenty of AAA game companies fail at it.
Around 1999 there was a car racing game called Screamer, which was great, but the tire grip depended on framerate. The slower your PC the faster you could take corners. I replayed it years later at 200fps and there was virtually no grip, like driving on ice
Rollcage came out the same year and was one of my favorite PC racing games, but it had the same problem. I loved that game but when I convinced a friend to buy it and tried to play him vs, he said that the steering was really bad. I thought he just needed to get good until I tried it myself. His system was faster and the steering was clearly tied to the frame rate. I thought that was an amateur mistake by the developer, but this talk has convinced me that it's a more subtle problem than I thought, and I've probably made similar mistakes in my own game development.
@@adsilcott Oh, you don't have to go so far back, in F1 2022 and 2023 (don't know about 2024), physics are framerate dependent. I mean, you don't have 0 grip at high framerates, but with a low enought performance you could almost ignore curbs. Also, random fact, I was searching for Screamer, and it seems that the studio is now the one that works in Moto GP games, Ride franchice, the new Hotwheel game and some more.
Need for Speed: Rivals (by EA) has the whole physics system tied to the framerate. If you are lagging the whole game will play slower, and if you unlock it to high fps the game plays like a timelapse
@@curryleavesbydhanya wow, that's bad for a "modern" title. You could also have slowmotion in Grand Prix 2 back in 1996, when the PC was too slow, but apart from that it was accurate
The lerp saga continues Also dang learning quarternions before differential equations feels like the wrong order of things 😂 I for sure assumed you'd have that down before. But probably just my uni bias since i still dont get quarternions but am quite comfortable with differential equations
I think it's a side effect of not having an academic background in math, the way/order I study things is entirely by happenstance, so I tend to have a very lopsided skill set
@@APaleDot Actually quaternions aren’t just algebra because they for example have a smooth structure. Like complex numbers (well and even real numbers)-complex analysis is even very famous, but quaternionic analysis is a bit bad so it’s usually not that well-known like real and complex cases. And I think one can study differential equations for a long long time, there’s just so much stuff, including hairy analytic stuff and weak solutions and all the worst that math can throw at you. Especially if we include PDEs (really they are a separate subject just for technical reasons and due so a couple of differences in application). So if you even studied ODEs and maybe PDEs at uni and even if you had a math specialty, you can still be far from having a grasp on them that is sufficient for many many areas and tasks. Likewise for quaternions and all other math though: for example I don’t think people usually are aware that you can forget about concrete i, j, k (like we can swap ±i in ℂ, we can rotate the imaginary unit sphere any way we wish and this won’t change any algebra, so if our results don’t directly manipulate with i, j, k, they’re probably more useful etc., compared to if they do depend on occurrences of i, j, k-well, except for quaternion implementation and numerical algorithms, I suppose).
If df(x) = af(x), chances are good you have an exponential; _the_ function whose rate of change is proportional to its value. Fun fact, a 90° turn is legally a constant proportion, so circular motion, whose rate of change is a vector 90° from its current position, is also an exponential. Often this "90° turn" is written as "i", so circular motion is expressed as e^it, which may or may not equal cos(t) + isin(t). Now we just need to do _spherical_ LERP smoothing along a B-spline of dual-quaternions, and we'll still probably not have the most complicated way to get from point-A to point-B.
This is the first explanation I've heard of complex exponentials as sinusoids that has actually felt intuitive to me! Crazy to have stumbled across such a gem in a UA-cam comment section :) I hope you have a wonderful day!
@@cineblazer Yep, the hallmark of circular motion is that the velocity is always perpendicular to the displacement from the center of the circle. With complex numbers, you just multiply by 'i' to rotate by 90°. So you could say that the velocity is just the displacement multiplied by 'i' (or some constant multiple of 'i'). Since exponentials are always proportional to their derivatives, as shown by d(e^(ax)) = ae^(ax) dx, we can see that if the displacement is s(t) = e^(it) then ds(t) = v(t) = ie^(it) dt. So we have v(t) = is(t), fulfilling the requirement for circular motion.
@@cineblazer Any function that only depends on a prior position and a time since said position can be described as an exponential. This includes exponential growth & decay, rotations, and even in a way _translations._ This even relates to the limit definition of the exponential exp(x) = lim n->∞ (1 + x/n)^n, where you're essentially breaking x up into infinitely many small chunks, and then repeatedly scaling the identity by the 1 + a single chunk. Whether that's by repeatedly eliminating 30% of a set of remaining dots, or simply taking one step after another to slowly inch your way across a mile, they all boil down to the same concepts. There are no coincidences in math; only special cases.
Another secret is if you remember from Freya’s “why can’t you multiply vectors” talk, i is a unit bivector - an oriented plane segment representing rotation. The fact that e^i represents rotation is then no coincidence! In fact, with the 3D bivectors i j and k, if you have some bivector B = a*i + b*j + c*k, the *exponential* of that bivector e^(B*theta) works **exactly** like e^(i*theta) - it gives you a rotor* representing the rotation in the plane of the original bivector by the angle theta. Just like the exponential discussed, the bivector itself is another constant value - in fact, as alluded to in her talk, it is **the circulation bivector!!** This means that the formula for following a curve can be described as iteratively applying e^C(t)dt (or the corresponding integral). [I think the formula’s right I may be off by a few factors] *rotors are only mentioned briefly in the talk, but they are just scalars + bivectors. You need scalars as well to represent arbitrary (non-90*) rotations for the same reason you need scalars in complex numbers.
Hmm, not a game dev but I find it fascinating that the same smoothing idea can show up in many places. In game dev, it’s lerp smoothing. In finance, it’s the exponential moving average (EMA). In my field, electrical engineering, I also just call it the EMA, but others call it the exponential filter or even the “1-pole IIR filter”. And, well, I never considered going about this issue without bringing up Fourier and cutoff frequencies, but here’s an explanation using just algebra (not even calculus!). Like, I never made that connection between the framerate dependency and the decay. Very interesting… EDIT: I remember now you mentioned that you do that clamp to prevent wiggling. When that’s done, it does becomes slightly different from the EMA. What you’re seeing is probably the Gibbs phenomenon. It takes particular filters like the Bessel filter to get rid of it without clamping, but I’ve also seen people just do clamping. For example, some variations of JPEG rendering do this so that you don’t get JPEG artifacts.
FWIW, I used this for smoothing out joint animation data coming out of a Kinect sensor. Sometimes joints jitter, so you can use this to smooth out the outcome without keeping history of data (other smoothing filters tend to need data from previous frames to do so). It's like the lowest hanging fruit smoothing.
LOVE your ability to take concepts i genuinely thought i knew almost everything about and then expand on them until even someone who’s been doing this stuff for a long time learns something new. Even concepts i was fairly familiar with you explained in a way I hadn’t exactly heard before, which helps recontextualise the tools we depend on!
yeah, it's one of those "it's close enough to working and I know it's wrong but it seems mostly fine so I'm going to ignore it until it becomes a problem" things you know
I saw the graph of the lerp smoothing function, and realized that it looked identical to a smoothing function I was using on some sampled data to clean it up - the "exponentially weighted moving average". I recently had to reformulate that to handle data being sampled at uneven intervals instead of at constant intervals… basically, making it framerate independent :)
I'm not a game dev, and I don't currently have any use for this awesome technique, but learning about it from you made my brain tingle in the best possible way, thank you!
Well, I was happy about anticipating the framerate dependence issue, and then it was a thrilling hour of hoping to keep up with the math. Exactly what I came for!
@Freya Holmér Great work of vulgarization of the lerp function. It is always nice when all this research and exploration is put into a nice little narrative. I'm sure it took (x * 1h, where x > 3) to prepare. Continue to share with us your enthusiasm about computer graphics. I, my self, am interested PIDs control and lerp is kind of open loop control.
lerp is linear interpolation. it's being able to go in a straight line between two 1D locations. it's as elementary as elementary math can be. this is not an education.
Love this. Your presentation is super engaging. I've decided to take some time to go through your math for game devs playlist as well. Thanks for the learning
When I tried to solve similar problem I somehow realized that there's relationship between log and square/cubic/...-roots and I actually don't understand why these relations work. Or why there are even things such as fractional powers and roots. And how logs turn multiplication to adition. I took old slide rule and started to reverse-engineering it to the level I was able to independently reinvent logarithms (having advantage that I had some clue what I'm doing) and find them using square roots iteratively (which are basically fractional powers). It was nice feeling when all puzzle pieces worked together. After half year, I almost forgot everything I learned.
Yup. It's always a bit of a surprise how much of math can be broken down and understood like that. Part of the problem is that, in school, we are told to memorize how it works and then process stuff like a computer. It makes math feel scary, since we aren't computers. But when we have some idea of what we're trying to do, and why it's important, and have some time to poke and prod at how stuff works, it's remarkable how much of math just *falls out* naturally. It's what mathematicians mean when they say it's beautiful.
Understanding logarithms and those relationships has been the most useful and magical bit of high school math for me. It comes up everywhere. It's amazing humanity didn't discover it until modern times.
yeah! fractional/rational powers are really unintuitive! I think most people start using them around the time you start using calculators though, and so I think it largely goes unquestioned even though it's (as far as I know) also often unexplained around the time of high school, at least it was in my case! I really wanted to go on a tangent about rational exponents in my (yet to be published) blog post but it's already ridiculously long so I ended up cutting it 💀 maybe I'll save that for the future!
I'm only halfway trough, and this is already super interesting and incredibly well made even for me, who is neiter a maths person or in gamedev. Hats off, Freya!
This is awesome Freya, I just tested the TLDR code and it works well. I love that if you use < -1 in the decay it will repel. I'd love to see the TLDR copy-paste version of the spring function but I'll wait for the blog post. Now I have to replace all those bad lerps in my game, which I'm grateful for knowing frame independence is closer.
A great lesson and future blog post. Love the visual graphics, and read the FAQ (it literally answered the question I had). The nice thing about iterated approximations is they only need to be good enough for the level of context required. With Zeno's example, if walking 1/2-way to a destination, as some point, the remaining distance will be smaller than his foot, so can't take another full step. Same with computer graphics, at some point the next step will be smaller than a pixel. This quantum (minimum iteration unit) could a parameter that stops the iteration process loop, when it reaches a good enough quantum value.
This was great. Thanks for all your cool content Freya! I especially enjoy the math for game dev series and as someone who did ALOT of procedural content generation(mainly procedural textures/graphics, in software on the CPU) before it was cool and before everything was done on the GPU, i also enjoyed the vids about procedural geometry as well as the shader basics one as that is some stuff i need to get updated on/learn in order to stay up to date.
This is actually insane that I was running into THIS EXACT PROBLEM over this weekend. I was porting over a minigame from TF2 called dodgeball into Minecraft and they used a LERP function to make the "ball" follow the player. The issue is that TF2 ticks at 66tps and Minecraft does 20tps so it was feeling different. Long story short I also found one of the equations you described to solve the issue, awesome talk :)
Fun fact, we just discovered (with a cool perspective twist) Exponential Moving Average (EMA) smoothing, Very commonly used in DSP-related fields, like audio (smoothing parameter changes, to avoid gittery clicks in your audio), embedded (de-noising signals captured by sensors). This technique is a way to efficiently discretize exponential function(s), which, needless to say, ubiquitously model many physical phenomena. Awesome talk Freya!
I guess the best attitude towards math is to have fun with it. When there's a dire situation by no means search the answer for it, and one shouldn't feel shameful doing it because time is precious especially for development work (where work can't really be divided much). But when there's leisure, well, that's when to really know what's inside. Knowing the answer of random what-ifs always brings fun, and so does feeling the power in knowledge. Like, because knowledge is power, learning new knowledge really gives the power fantasy vibe, which feels really good even if those knowledge will just go wasted. Also, love how you got the cat ears on perfectly! It often feels really hard to get cat ears on without them looking slightly off or asymmetric, for whatever reason, almost like there's a curse that prevents cat ears to be worn properly or something.
I got an ad break with some lady who asked: "Are you having a hard time understanding calculus? Are you having a hard time understanding your instructor?" And the answer is no, this was a very clear presentation, neat
Oh wow! So cool seeing something I had to figure out for myself get explained so well. I was using lerp smoothing to animate a camera following a route on a 3d map, and had to solve for framerate independence. Funnily that approach had a lot of flaws so it got scrapped for a better camera following technique lol.
There's also the fun problem of trying to make the 10fps client match better, or making this work with integers or fixed point numbers. Best solution to the latter I have found so far is to keep have a slower-updating counter, and then go back to using it every time it updates.
@@sheepcommander_ Well, you can see in the slide with all of the overlapping graphs, that there is still some inaccuracy, especially when Freya was moving the target quickly
@@shadamethyst1258 hmmmm I don't know if I'd call that "inaccuracy", it's more like, it's delayed, because it can't respond fast enough. I guess you *could* extrapolate to try and guess the next frame to stay ahead a little bit, but now you're clearly in the realm of inaccuracy, because now you're adding data that might be incorrect, as opposed to reading correct data a little later than you'd like, right?
@@acegikmo Well, I do wonder if a more accurate integration method wouldn't work better; you could always increase the accuracy by just computing substeps, though
So much overlap with audio in this one!! This is essentially the solution to a feature I've wanted to add to a stereo pingpong delay for a while now, but couldn't quite figure out. The trick, I see now, is to raise the per-repetition scaling factor to a factor corresponding to the ratio of frame durations, or in my case delay lengths. Gonna mull this over and see how well it works.. should be exactly what I needed.
Hahaha yeah it does work. My thing uses a feedback parameter which is the same as R in this. Tested it a bit, all I gotta do to match the decay curves for two audio delay lines of different duration is raise one channel's feedback parameter to the power of that channel's duration divided by the other channel's duration. That will either slow down the decay of a shorter cycle, or speed up the decay of a longer cycle. Did basic tests and it came out perfect using an analog for integer ratios. Gonna take a bit to implement in the actual audio version but I am very encouraged. Thanks for this and all your brilliant contributions to programming in general, it reaches out far beyond just graphics programming.
Audio version implemented and is indeed working. Keeps a stereo image in place as it decays, instead of it splitting apart further and further as the slower side takes longer to decay due to its longer repeat cycle. Very cool!
@@seedmole hah, yeah I've had many people mention audio processing! this is a rabbit hole I've only ever tangentially been around (when making music), but I worry if I look into the math I'll be lost in that world for another 4 years or something
@freya - I agree with your approach to problem-solving. It's much more valuable to understand WHY something works as well as just knowing that it works! You're awesome by the way! 🙂
Excellent presentation. I spent a few weeks working around this very problem in SDR# after a crappy Windows update that messed with the timing resolution. The solution involves timestamping and dynamic time compensation.
I like how this is basically a talk for high schoolers, but it's useful for game devs who might have missed some of their high school math. This is a great direct application of precalculus. High schoolers, you want to see how precalculus is used in real life, here you go! Seriously, if you know any high schoolers that are interested in game dev, they should see this; it's accessible and might give them a reason to remember this stuff. This is also the exact same math that goes into calculating interest; kids -- or older people who just didn't catch all their math in high school -- who are interested in finance might get a kick out of it if the connection is made clear to them.
hah, thanks! secretly this is because I never studied math beyond high school c: but also I think it helps people catch up, and makes it more broadly appealing!
I really love your content. I'm a cs teacher at a school and really love your educational style. You know how to generate real knowledge in other peoples brains.
This is actually very useful and topical for a personal project I recently have been working on - it's a game that involves time travel, and at some point while making it I realized that the objects more or less need to have their parameters all defined as a function of time rather than actually simulating stuff (well, there are simulation steps in there as "keyframes" are taken for the player inputs, but their position etc can be derived given any value _t_ - meanwhile, background objects with simple motion (say, constantly rotating gears) can't just do "angle += theta * dt", but have to go by a cosine, etc...). This is some good information on some more complex motions and how to derive them, I'm sure it'll come in handy moving forward!
This is perfect timing! I’ve been working on a custom text editor for myself for fun, and I want the cursor to “smear” around. I was just using earrings, but this’ll make it feel really nice. Thank you so much for this talk!
great presentation, as always! as pointed out by others, this is called a one-pole low-pass filter in the signal processing world. they are used all over the place in audio programming, mainly for gently smoothing parameters without overshoot but also directly to filter out high frequencies from any signal. the filter state is updated every tick. in pseudocode, this looks something like: z1 = z1 + (in - z1) * a where z1 is the state, in is the input to the filter and a0 is a parameter calculated from the desired cutoff frequency -> a = exp(-2.0 * pi * Fc), where Fc is the normalized frequency (frequency/samplerate).
you can cascade several of these one-pole filters in series to get a steeper filter curve, but you have to introduce a correction factor which is multiplied with the term inside the exp() call, otherwise the cutoff frequency is shifted. something like C(M) = 1 / sqrt(pow(2, 1/M) - 1) where M is the number of one-pole sections retains an attenuation of 1/sqrt(2) at the cutoff point. cascading M one-pole smoothers only requires 2M adds sums and M extra multiplies (as the coefficient stays the same for every section). it's also worth mentioning that there are other implementations of one-pole filters with different numerical properties. since one-pole filters are just leaky integrators, it's possible to implement them using other integration schemes - trapezoidal integration is a popular one, while the "standard" way of implementing one-pole filters corresponds to forward euler integration.
another neat thing you can do with one-poles is to efficiently produce exponential curves by pre-calculating the decay coefficient once, replacing an exponential evaluation with a single multiply and add per tick. this is especially useful for producing amplitude envelopes (e.g. ADSR envelopes), where exponential curves produce loudness variations which match our logarithmic loudness perception better than linear segments. it's possible to sidestep the "zeno's paradox" issue by making the one-pole overshoot its target slightly (having it approach a point slightly "beyond" its target) and clamping the output to the actual target value.
all linear resonators (e.g. instrument strings/woodwind bores) and essentially all undriven oscillations exhibit exponential decay in their response to a momentary excitation. reverb also decays exponentially after the direct sound stops.
@@acegikmo i feel the same way funnily enough but with computer graphics, because i'm interested in making audio-reactive visuals for my music (which is "procedurally generated" in realtime, to borrow game dev terminology). so far i've messed around a bit with raymarching and ported some shaders to my programming environment (Cycling74's Max, check it out, i'm sure you'd enjoy messing around with it! there's a free trial version) but i tend to get stuck in these rabbit holes as well
Great video once again! Congratualtions. I am in awe for your framework that you use for your videos and presentation. It's so perfect for your mission and it becomes your signature move. Super inspiring. PS: The dig at J. Blow was a bit out of nowhere and unnecessary but funny too.
Wow this dropped at the perfect time. I used smooth lerping 5 times last week and always thing (Im sure this is more complicated than I think...) I waddled through the talk like a confused baby, and amazingly, I could understand and apply all the options at the end. Wow!
I can't find the thread anymore, but if I remember correctly he was saying that you should always use the frame-rate independent version of lerp, and she said that the naive version is fine for things that aren't gameplay changing (like visuals) if you just want the quickest, simplest solution. Then he basically said "you're the reason games suck nowadays" and blocked her.
Not that this at all justifies his obnoxious behavior, but I find the math behind why the solution you derived (in a wonderful presentation btw) isn't exact in all cases to be pretty cool. I believe the reason to use something like rk4 is that the analytical solution you showed is only exactly correct when the target point is not moving. This was demonstrated when you showed the 10fps point lagging a bit behind the others. That happens because it separated from the others while the target was moving. You can imagine a scenario where the target point actually moves past where the following point is going to end up, causing overshoot. Essentially, the tracking point is tracking where the target used to be, not where it's going to be, which is more wrong for lower framerates and higher target velocities. RK4 and other integration methods can correct for some of this error!
Differential equations should be taught before university. At least enough so people can get an understanding. What's your favorite ODE? Blocked by Jonathan Blow for being technically correct? Based. Seems familiar. 😂
I used to respect Jonathan Blow, still do to an extent. He's clearly a very smart developer, but he also has too much of an ego for my taste and often comes across as pretentions, so while I respect his skills, I don't respect his opinions as much for the last few years. Not surprising he'd block someone over something like that.
This is also something you sometimes run across in simulations. For example often times some dampening factor to the momentum is used which effectively works like lerp smoothing towards 0 and of course it has those same issues. It's common to multiply the percentage of dampening with delta time but as you explain in your pinned comment that still has issues, the difference can be massive. Especially with simulation, where you might run a low-resolution one with a real-time delta time but run a more complicated high res one with super small delta time. Even worse, if you use delta time then depending on your percentage, you might just lerp with a factor of 1.
Ha! I love lerping. It always felt so elegant (except for the frame rate thing) As you were talking about solving for R I kept thinking... wouldn't be better to just solve for half the distance or something. THANKYOU FOR DOING JUST THAT!
Your explanation of problem-solving at 51:47 matches exactly with how I like to think, which also jibes with me enjoying your videos immensely (aside from the top-notch production quality). I don't just want to know the solution to my current problem, I want to know WHY it's the solution and WHY it works. That takes more time up front, but if I ever have a similar problem in the future, having that knowledge helps me figure out the solution to the new problem, so I usually consider it a worthy investment.
Holy cow, the interactive animations in the presentation are so illustrative! (I'm only partway through at this point.) I've always wanted to do this sort of thing. I imagine it can help give excellent A's during a Q&A at the end.
At 5:50 when you said to define a conversion factor I went "Ah, dimensional analysis, I'll be impressed if she mentions it" Then you started to talk about dimensional analysis right after and I was somewhat surprised, though I probably shouldn't have been since this seems really professional so far. I'll see if I can understand the rest of the video after this point, and think more than "That seems to have good logic" that is common when stuff above my understanding is described, but still understandable due to good explanations. Edit: I tried to do lerp smoothing in Desmos while watching, and failed. I could follow from step to step but it introduced too many new concepts to nail on a first watch. I'm sure that a rewatch would let me understand it more fully, and then do it in Desmos fine. This was a really nice explanation though, with it being very understandable and clear, and far better than the videos I watched for my Calc I, II, and III classes.
I can relate so hard to the part about just having learnt dimensional analysis! A few years ago I was trying to get finished with some missing grades in classical mechanics. And that time I had a teacher that understood the importance of knowing something rather than memorizing it. And when he taught us that, I just felt pure and plain shock. Because never in my life had I considered that you could do math like that And pretty much at that point I went from a failing student to a straight A one 😅
I knew off the rip it would land on the exponential decay function, then I was a little stunned that it somehow landed on (...)^n, then I vaguely recalled something from one of the calc's relating (...)^n to exponential decay, then the mention of diffeq caused both delight and mortal dread. :D The term geometric series popped out of the vault, possibly because it was one of the least comfortable sections for me at the time. Keep up the good work!
How is this a discussion in 2024 when PID control was discovered in 1911? Lerp appears to be proportional control without the time element. Other advanced techniques have been developed as well, such a MPC and LQR.
Just When i was Getting Into Lerp! Made me Realize that im just Starting out... Wooo! that is great! - Altough its Funny about Spring Theory, and Using Pow Equats to Lower Performance... Got to Learn how you Reverse Engineer it....
In an alternate timeline Freya ate a bagel instead of a croissant one morning and became an options trader instead of a game dev. (This lesson has many parallels to delta, gamma, theta, and vega in options trading.) Anyway I'm glad I'm in this timeline.
As someone who works in robotics, it's so fascinating to see how different developers use the lerp function. During this talk, it's mostly utilized as a low-pass filter, which is a first order differential equation (ODE). So it's not surprising to see the recursive functions and exponentials to be present in the final solution (e.g. solution to the ODE contains e and the Taylor expansion of e looks recursive!) In the literature, the lerp function when used in this way is a particular implementation of a discrete low-pass filter and there's a relationship of how to select the parameter F so that a target low-pass frequency rate is correctly enforced if the known sampling rate, delta time, is known. I've enjoyed seeing Freya derived the inner-workings of the lerp function from an applied first principles approach but without actually calling it a low-pass filter and mostly focusing on the problems faced by developers in the game industry. I also learned something: a = lerp(a, b, 1-pow(r,dt)) is a nice function that enforces smoothing, incorporates frame rate compensation, while retaining the intuitive meaning of the 'r' parameter. But as she mentioned a = b + (a-b)*exp(-decay*dt) is the more efficient implementation. Great job! I think every game dev practitioner can learn something from this talk.
thank you! and yeah, I usually create things from a practical and exploratory perspective, mostly because that's my actual discovery process! and because it usually helps other people like me ease into the concepts :)
Thank you Freya, that is super useful! Even though most of the explanation totally went over my head (I'm crap at math), I can still use that function in my projects.
Funny how these ideas appear in many disciplines. From a signal processing perspective lerp is cheap and dirty interpolation but gets the job done - windowed sinc interpolation (FIR style) is the real deal but costs more - an exponential integrator or IIR filter like this uses recursion, is cheap, but has non-linear phase. In this case it's an excellent conclusion. Love it! Power to tha low pass filter :)
![Vectors & Dot Product • Math for Game Devs [Part 1]](/img/n.gif)
FAQ!
*Q: What about lerp(a,b,F*Δt)?*
A: That's an approximation, and it happens to be a pretty good approximation! Especially if you are running at a high framerate, but it can falter pretty violently if you get lag spikes, or if you use large values of F. Even if you clamp it to not overshoot a value of 1, you can get disparate outcomes depending on framerate. Now, this whole talk is about half of what my (upcoming) blog post will cover! In the blog post, I spend a *lot* of time talking about approximations, including this one, and exactly where and how it fails c:
*Q: Can't you just use a fixed timestep loop, separate from the rendering update loop?*
A: You totally can! But if your fixed timestep is lower than your render framerate (which it usually is - unity defaults to 50fps in FixedUpdate for example), then you now have a new problem. Your motion will look stuttery because the position isn't updated as often as your rendering is, and so now you need to make it interpolate across those gaps before the next FixedUpdate hits. Now, in unity specifically, and specifically for rigidbodies only, there is support for interpolation, if enabled. However, keep in mind that this is an additional hack/patch, on top of your initial hack, of not using a framerate independent smoothing method. Personally I prefer just to use the framerate independent variant at the render framerate, unless it has to be a part of some existing timestep ecosystem, such as when you want it for rigidbodies
*Q: How did you make these slides/animations?*
A: It's a long story! Usually I use unity together with my animation tool + shapes, my vector graphics library. But this time around I used html/js/css, since this started as a (yet to be released) blog post on the same topic. I use html5 canvas to draw 2D graphics, and I use KaTeX to convert LaTeX into html elements. Pretty much everything else is custom made, from the slide nav to the interaction to the animations etc.! I have actually considered making a video about my descent into web dev in making this, because I have OPINIONS about javascript, lordie christ ;-;
*Q: Will you publish the slides?*
A: I already have! They're available for all my patreon supporters right here! www.patreon.com/posts/105228270
*Q: Will you publish something about the spring physics you showed near the end?*
A: Eventually yes! likely on my blog as well, maybe as part of the lerp smoothing blog post, or as its own separate thing afterwards since my lerp smoothing post is already enormous
Q: What about the y′′+2ζωy′+ω2y=0 ? It looks interesting, but even chatgpt dont know that and only 2 pages on google, not really mentioning a code! Show me a pseudocode for that pretty please :). Or confirm K1 K2 K3 K4 solution by chatgpt.
PLEASE do make that video about web dev. I am trying my hand at explanation-blogging and it's not easy, I want to see how you do it with your kind of quality.
> because I have OPINIONS about javascript, lordie christ ;-;
I am putting this post on my Christmas wish list. I recently fell back into wanting to do some web dev, and... yes. Lots of opinions, lots of feelings :')
That's super cool, I'd love to see your thoughts on web dev, especially how you're doing the canvas stuff. Right now I'm working on my first browser game (it's a picross/nonogram clone) and I'm only using the canvas, so it would be neat to hear about how you achieved certain things like smooth animations and making it run efficiently.
i need to know how to perfectly lerp
Freya is such a rock star educator that people are queuing in line to watch her teach something.
literally though?? I don't think it's visible in the video, but there were *so* many people, including a crowd gathered, *standing* behind all the (filled) seats. after my talk I spent about 3 hours nonstop talking to people who had questions! it's weird, but also really fun and flattering ❤
@@acegikmo
Well, Guido has it right. I don't know how the hell this got into my feed, but it's a pleasure to run across you for sure! 🤩🤣
@@acegikmoit’s so well earned. Absolutely S-rank.
@@acegikmo that proves that you are knowledgeable, intelligent, has the rare skill to teach others, and kind person.
“her”?
I'm going into the video thinking "oh yeah, I know that" and in the middle of the talk you go deeper into a topic than I ever did and learn something new. It's amazing! That's why I always watch your videos!
100%, she gets me every time
haha, good! I generally try and capture as many people as possible in the beginning, so that the more advanced stuff is built on a foundation I've communicated, rather than assuming everyone knows that stuff already. sometimes it's useful to get reminders of things you already know too, so I think it's worth it! even if some people might find it basic or boring for a bit
@@acegikmo Absofuckinglutely!
@@acegikmo so I was reading the newly released Tom7 paper (called Badness 0, it's on HN frontpage) and he was talking about how some people explore subjects "depth first" and he gave an example of Donald Knuth. I was just thinking how your videos have so much depth too
'babe wake up new hour long freya holmér presentation just dropped'
so real
Cool connection at 16:30:
It takes 53 iterations of halving. The mantissa of a 64bit float only has 53 (effective) bits of precision. This is not a coincidence!
Specifically, a float is stored as 1.mantissa x 2^exponent, where the mantissa is a binary number.
The number closest (but not equal) to 2 is hence 1.111...1b x 2^0 with 53 1s, equal to 1.99999999999999977795539507497 in decimal. Up to rounding, this is f(52) shown in the video!
On the next iteration, there is no nearer representable number -- it simply rounds to 2!
And indeed, it takes 52 iterations to get to this point, because at the nth iteration, a is 2^-n away from 2. The difference between 2 and 1.111...1b? 0.000...1b = 2^-52!
Bonus points if somebody knows how many iterations the example would have taken to get from 1 to 0 instead of to 2, taking subnormals into account
44100 is a constant you may be aware of. if not, you're too young.
44100 is a quarter, times 420, times 420.
a quarter
I feel like 2^-52! is overkill. Could have just been 2^-52. 😏
@@abebuckingham8198 2⁻⁵²! is actually around 1
@@F_A_F123 people don't respect the need for precision nearly as much as they should. why shouldn't we just round pi to 3.14? what could possibly go wrong?
Freya = the best math teacher I had + cat ears
If my teacher wears cat ears, I would never sleep in the class again
i didnt even notice the cat ears lmao
Cat ears are a subset of best teacher ever criteria
"Shit! I fixed it!" Debating whether that's a good or bad thing to hear a coworker say
If you just don't mention it, they'll put the bug back in the code and get free job security. So that depends on what you think about keeping that coworker around!
@@catmacopter8545if the fix was an accident and they aren’t sure why it is fixed, it can be vexing to not know the origin such that you could prevent it from recurring.
Hi Freya, you probably don't remember me in particular, but I hope you remember my class!
Me (and hopefully everyone) from Futuregames 2022 Game Programmer just wants to say "thank you!" for all you did for us! I was saddened that you got sick for our other course with you and I hope you feel much better these days! Every time I think of Lerp, I think of you!
With loving regards, an alumni from FutureGames!
@@acegikmo Oh no! You're not a teacher anymore? What are you currently doing?
@@acegikmo And now I'm confused... why is this verified account not 'Freya Holmér' but 'acegikmo'?
@@pinkeHelga because UA-cam decided to start showing handles instead of display names in comments for some weird reason
@@acegikmo Handles are unique, UA-cam ended the fun of trolling by impersonating other people in comments. Took them almost 20 years.
I love the cat ears, never thought a mathematician would be wearing nekomimi.
@@tbird81 she isn't, but you are
@@tbird81 Honest counter-question-why do you care? Why is the gender she was assigned at birth something you want to know?
@@tbird81 how is that relevant?
hmm, I think for me the cringiest part is that I didn't get a haircut before this event, my style works much better when my wavy inverted bob cut is properly showing, but now that it's grown out it just kinda looks awkward-medium length wavy
@@acegikmoyo I know nothing about gamedev or smth you doing, but I think you are not awkward and kinda cute really. goals tbh, maybe in 1-2 years all be a programmer/gamedeveloper wearing cat ears too ❤ you make it sound less confusing so thanks
6:22 oooh dimensional analysis is great! I love it, glad you learned about it.
It doesn't solve everything, but every once in a while it lets you skip some hard work
It's also a check that your calculations make sense to begin with, that the units at the end have the dimensions you actually want, so you don't waste a bunch of time, say, finding a neat, closed form solution to a nasty, nested integral that is actually fundamentally wrong 🙃
Great discussion. How to convert from a per frame implementation to a time based implementation should be game dev 101, but we've seen plenty of AAA game companies fail at it.
Yeah.
I dont have a memory of a recent AAA game failing at framerate dependent bugs.
@@mickaelgodardevery game that ties physics to frame rate and locks your game at an fps because of it is a failure
@@darthnix Agreed, but which one did that recently?
@@mickaelgodard name any Nintendo game
i'm a mathematician, and i love your stuff! i show your videos to my students as supplemental material and they find it soooo helpful. keep it up!!
Around 1999 there was a car racing game called Screamer, which was great, but the tire grip depended on framerate. The slower your PC the faster you could take corners.
I replayed it years later at 200fps and there was virtually no grip, like driving on ice
Rollcage came out the same year and was one of my favorite PC racing games, but it had the same problem. I loved that game but when I convinced a friend to buy it and tried to play him vs, he said that the steering was really bad. I thought he just needed to get good until I tried it myself. His system was faster and the steering was clearly tied to the frame rate. I thought that was an amateur mistake by the developer, but this talk has convinced me that it's a more subtle problem than I thought, and I've probably made similar mistakes in my own game development.
@@adsilcott Oh, you don't have to go so far back, in F1 2022 and 2023 (don't know about 2024), physics are framerate dependent. I mean, you don't have 0 grip at high framerates, but with a low enought performance you could almost ignore curbs.
Also, random fact, I was searching for Screamer, and it seems that the studio is now the one that works in Moto GP games, Ride franchice, the new Hotwheel game and some more.
Need for Speed: Rivals (by EA) has the whole physics system tied to the framerate. If you are lagging the whole game will play slower, and if you unlock it to high fps the game plays like a timelapse
@@curryleavesbydhanya wow, that's bad for a "modern" title.
You could also have slowmotion in Grand Prix 2 back in 1996, when the PC was too slow, but apart from that it was accurate
Gotta admit, at this point I see Freya, I click.
(I'm not sure I'd even encountered the word before, let alone considered how to smooth one.)
The lerp saga continues
Also dang learning quarternions before differential equations feels like the wrong order of things 😂 I for sure assumed you'd have that down before. But probably just my uni bias since i still dont get quarternions but am quite comfortable with differential equations
nah, quaternions are just algebra.
@@APaleDotyeah agreed, it's just not introduced because within math outside of graphics it's relatively niche
I think it's a side effect of not having an academic background in math, the way/order I study things is entirely by happenstance, so I tend to have a very lopsided skill set
@@APaleDot Actually quaternions aren’t just algebra because they for example have a smooth structure. Like complex numbers (well and even real numbers)-complex analysis is even very famous, but quaternionic analysis is a bit bad so it’s usually not that well-known like real and complex cases.
And I think one can study differential equations for a long long time, there’s just so much stuff, including hairy analytic stuff and weak solutions and all the worst that math can throw at you. Especially if we include PDEs (really they are a separate subject just for technical reasons and due so a couple of differences in application).
So if you even studied ODEs and maybe PDEs at uni and even if you had a math specialty, you can still be far from having a grasp on them that is sufficient for many many areas and tasks. Likewise for quaternions and all other math though: for example I don’t think people usually are aware that you can forget about concrete i, j, k (like we can swap ±i in ℂ, we can rotate the imaginary unit sphere any way we wish and this won’t change any algebra, so if our results don’t directly manipulate with i, j, k, they’re probably more useful etc., compared to if they do depend on occurrences of i, j, k-well, except for quaternion implementation and numerical algorithms, I suppose).
If df(x) = af(x), chances are good you have an exponential; _the_ function whose rate of change is proportional to its value.
Fun fact, a 90° turn is legally a constant proportion, so circular motion, whose rate of change is a vector 90° from its current position, is also an exponential. Often this "90° turn" is written as "i", so circular motion is expressed as e^it, which may or may not equal cos(t) + isin(t).
Now we just need to do _spherical_ LERP smoothing along a B-spline of dual-quaternions, and we'll still probably not have the most complicated way to get from point-A to point-B.
This is the first explanation I've heard of complex exponentials as sinusoids that has actually felt intuitive to me! Crazy to have stumbled across such a gem in a UA-cam comment section :)
I hope you have a wonderful day!
@@cineblazer
Yep, the hallmark of circular motion is that the velocity is always perpendicular to the displacement from the center of the circle. With complex numbers, you just multiply by 'i' to rotate by 90°. So you could say that the velocity is just the displacement multiplied by 'i' (or some constant multiple of 'i').
Since exponentials are always proportional to their derivatives, as shown by d(e^(ax)) = ae^(ax) dx, we can see that if the displacement is s(t) = e^(it) then ds(t) = v(t) = ie^(it) dt.
So we have v(t) = is(t), fulfilling the requirement for circular motion.
@@cineblazer Any function that only depends on a prior position and a time since said position can be described as an exponential. This includes exponential growth & decay, rotations, and even in a way _translations._
This even relates to the limit definition of the exponential exp(x) = lim n->∞ (1 + x/n)^n, where you're essentially breaking x up into infinitely many small chunks, and then repeatedly scaling the identity by the 1 + a single chunk. Whether that's by repeatedly eliminating 30% of a set of remaining dots, or simply taking one step after another to slowly inch your way across a mile, they all boil down to the same concepts.
There are no coincidences in math; only special cases.
Wait isn't that just tan(a)?
Another secret is if you remember from Freya’s “why can’t you multiply vectors” talk, i is a unit bivector - an oriented plane segment representing rotation. The fact that e^i represents rotation is then no coincidence! In fact, with the 3D bivectors i j and k, if you have some bivector B = a*i + b*j + c*k, the *exponential* of that bivector e^(B*theta) works **exactly** like e^(i*theta) - it gives you a rotor* representing the rotation in the plane of the original bivector by the angle theta. Just like the exponential discussed, the bivector itself is another constant value - in fact, as alluded to in her talk, it is **the circulation bivector!!** This means that the formula for following a curve can be described as iteratively applying e^C(t)dt (or the corresponding integral). [I think the formula’s right I may be off by a few factors]
*rotors are only mentioned briefly in the talk, but they are just scalars + bivectors. You need scalars as well to represent arbitrary (non-90*) rotations for the same reason you need scalars in complex numbers.
Hmm, not a game dev but I find it fascinating that the same smoothing idea can show up in many places. In game dev, it’s lerp smoothing. In finance, it’s the exponential moving average (EMA). In my field, electrical engineering, I also just call it the EMA, but others call it the exponential filter or even the “1-pole IIR filter”. And, well, I never considered going about this issue without bringing up Fourier and cutoff frequencies, but here’s an explanation using just algebra (not even calculus!). Like, I never made that connection between the framerate dependency and the decay. Very interesting…
EDIT: I remember now you mentioned that you do that clamp to prevent wiggling. When that’s done, it does becomes slightly different from the EMA. What you’re seeing is probably the Gibbs phenomenon. It takes particular filters like the Bessel filter to get rid of it without clamping, but I’ve also seen people just do clamping. For example, some variations of JPEG rendering do this so that you don’t get JPEG artifacts.
FWIW, I used this for smoothing out joint animation data coming out of a Kinect sensor. Sometimes joints jitter, so you can use this to smooth out the outcome without keeping history of data (other smoothing filters tend to need data from previous frames to do so). It's like the lowest hanging fruit smoothing.
once i saw those cat ears i knew i was listening to a professional
LOVE your ability to take concepts i genuinely thought i knew almost everything about and then expand on them until even someone who’s been doing this stuff for a long time learns something new. Even concepts i was fairly familiar with you explained in a way I hadn’t exactly heard before, which helps recontextualise the tools we depend on!
I love Freya’s talks, where she explains things with beautiful animations and cats icons
Thank you so much! I used lerp smoothing for a project and I fully understood all its issues, but I had no idea how to solve it. This was very useful!
yeah, it's one of those "it's close enough to working and I know it's wrong but it seems mostly fine so I'm going to ignore it until it becomes a problem" things you know
freya more like feynman with how much you make learning feel like discovery
hmmm... with the cat ears, why not call her feynyan?
Omg it's been a while, I'm so glad you're still making educational content
I saw the graph of the lerp smoothing function, and realized that it looked identical to a smoothing function I was using on some sampled data to clean it up - the "exponentially weighted moving average". I recently had to reformulate that to handle data being sampled at uneven intervals instead of at constant intervals… basically, making it framerate independent :)
I'm not a game dev, and I don't currently have any use for this awesome technique, but learning about it from you made my brain tingle in the best possible way, thank you!
I am consistently impressed with how well you present this type of material. Always well made, and always worth the watch. Thanks!
The exp() and exp2() methods also approach 1 asymptotically and thus, naturally prevent overshoot, even at extremely low framerates.
Well, I was happy about anticipating the framerate dependence issue, and then it was a thrilling hour of hoping to keep up with the math. Exactly what I came for!
hey you really inspire me; I could never get up and teach.. but self-teaching and getting out there seems great
@Freya Holmér Great work of vulgarization of the lerp function. It is always nice when all this research and exploration is put into a nice little narrative. I'm sure it took (x * 1h, where x > 3) to prepare. Continue to share with us your enthusiasm about computer graphics. I, my self, am interested PIDs control and lerp is kind of open loop control.
lerp is linear interpolation. it's being able to go in a straight line between two 1D locations.
it's as elementary as elementary math can be. this is not an education.
Love this. Your presentation is super engaging. I've decided to take some time to go through your math for game devs playlist as well. Thanks for the learning
When I tried to solve similar problem I somehow realized that there's relationship between log and square/cubic/...-roots and I actually don't understand why these relations work. Or why there are even things such as fractional powers and roots. And how logs turn multiplication to adition. I took old slide rule and started to reverse-engineering it to the level I was able to independently reinvent logarithms (having advantage that I had some clue what I'm doing) and find them using square roots iteratively (which are basically fractional powers). It was nice feeling when all puzzle pieces worked together. After half year, I almost forgot everything I learned.
Yup. It's always a bit of a surprise how much of math can be broken down and understood like that. Part of the problem is that, in school, we are told to memorize how it works and then process stuff like a computer. It makes math feel scary, since we aren't computers. But when we have some idea of what we're trying to do, and why it's important, and have some time to poke and prod at how stuff works, it's remarkable how much of math just *falls out* naturally. It's what mathematicians mean when they say it's beautiful.
Understanding logarithms and those relationships has been the most useful and magical bit of high school math for me. It comes up everywhere. It's amazing humanity didn't discover it until modern times.
yeah! fractional/rational powers are really unintuitive! I think most people start using them around the time you start using calculators though, and so I think it largely goes unquestioned even though it's (as far as I know) also often unexplained around the time of high school, at least it was in my case!
I really wanted to go on a tangent about rational exponents in my (yet to be published) blog post but it's already ridiculously long so I ended up cutting it 💀 maybe I'll save that for the future!
I'm only halfway trough, and this is already super interesting and incredibly well made even for me, who is neiter a maths person or in gamedev. Hats off, Freya!
This is awesome Freya, I just tested the TLDR code and it works well. I love that if you use < -1 in the decay it will repel. I'd love to see the TLDR copy-paste version of the spring function but I'll wait for the blog post. Now I have to replace all those bad lerps in my game, which I'm grateful for knowing frame independence is closer.
A great lesson and future blog post. Love the visual graphics, and read the FAQ (it literally answered the question I had).
The nice thing about iterated approximations is they only need to be good enough for the level of context required. With Zeno's example, if walking 1/2-way to a destination, as some point, the remaining distance will be smaller than his foot, so can't take another full step. Same with computer graphics, at some point the next step will be smaller than a pixel. This quantum (minimum iteration unit) could a parameter that stops the iteration process loop, when it reaches a good enough quantum value.
This was great.
Thanks for all your cool content Freya! I especially enjoy the math for game dev series and as someone who did ALOT of procedural content generation(mainly procedural textures/graphics, in software on the CPU) before it was cool and before everything was done on the GPU, i also enjoyed the vids about procedural geometry as well as the shader basics one as that is some stuff i need to get updated on/learn in order to stay up to date.
there is absolutely no one better to teach me something about interpolation than a brilliant woman in cat ears
This is actually insane that I was running into THIS EXACT PROBLEM over this weekend. I was porting over a minigame from TF2 called dodgeball into Minecraft and they used a LERP function to make the "ball" follow the player. The issue is that TF2 ticks at 66tps and Minecraft does 20tps so it was feeling different. Long story short I also found one of the equations you described to solve the issue, awesome talk :)
Absolutely loved the presentation, as always Freya, you’re a great science communicator.
Thank you! Awesome talk!
thank you for existing 💗
Fun fact, we just discovered (with a cool perspective twist) Exponential Moving Average (EMA) smoothing, Very commonly used in DSP-related fields, like audio (smoothing parameter changes, to avoid gittery clicks in your audio), embedded (de-noising signals captured by sensors). This technique is a way to efficiently discretize exponential function(s), which, needless to say, ubiquitously model many physical phenomena.
Awesome talk Freya!
its time to multiply movement by deltatime
I guess the best attitude towards math is to have fun with it. When there's a dire situation by no means search the answer for it, and one shouldn't feel shameful doing it because time is precious especially for development work (where work can't really be divided much). But when there's leisure, well, that's when to really know what's inside. Knowing the answer of random what-ifs always brings fun, and so does feeling the power in knowledge. Like, because knowledge is power, learning new knowledge really gives the power fantasy vibe, which feels really good even if those knowledge will just go wasted.
Also, love how you got the cat ears on perfectly! It often feels really hard to get cat ears on without them looking slightly off or asymmetric, for whatever reason, almost like there's a curse that prevents cat ears to be worn properly or something.
babe wake up freya just uploaded
I got an ad break with some lady who asked: "Are you having a hard time understanding calculus? Are you having a hard time understanding your instructor?" And the answer is no, this was a very clear presentation, neat
I love you just had a mini talk after your talk xP
I took an ODE course as part of my degree, and it is pretty awesome when you finally understand everything and can apply it to things like this!
Oh wow! So cool seeing something I had to figure out for myself get explained so well. I was using lerp smoothing to animate a camera following a route on a 3d map, and had to solve for framerate independence. Funnily that approach had a lot of flaws so it got scrapped for a better camera following technique lol.
There's also the fun problem of trying to make the 10fps client match better, or making this work with integers or fixed point numbers.
Best solution to the latter I have found so far is to keep have a slower-updating counter, and then go back to using it every time it updates.
this doesn't match the 10fps client?
@@sheepcommander_ Well, you can see in the slide with all of the overlapping graphs, that there is still some inaccuracy, especially when Freya was moving the target quickly
@@shadamethyst1258 In godot i saw people talking about using the Engine.get_physics_interpolation_fraction() but i still don't understand it
@@shadamethyst1258 hmmmm I don't know if I'd call that "inaccuracy", it's more like, it's delayed, because it can't respond fast enough. I guess you *could* extrapolate to try and guess the next frame to stay ahead a little bit, but now you're clearly in the realm of inaccuracy, because now you're adding data that might be incorrect, as opposed to reading correct data a little later than you'd like, right?
@@acegikmo Well, I do wonder if a more accurate integration method wouldn't work better; you could always increase the accuracy by just computing substeps, though
So much overlap with audio in this one!! This is essentially the solution to a feature I've wanted to add to a stereo pingpong delay for a while now, but couldn't quite figure out. The trick, I see now, is to raise the per-repetition scaling factor to a factor corresponding to the ratio of frame durations, or in my case delay lengths. Gonna mull this over and see how well it works.. should be exactly what I needed.
Hahaha yeah it does work. My thing uses a feedback parameter which is the same as R in this. Tested it a bit, all I gotta do to match the decay curves for two audio delay lines of different duration is raise one channel's feedback parameter to the power of that channel's duration divided by the other channel's duration. That will either slow down the decay of a shorter cycle, or speed up the decay of a longer cycle. Did basic tests and it came out perfect using an analog for integer ratios. Gonna take a bit to implement in the actual audio version but I am very encouraged. Thanks for this and all your brilliant contributions to programming in general, it reaches out far beyond just graphics programming.
Audio version implemented and is indeed working. Keeps a stereo image in place as it decays, instead of it splitting apart further and further as the slower side takes longer to decay due to its longer repeat cycle. Very cool!
@@seedmole hah, yeah I've had many people mention audio processing! this is a rabbit hole I've only ever tangentially been around (when making music), but I worry if I look into the math I'll be lost in that world for another 4 years or something
@@acegikmoUp next on Freya Holmér: The Decay of Soundwaves /s
As always, incredible!
@freya - I agree with your approach to problem-solving. It's much more valuable to understand WHY something works as well as just knowing that it works! You're awesome by the way! 🙂
38:06 I think you can just swap a and b instead of doing 1-pow 😇
yeah, but that's a dangerously subtle change that risks introducing human error and confusion
It is indeed beautiful and mesmerizing; thank you Freya for explaining it so well
Excellent presentation. I spent a few weeks working around this very problem in SDR# after a crappy Windows update that messed with the timing resolution. The solution involves timestamping and dynamic time compensation.
I like how this is basically a talk for high schoolers, but it's useful for game devs who might have missed some of their high school math. This is a great direct application of precalculus. High schoolers, you want to see how precalculus is used in real life, here you go! Seriously, if you know any high schoolers that are interested in game dev, they should see this; it's accessible and might give them a reason to remember this stuff. This is also the exact same math that goes into calculating interest; kids -- or older people who just didn't catch all their math in high school -- who are interested in finance might get a kick out of it if the connection is made clear to them.
hah, thanks! secretly this is because I never studied math beyond high school c: but also I think it helps people catch up, and makes it more broadly appealing!
I really love your content. I'm a cs teacher at a school and really love your educational style. You know how to generate real knowledge in other peoples brains.
I was there!, I enjoyed a lot your presentation, very stylish as always! It was great to welcome you in Spain
and thanks for joining the talk!!
This is actually very useful and topical for a personal project I recently have been working on - it's a game that involves time travel, and at some point while making it I realized that the objects more or less need to have their parameters all defined as a function of time rather than actually simulating stuff (well, there are simulation steps in there as "keyframes" are taken for the player inputs, but their position etc can be derived given any value _t_ - meanwhile, background objects with simple motion (say, constantly rotating gears) can't just do "angle += theta * dt", but have to go by a cosine, etc...). This is some good information on some more complex motions and how to derive them, I'm sure it'll come in handy moving forward!
Speaking of time-travel, the guy who blocked Freya on twitter has a talk about how he implemented Braid. Have you seen it?
You're great at explaining.
Very thorough and intuitive. And I think we now have the P part of a PID controller.
This is perfect timing! I’ve been working on a custom text editor for myself for fun, and I want the cursor to “smear” around. I was just using earrings, but this’ll make it feel really nice. Thank you so much for this talk!
This is a brilliant presentation - I love the attention to detail in the visualizations! It gets me fired up about being an educator.
great presentation, as always!
as pointed out by others, this is called a one-pole low-pass filter in the signal processing world. they are used all over the place in audio programming, mainly for gently smoothing parameters without overshoot but also directly to filter out high frequencies from any signal. the filter state is updated every tick. in pseudocode, this looks something like:
z1 = z1 + (in - z1) * a
where z1 is the state, in is the input to the filter and a0 is a parameter calculated from the desired cutoff frequency -> a = exp(-2.0 * pi * Fc), where Fc is the normalized frequency (frequency/samplerate).
you can cascade several of these one-pole filters in series to get a steeper filter curve, but you have to introduce a correction factor which is multiplied with the term inside the exp() call, otherwise the cutoff frequency is shifted. something like C(M) = 1 / sqrt(pow(2, 1/M) - 1) where M is the number of one-pole sections retains an attenuation of 1/sqrt(2) at the cutoff point. cascading M one-pole smoothers only requires 2M adds sums and M extra multiplies (as the coefficient stays the same for every section). it's also worth mentioning that there are other implementations of one-pole filters with different numerical properties. since one-pole filters are just leaky integrators, it's possible to implement them using other integration schemes - trapezoidal integration is a popular one, while the "standard" way of implementing one-pole filters corresponds to forward euler integration.
another neat thing you can do with one-poles is to efficiently produce exponential curves by pre-calculating the decay coefficient once, replacing an exponential evaluation with a single multiply and add per tick. this is especially useful for producing amplitude envelopes (e.g. ADSR envelopes), where exponential curves produce loudness variations which match our logarithmic loudness perception better than linear segments. it's possible to sidestep the "zeno's paradox" issue by making the one-pole overshoot its target slightly (having it approach a point slightly "beyond" its target) and clamping the output to the actual target value.
all linear resonators (e.g. instrument strings/woodwind bores) and essentially all undriven oscillations exhibit exponential decay in their response to a momentary excitation. reverb also decays exponentially after the direct sound stops.
hmmm I've been kinda scared of signal processing because if I start, I'd be stuck in that math snipe for years
@@acegikmo i feel the same way funnily enough but with computer graphics, because i'm interested in making audio-reactive visuals for my music (which is "procedurally generated" in realtime, to borrow game dev terminology). so far i've messed around a bit with raymarching and ported some shaders to my programming environment (Cycling74's Max, check it out, i'm sure you'd enjoy messing around with it! there's a free trial version) but i tend to get stuck in these rabbit holes as well
The visuals on this presentation are so freaking good! It's just perfect!
Great video once again! Congratualtions.
I am in awe for your framework that you use for your videos and presentation. It's so perfect for your mission and it becomes your signature move. Super inspiring.
PS: The dig at J. Blow was a bit out of nowhere and unnecessary but funny too.
Omg I love the cat ears I want to wear cat ears and teach physics someday
i love how she rocks the ears, just mazing style
Wow this dropped at the perfect time. I used smooth lerping 5 times last week and always thing (Im sure this is more complicated than I think...)
I waddled through the talk like a confused baby, and amazingly, I could understand and apply all the options at the end. Wow!
Why did Jonathan Blow block you? 😲
bump
Delta t insecurity
I can't find the thread anymore, but if I remember correctly he was saying that you should always use the frame-rate independent version of lerp, and she said that the naive version is fine for things that aren't gameplay changing (like visuals) if you just want the quickest, simplest solution.
Then he basically said "you're the reason games suck nowadays" and blocked her.
pretty much! he specifically argued for numeric solutions like RK4, which, I'm not sure why, since there's a fast analytic solution anyway
Not that this at all justifies his obnoxious behavior, but I find the math behind why the solution you derived (in a wonderful presentation btw) isn't exact in all cases to be pretty cool. I believe the reason to use something like rk4 is that the analytical solution you showed is only exactly correct when the target point is not moving. This was demonstrated when you showed the 10fps point lagging a bit behind the others. That happens because it separated from the others while the target was moving. You can imagine a scenario where the target point actually moves past where the following point is going to end up, causing overshoot. Essentially, the tracking point is tracking where the target used to be, not where it's going to be, which is more wrong for lower framerates and higher target velocities. RK4 and other integration methods can correct for some of this error!
Differential equations should be taught before university. At least enough so people can get an understanding.
What's your favorite ODE?
Blocked by Jonathan Blow for being technically correct? Based. Seems familiar. 😂
I used to respect Jonathan Blow, still do to an extent. He's clearly a very smart developer, but he also has too much of an ego for my taste and often comes across as pretentions, so while I respect his skills, I don't respect his opinions as much for the last few years. Not surprising he'd block someone over something like that.
Now I wanna read the twitter thread that led to Jonathan Blow blocking Freya. Tried searching for it on Twitter but couldn't find it XD
@@DreadKyller I think life experience makes it very clear that intelligence is not a worthwhile reason to respect someone.
This is also something you sometimes run across in simulations. For example often times some dampening factor to the momentum is used which effectively works like lerp smoothing towards 0 and of course it has those same issues. It's common to multiply the percentage of dampening with delta time but as you explain in your pinned comment that still has issues, the difference can be massive. Especially with simulation, where you might run a low-resolution one with a real-time delta time but run a more complicated high res one with super small delta time.
Even worse, if you use delta time then depending on your percentage, you might just lerp with a factor of 1.
Ha! I love lerping. It always felt so elegant (except for the frame rate thing) As you were talking about solving for R I kept thinking... wouldn't be better to just solve for half the distance or something. THANKYOU FOR DOING JUST THAT!
Loved this talk
Your explanation of problem-solving at 51:47 matches exactly with how I like to think, which also jibes with me enjoying your videos immensely (aside from the top-notch production quality). I don't just want to know the solution to my current problem, I want to know WHY it's the solution and WHY it works. That takes more time up front, but if I ever have a similar problem in the future, having that knowledge helps me figure out the solution to the new problem, so I usually consider it a worthy investment.
Holy cow, the interactive animations in the presentation are so illustrative! (I'm only partway through at this point.) I've always wanted to do this sort of thing. I imagine it can help give excellent A's during a Q&A at the end.
I have a PhD and I would love to had you while I was learning molecular mechanics. Thank you very much for sharing ❤
At 5:50 when you said to define a conversion factor I went "Ah, dimensional analysis, I'll be impressed if she mentions it" Then you started to talk about dimensional analysis right after and I was somewhat surprised, though I probably shouldn't have been since this seems really professional so far. I'll see if I can understand the rest of the video after this point, and think more than "That seems to have good logic" that is common when stuff above my understanding is described, but still understandable due to good explanations.
Edit: I tried to do lerp smoothing in Desmos while watching, and failed. I could follow from step to step but it introduced too many new concepts to nail on a first watch. I'm sure that a rewatch would let me understand it more fully, and then do it in Desmos fine. This was a really nice explanation though, with it being very understandable and clear, and far better than the videos I watched for my Calc I, II, and III classes.
Pure awesome. Learnt lots, you rock!
I remember being so proud when I figured this out for myself years ago 😁
this is practically pre-calculus!
So lerp smoothing is iteration of the lerp function? Like the chaos game and like the logistic map?
I validate and specialize you. very nice talk.
I can relate so hard to the part about just having learnt dimensional analysis! A few years ago I was trying to get finished with some missing grades in classical mechanics. And that time I had a teacher that understood the importance of knowing something rather than memorizing it. And when he taught us that, I just felt pure and plain shock. Because never in my life had I considered that you could do math like that
And pretty much at that point I went from a failing student to a straight A one 😅
Am happy to see Freya Again wohoo ,lets gooo!!!
I knew off the rip it would land on the exponential decay function, then I was a little stunned that it somehow landed on (...)^n, then I vaguely recalled something from one of the calc's relating (...)^n to exponential decay, then the mention of diffeq caused both delight and mortal dread. :D
The term geometric series popped out of the vault, possibly because it was one of the least comfortable sections for me at the time.
Keep up the good work!
How is this a discussion in 2024 when PID control was discovered in 1911? Lerp appears to be proportional control without the time element. Other advanced techniques have been developed as well, such a MPC and LQR.
right? I can't believe people still teach arithmetic, that stuff was developed in 3000 BCE
Just When i was Getting Into Lerp!
Made me Realize that im just Starting out... Wooo!
that is great! - Altough its Funny about Spring Theory, and Using Pow Equats to Lower Performance... Got to Learn how you Reverse Engineer it....
The simplification shenanigans at "Unraveling recursion" are gold.
Note that lerp(a,b,1-t) = lerp(b,a,t), so we can express the final answer in code as
a = lerp(b, a, exp(-dt*decay))
In an alternate timeline Freya ate a bagel instead of a croissant one morning and became an options trader instead of a game dev. (This lesson has many parallels to delta, gamma, theta, and vega in options trading.) Anyway I'm glad I'm in this timeline.
freya talks are always so amazing
As someone who works in robotics, it's so fascinating to see how different developers use the lerp function. During this talk, it's mostly utilized as a low-pass filter, which is a first order differential equation (ODE). So it's not surprising to see the recursive functions and exponentials to be present in the final solution (e.g. solution to the ODE contains e and the Taylor expansion of e looks recursive!) In the literature, the lerp function when used in this way is a particular implementation of a discrete low-pass filter and there's a relationship of how to select the parameter F so that a target low-pass frequency rate is correctly enforced if the known sampling rate, delta time, is known.
I've enjoyed seeing Freya derived the inner-workings of the lerp function from an applied first principles approach but without actually calling it a low-pass filter and mostly focusing on the problems faced by developers in the game industry. I also learned something: a = lerp(a, b, 1-pow(r,dt)) is a nice function that enforces smoothing, incorporates frame rate compensation, while retaining the intuitive meaning of the 'r' parameter. But as she mentioned a = b + (a-b)*exp(-decay*dt) is the more efficient implementation.
Great job! I think every game dev practitioner can learn something from this talk.
thank you! and yeah, I usually create things from a practical and exploratory perspective, mostly because that's my actual discovery process! and because it usually helps other people like me ease into the concepts :)
Amazing, you are a genius! Thank you for making this show about math & programming for us to watch!
Thank you Freya, that is super useful! Even though most of the explanation totally went over my head (I'm crap at math), I can still use that function in my projects.
Funny how these ideas appear in many disciplines. From a signal processing perspective lerp is cheap and dirty interpolation but gets the job done - windowed sinc interpolation (FIR style) is the real deal but costs more - an exponential integrator or IIR filter like this uses recursion, is cheap, but has non-linear phase. In this case it's an excellent conclusion. Love it! Power to tha low pass filter :)
Top 5 mathematicians to see live. "What do you do fam?" "I go to math shows. Have you ever seen Freya Holmer differentiate a spline?"
Woohoo, another presentation tk watch multiple times!
Wow Vtuber tech has gotten really advanced
A day with a new video from Freya is a good day.