I think at 14:31 it would be more correct to set expire to: (86400 - math.floor(time.time()) % 86400) + 1 Because if the user has not used the API throughout the day, but uses it the last second of the day, the key will remain in the database for another day, since expire is set to this period.
Second method can be done in more easier way. Simply, store epoch time in the value (if does not exists). In each request, increment it for some number (this number relates to the "window" lenght). Set the expiration to the value. Check difference between the value and current epoch time (this determines number of requests during the window)
Excellent presentation with lots of good detail. Walking through the code examples is awesome. Thanks! Unfortunately the audio is garbled for several minutes and I wonder if I am missing out on some good content. Can you please fix and repost?
In the second approach, when you are setting expiry for each record for a day, Why do you have to again actively remove keys that are older than 1 day using zremrangebyscore ?
the second approach uses sorted sets... isnt that expensive... can we not use hyperloglog without being too expensive... imagine a uses case where there is spike in traffic and there is a large sorted set that needs to be read for every request?
why the key should contain the timestamp of the start of day? Isn't that possible to save the key without the timestamp and rely on the ttl by setting it only when no record exist (=first request in a time windows of a day)?
Thank you very much for this insightful video! Is my understanding correct that zadd call actually increase memory footprint of the particular sortedset instance inside of the Redis instance? We do this for every API call being evaluated - so, if I understand correctly, memory footprint of Redis instance linearly depends on API call number . Could this be used to attack our system and if so how to configure Redis to prevent this?
Really glad you liked it! We are working to bring a lot more content to our UA-cam very soon! subscriber here, or jump into our discord channel to chat with active Redis users who can answer your questions! Join our Discord server → discord.gg/redis
Upstash is making a lot of money off of devs who CAN'T setup up a rate limiter with your product... What more should I say... And here, you guys have come up with a code that is in python... poor choices...
Thanks for the explanation. First method seems to be the fixed window approach and the second one is specifically the Sliding Window Log method.
audio issues between 22:20 - 23:15
I think at 14:31 it would be more correct to set expire to: (86400 - math.floor(time.time()) % 86400) + 1
Because if the user has not used the API throughout the day, but uses it the last second of the day, the key will remain in the database for another day, since expire is set to this period.
Second method can be done in more easier way. Simply, store epoch time in the value (if does not exists). In each request, increment it for some number (this number relates to the "window" lenght). Set the expiration to the value. Check difference between the value and current epoch time (this determines number of requests during the window)
i think your approach cannot be done atomically, so it might be subject to race conditions
Good explanation. Extremely helpful.
Excellent presentation with lots of good detail. Walking through the code examples is awesome. Thanks! Unfortunately the audio is garbled for several minutes and I wonder if I am missing out on some good content. Can you please fix and repost?
On slide 15:25, why did you read the value after you issued (incrby) ? Because incrby already returns the new value
Same question
In the second approach, when you are setting expiry for each record for a day, Why do you have to again actively remove keys that are older than 1 day using zremrangebyscore ?
Nice explanation 👍
the second approach uses sorted sets... isnt that expensive... can we not use hyperloglog without being too expensive... imagine a uses case where there is spike in traffic and there is a large sorted set that needs to be read for every request?
why the key should contain the timestamp of the start of day? Isn't that possible to save the key without the timestamp and rely on the ttl by setting it only when no record exist (=first request in a time windows of a day)?
I believe that is the good approach .. although as a gate keeping mechanism, its better to remove the key if it is longer than 24 hours as well.
Thank you very much for this insightful video!
Is my understanding correct that zadd call actually increase memory footprint of the particular sortedset instance inside of the Redis instance? We do this for every API call being evaluated -
so, if I understand correctly, memory footprint of Redis instance linearly depends on API call number . Could this be used to attack our system and if so how to configure Redis to prevent this?
Really glad you liked it! We are working to bring a lot more content to our UA-cam very soon! subscriber here, or jump into our discord channel to chat with active Redis users who can answer your questions! Join our Discord server → discord.gg/redis
22:39 audio issues make it unlistenable
Please remake this video
#cool
😎
Upstash is making a lot of money off of devs who CAN'T setup up a rate limiter with your product...
What more should I say...
And here, you guys have come up with a code that is in python...
poor choices...
Thank for the feedback!
several minutes of this video is garbled.. you need to repost..