Caching Your API Requests (JSON) In Python Is A Major Optimization

Today we will be looking at how we can cache our JSON data from our API requests to optimize our code. It's much faster to load local data rather than making a request every single time. Especially if you don't need the API to refresh the data frequently.

[@julianunes2124]

Hey there, this video helped me a lot today. Thanks! :)

[@ricgondo]

Wow, I wish I knew this some time ago. Cool!

[@saifshaikh5559]

Why I am not able cached google maps api requests?

[@20SideDieTriva]

have yiu tried pythons requests_cache? it's very robust

[@Indently]

Not at all, thanks for the recommendation, I'll check it out!

[@silkogelman]

@@Indently requests_cache is really awesome.
You'll probably almost never apply the method in your video again as with requests_cache your caching system is easier to implement, maintain and has lots of ways to be configured. (and applied: valuable for testing and production environments)
I used to do it the way you show in your video too, but with requests_cache I never looked back.
It has great documentation, but there's no detailed video about it yet.
So there may be an opportunity for your UA-cam channel there.

[@pietart3596]

@@silkogelman I agree using requests libraries for production environment is very beneficial. For personal projects Indently's method is easier and more optimised. We aren't depending on an entire third party library

[@starogre]

what data wouldnt change from json exactly? maybe like news articles? or something that never gets updated? any other examples? movie names?

[@EnzoSantos]

Basically, something that changes once in a blue moon. For example, I was using the ytmusicapi in Python these days, which returns metadata about songs, albums, and playlists from UA-cam Music. I know it's very unlikely the "The Dark Side of The Moon" album to change their duration or song order between one request and another, so we can cache it. Another example is a Wikipedia page from a dead person or the script of a movie.

[@Sameer_Pattnaik]

It doesn't work from my side

[@borutouzumaki2120]

Hi this tutorial really help, but can we set the cache to be expired after 30minutes or an hours?

[@Indently]

Of course!

[@borutouzumaki2120]

Could uh please tell me how to do that? Because um still new in python

[@swagatopablo]

@@borutouzumaki2120 Not to rain on the parade here, and it is always commendable when people share educational stuffs to benefit the community. That said...
It is not a great design choice to cache on your disk, that too with plain text. That not only makes the server stateful, it can even be slower than the network (if you are making request from on-prem network). But regardless, the video explains the concept and implementation of caching quite well.
To answer your question directly, if you are to follow this, whenever you write the json, along with the relevant api response, you can add an 'expires' key to indicate when it will expire. Can set it 30 minutes (or any time) from the write timestamp, you can use the unix epoch as the simplest representation of a time instant here to serve the purpose. Look up python pendulum library for time operations. At read time, if the current epoch is higher than expires, then you should refresh the json.
But a far better choice is to use some service like Redis for caching. (You need do a bit of your research on how to use it.) You can also set up redis keys to self destroy after some time for cache eviction.

[@BroganMcShane-h1k]

The structure of your code could be improved, it doesn't read well

[@Indently]

Noted

[@florinbujoreanu1020]

I think he joked.. He can't be serious. @@Indently

[@disrael2101]

Any automation bootcamp/course coming up?