Fetching 25K+ Rows of GSC Data using GSC API🚀 with Python🐍

After watching this video you will be able to access more than 25K rows of data from Google Search Console. Then you could build a data frame, and export the data to CSV or Excel. You will also learn how to user Dimensions and Dimension Filters in GSC API. 💯
✅GitHub Link: github.com/TheMihirNaik/googl...
✅W3 Schools Python: www.w3schools.com/python/defa...
✅Google Search Console API Overview: developers.google.com/webmast...
✅Google Search Console Reference: developers.google.com/webmast...
✅PyDoc reference for the Google Search Console API:googleapis.github.io/google-a...
✅Developer's guide for the Google API Client Library for Python:github.com/googleapis/google-...
Link to this Playlist : • Google Search Console ...
💡Timeline:
00:00 Thank you for the positive response and previous feedback.
00:28 Connecting on Social
02:10 Addressing the feedback
03:45 Project Page & Comment Section for GSC API x Python
05:38 Recap of Previous Video
💡Simplifying the previous code:
06:10 Starting of Explanation of 2nd Video
07:58 Consolidating & Simplifying the code from 1st Video
09:26 Shoutout to Rakesh Patel
10:30 Accessing GSC Data using the single API call
💡Writing While Loop for 25K+ rows of Data
12:26 Challenge of Accessing more than 25K+ rows of data
12:50 Python Operators, Conditions, String, List, Dictionary, For Loop, While Loop
15:26 Logic of While Loop to fetch 25K+ rows using multiple API calls.
💡Using Pandas & Building Dataframe:
25:24 How to Build a Dataframe using Pandas?
34:44 Export to CSV or Excel
💡Using Dimensions and Dimension Filters:
36:44 Using Dimensions
39:44 Using Dimensions Filters
Using Dimension Filters
43:54 Closing Remarks

[@Yooyolo518]

Awesome. Your video recording is really helpful. As a fish, it will be more helpful for fish if you can add some tips on operating buttons in your video.

[@hubertcombomarketing2693]

Thanks again Mihir

[@rabihbadr54]

Great job man. Thank you for the great content.

[@TheMihirNaik]

Glad you enjoy it!

[@wastelandia]

Amazing, thank you! I was suffering with this for quite some time =D

[@TheMihirNaik]

Glad you found it helpful.

[@manjugumpula]

Thank you so much for sharing.

[@TheMihirNaik]

You are so welcome!

[@oddocid9734]

Really helpfull 😄

[@TheMihirNaik]

Glad it was helpful!

[@Moonshine91]

Thank you so much for this great content! Can you please do a tutorial next to show how to we can push this into Adobe Analytics? i.e. import organic google search data to Adobe Analytics?

[@TheMihirNaik]

I don’t think you can import GSC data into Adobe Analytics. But also I dont know Adobe Analytics alot so likely I wont be doing it.

[@mudit2187]

First of all would like to commend you Mihir for such a brilliant video explaining the GSC API, I could automate a lot of stuff my team does following you. Just one suggestion the condition you put in the beginning startRow == 0 or startRow % 25000 == 0 are the same you can do with only startRow % 25000 == 0 because when startRow is Zero then the parameter 0 % 25000 will also evaluate to 0. Therefore two conditions are not required.

[@TheMihirNaik]

Hi Mudit, thanks for liking the content.
I completely agree with you. It's a great catch. When I published this code, I was very very early stage of my coding journey. Eventually I got little better structured my code better.
I haven't published a new version of the GSC code yet, but in the new one I have taken care of the same.

[@SaraswathiMoram]

Great video, Thank you for sharing with us. I have created file, but its not downloading.

[@moellerseo]

Keep up the good work my friend. How do you like to work with python? I have tried options like PythonAnywhere and running a linux system on my windows PC.

[@TheMihirNaik]

Thanks Marc!
For quick scripting I use Colab.
I am on Windows 10 and I have installed python locally as well. I use VS Code to write code in flask or fastapi framework. I use Github Desktop for version control and push/pull requests.
I use Heroku for deployment. I spend around $7 a month for heroku dynos. I tried PythonAnywhere but then I like Heroku better because it allows me to use Redis and bunch of other stuff.

[@SreevathsaBV]

@TheMihirNaik Thanks again for putting this together. Really insightful. Quick question, does the query limit of 25k applies to per query sent across to API or is it per property? Please share what the dead end of the query limit is.

[@TheMihirNaik]

25k rows limit is per query. It means the maximum rows of data Google will return per API call.
But there are other property level, account level and project level limits as well. I will have to check them. I will include it in one of the video with sampling.

[@SreevathsaBV]

@@TheMihirNaik thanks for the reply.

[@pragneshshrimal4788]

Thank you for the helpful video. Could you help me in extracting data that has been 'excluded by the 'noindex' tag'? Can you provide guidance on the appropriate filter and dimensions to use?

[@TheMihirNaik]

There is no way you can pull that data as of yet, but you can take all the URLs, get their inspections status, and then filter by the status you want.

[@sardarabdullahkhawar8487]

Should the email that is used to create Google Search Console Account and the email used to create GCP account be identical? or we can use separate email to create GCP account?

[@TheMihirNaik]

You could create GCP Account with a separate address.

[@waqashussain7770]

And how to pull this data in Bigquery ? Kindly help

[@TheMihirNaik]

I don’t really know BigQuery alot. But there are tutorials on that.

[@indianstreetbets_]

Can you share this collab with us?

[@TheMihirNaik]

Yes the github link is in the description.