How Google Search crawls pages

Discover the steps that go into how Google Search “crawls"" or finds and fetches web pages to display on Search, from URL discovery to fetching and rendering. Learn what Googlebot is, what it does, if Googlebot crawls every URL it finds, why sitemaps can be helpful for your website, and much more!
Chapters:
0:00 - Intro
0:49 - What is crawling?
1:01 - URL discovery
2:09 - What is Googlebot?
3:54 - What is fetching and rendering?
5:15 - Sitemaps (helpful website tools)
6:17 - Wrap up
Watch more How Search Works → goo.gle/HowSearchWorks
Subscribe to Google Search Central Channel → goo.gle/SearchCentral
#GoogleSearch #SEO
Speaker: Gary Illyes;
Products Mentioned: Search Relations;

[@GoogleSearchCentral]

🔔 Subscribe and turn on the notification bell so you don’t miss episode 3 on “Indexing”. → goo.gle/SearchCentral

[@THE-X-Force]

It grabs the code, but what about the images / videos?

[@brekise1974]

Movies

[@BrendaMalone]

Hey Gary, Blink twice if you need help. 🙃

[@SamuelLavoie]

"I do love JavaScript" 🙄
We know the feeling Gary 😅

[@onreact]

JavaScript was my first love.
Hence my name onReact.

[@almirathunstrom7216]

These series are HIGHLY underrated. They have helped me increase my visibility, and to understand that growth needs to happen organically. Thank you so much, and Gary's voice is so calming :). It's like Calm Space for learning!

[@shashankdubeylucknow]

This is awesome video to watch

[@juanjimenez8438]

Espero con ansias el proximo video, con tantos problemas que surgen al momento de indexar y para personas con conocimiento limitados podria ser de gran ayuda. Como en mi caso quizas algo este haciendo mal, ademas de que me ayudara a comprender mejor como funciona la indexacion y el paso a paso para una buena y optima indexacion de mi pagina. ❤

[@keuIe]

Client site rendered costs Google lot more action than a server rendered site. therefore it's a lie that you crawl these lashes as often as the other one.

[@CumeLegal]

What's the difference between site and server rendered ?

[@uzmazaki-rh2vc]

​@@CumeLegalm

[@honestmentoring]

Excellent, I love the visual animations to help understand such abstract process! Thank you!!!

[@peachfromtokyo]

Thank you for the good contents :D

[@thinkwithgiorgio]

Why did you not use card, end screen and link properly in the description?

[@DavidCancii]

Very good video

[@heshamabdo6024]

Thank You

[@LouStoriale]

This guy is much better.

[@onreact]

"This guy" has a name: Gary. The "other guy" called John is also fine.

[@freddym6678]

really , great information , i

[@yanggitak]

색인 과정도 많이 흥미로울 것 같습니다. 3주 동안 왜 색인이 안되었는지 고민하였고, 결국에는 해당 페이지의 고유한 콘텐츠가 부족해서, 혹은 메타 키워드가 동일하게 h1 태그에 있어서 발생했습니다. 구글 서치 콘솔에서는 '리다이렉션 오류', '사용자가 선택한 표준이 없는 중복 페이지' 라는 명칭으로 계속 떠서 정확한 이유를 몰랐지요...

[@ShaheeraRahmatee]

😢❤6:12

[@AutomateTheSheetOutOfIt-fu6ue]

I thought it was creeping around my website, I am glad it is not as creepy :D

[@ovorado-theplacetoexcel9530]

Great

[@NathanVeenstra]

Wait. Did Gary just appear in a video? 🤪

[@user-sg2sy4bn8r]

good

[@usaintltrade]

🥇

[@munip-atori6334]

🎉🎉

[@gilgameshsouth6464]

One thing I don't understand with URL discovery is how this hypothetical news page finds these unknown websites, given that they don't appear in the google search engine.

[@bellafong2328]

how to connect google search console with alibaba ？ Someone can help me?😢

[@padthaizilla9165]

Is sitemap still relevant?

[@rockersmedia1377]

Nowadays Google
Crawling, indexing, but not appearing in search results... after a bunch of algorithm update last year...A victim here...

[@NirmalaSanthakumar]

Yes, I too,
have the same issue. Two of my blog posts are crawling and indexing, but not visible on Google search results for the Targeted keywords

[@dinesh759]

@@NirmalaSanthakumar Focus on Long tail keywords. Competition is less.

[@NirmalaSanthakumar]

​@@dinesh759you are not getting my point! My blog posts disappeared from Google, but they are getting indexed and crawled

[@ShaheeraRahmatee]

5:50 ​@@NirmalaSanthakumar

[@antorsaha5296]

Ook🎉

[@gw8900]

Маша, почему комменты к видео не на русском? Их тебе тоже AI бот пишет? Покажи как это делается

[@onreact]

Thank you for stressing how important linking out is. Most people neglect that.

[@alive4536]

I don't even have a site sir.

[@Frank-uj6vr]

Search is dead in the AI age.

[@t-800o]

No

[@illuminatiZ]

Nothing new under the sun.