If you enjoyed this video don't forget to the like it and subscribe! Then check out this playlist about speeding up WordPress: ua-cam.com/video/cBDY-CXWmI0/v-deo.html
Every time I crawl my site with semrush, it keeps detecting errors that my some pages could not be crawled, pls does anyone have an idea how to fix that issue
For sure, here you go: Robots.txt Google Search Console link: support.google.com/webmasters/answer/6062598?hl=en Black Hole for Bad Bots tutorial: ua-cam.com/video/0KAzGDHq-1M/v-deo.html
Hi Sidhant, I don't at the moment. From my experience, you'll need to (1) have the development skills, which it sounds like you have and then (2) you have to go out and find clients. Part 2 is the hardest for most people. You have to go where your ideal clients meet up (conferences, meet ups, etc.) or you have to go door to door with an offer they can't refuse.
Thanks to your previous tutorial, I have installed IThemes Security and activated the "hide backend" feature. Will the robots.txt file provide additional site security that is not already installed?
Hi Frank, If you use the "hide backend" feature of iThemes Security you wouldn't need the user agent rules in the robots.txt file. The only thing that may benefit you is having the link to the sitemap in the robots.txt file. But you can also add your sitemap directly to the webmaster tools in major search engines instead. I hope that helps :) Let me know if you have any further questions. Thanks for watching!
Happy forms is pretty awesome and the free version is quite powerful. For free forms I also like Contact Form 7. I know it's old and a bit clunky, but you can build out lots of simple form types and there are plenty of addons for more functionality. If the form needs to have calculations done on data entered in the fields I like Gravity Forms. I like Recaptcha v3 and/or honeypot fields to reduce spam. ..I hope that helps, let me know if you have any further questions. Thanks for watching!
Great Bjorn! Really thankfull! My site got broken with the "500 Internal Server Error" after adding the "Really Simple SSL" Plugin to my Wordpress and Google Search Console is telling me that's because my server (at Google Cloud Platform) bloqued Google search engines (Googlebot) so they can not find my domain. Google Search Console though the robots.txt taster tool is prompting me to go back to the last usefull "robots.txt" downloading it to my computer and uploading it to my server. I can´t find any "robots.txt" file at my "html" directory; acording to your video I just have to upload the one downloaded from "robots.txt taster" tool, is this right? I really apreciate very much in advanced and hope you can help me. Best regards.
Hey so I have a large website and when I click on to coverage I have the discovered- but not indexed yet message also when I try to use the URL inspection it displays noindex. kindly help since only a few of my pages are being indexed
hello, only the homepage of my site comes up in google searches. None of the articles on my site appear in google searches. Do you have any idea about the reason? I registered the property on google search console, submitted the sitemap to google. I checked the robots.txt file but I can't seem to solve this problem in any way.
If you enjoyed this video don't forget to the like it and subscribe! Then check out this playlist about speeding up WordPress: ua-cam.com/video/cBDY-CXWmI0/v-deo.html
Hi, When I put my sitemap on just like you did it comes up with the red X on the left?
Every time I crawl my site with semrush, it keeps detecting errors that my some pages could not be crawled, pls does anyone have an idea how to fix that issue
Could you please add links to the Robert text tester and the blackhole Thanks in advance
For sure, here you go:
Robots.txt Google Search Console link: support.google.com/webmasters/answer/6062598?hl=en
Black Hole for Bad Bots tutorial: ua-cam.com/video/0KAzGDHq-1M/v-deo.html
Hi there, thanks for the video, but this seems not to be the best practice anymore (cf usage of meta tags)..any update of this video planned?
As always thank you so much for this tutorial
You're welcome Nissim, thanks for watching! Let me know if you have any questions :)
Do you have any video to how to make money as wordpress developer
Hi Sidhant,
I don't at the moment. From my experience, you'll need to (1) have the development skills, which it sounds like you have and then (2) you have to go out and find clients. Part 2 is the hardest for most people. You have to go where your ideal clients meet up (conferences, meet ups, etc.) or you have to go door to door with an offer they can't refuse.
Thanks to your previous tutorial, I have installed IThemes Security and activated the "hide backend" feature. Will the robots.txt file provide additional site security that is not already installed?
Hi Frank,
If you use the "hide backend" feature of iThemes Security you wouldn't need the user agent rules in the robots.txt file. The only thing that may benefit you is having the link to the sitemap in the robots.txt file. But you can also add your sitemap directly to the webmaster tools in major search engines instead.
I hope that helps :) Let me know if you have any further questions. Thanks for watching!
What is your favorite contact form and spam blocker for that form?
Happy forms is pretty awesome and the free version is quite powerful.
For free forms I also like Contact Form 7. I know it's old and a bit clunky, but you can build out lots of simple form types and there are plenty of addons for more functionality.
If the form needs to have calculations done on data entered in the fields I like Gravity Forms.
I like Recaptcha v3 and/or honeypot fields to reduce spam.
..I hope that helps, let me know if you have any further questions. Thanks for watching!
Great Bjorn! Really thankfull! My site got broken with the "500 Internal Server Error" after adding the "Really Simple SSL" Plugin to my Wordpress and Google Search Console is telling me that's because my server (at Google Cloud Platform) bloqued Google search engines (Googlebot) so they can not find my domain. Google Search Console though the robots.txt taster tool is prompting me to go back to the last usefull "robots.txt" downloading it to my computer and uploading it to my server. I can´t find any "robots.txt" file at my "html" directory; acording to your video I just have to upload the one downloaded from "robots.txt taster" tool, is this right? I really apreciate very much in advanced and hope you can help me. Best regards.
Thank you so much. I was having a lot of trouble trying to have ahrefs crawl my site and adsense. wonderful and clear tutorial.
You Rock !!
Thank you for your efforts and time in!!
Thanks, you rock too!
Good stuff :)
Thanks Paul and thanks for watching!
Congrats and thanx !
Thanks and thanks for watching!
Hey so I have a large website and when I click on to coverage I have the discovered- but not indexed yet message also when I try to use the URL inspection it displays noindex. kindly help since only a few of my pages are being indexed
Bloody brilliant mate! Cheers from Australia! 🎉
Hi! How do I edit the robot txt file that's blocking Google to crawl my pages?
Should be in the public part of the website
It's the robot.txt make your blog post show in search results
hello, only the homepage of my site comes up in google searches. None of the articles on my site appear in google searches. Do you have any idea about the reason? I registered the property on google search console, submitted the sitemap to google. I checked the robots.txt file but I can't seem to solve this problem in any way.
I have the same problem. I hope someone will answer
Thanks
How to make a sitemap in 2022 wordpress ?
Thanks so much!!!! 🙏🏿🙏🏿🙏🏿🙏🏿🙏🏿🙏🏿
Hi! Do you know what could be happening with my file robots.txt? Every so often it automatically switches to
user-agent: *
Disallow:
Mine too
Prolly something with your server from your hosting company
awesome thank you!
Thank you!!!!