"However, some of the best geoguesser players can guess the location with only a single image and a couple seconds. This makes me wonder-" me: this shit makes me wonder if they could doxx me just taking a picture of a gas station near by-
watch GeoWizard's series "guessing the exact locations of my fans using one single photo" his fans send him a photo of themselves and he tries to guess the exact location. it sometimes take 10 minutes, sometimes some hours, but he usually always gets it right.. but yeah, their photos are rarely a random house or something, so it helps. but still !
doing smaller countries could be interesting considering how a lot of the areas would look the same-ish it would be interesting to see how good the ai guesses the areas
@@lederp42 Given enough flags in the training set it would definitely be able to. Flags are basically rectangles with pastel colors, super easy to recognize for a computer ;)
Since this video has been getting a lot of views suddenly I figured I'd leave an update. Firstly, thank you to everyone for the overwhelmingly positive support on this video, I never expected it to get this many views. I really enjoy reading all of the comments and seeing everyone's suggestions / improvements that could be made to this project. The only thing I've done with this project in the past year is that I expanded it to work on the entire world and played some battle royale games with it. The results were interesting, I just never got around to making a video for it. I've wanted to set aside time to make more videos, and seeing the support on this video is definitely motivation to do so. Thanks to everyone who has subscribed in spite of the fact that I haven't uploaded in almost a year, I'm hoping to make more videos soon. Also, yeah I know the mic sucks in this video. I have a new one which should hopefully work better in the future.
Have you considered placing the guess exclusively within the box the AI is most sure about, and then determining where you put the guess (still within that one box) based on the weighted average? Meaning the guess would always be in the most likely box. How do you think that would affect the results?
Guessing from the video results, I also had the impression that this would be more accurate. It seems that the unsure boxes are skewing the guess away from the predicted location. More so when the prediction is close to the edges of the country and there are a lot of unsure boxes to the other sides.
My thought exactly. As soon as he announced his guess averaging technique I was skeptical of that being a good choice due to the nature of coastal areas looking similar...but being on opposite ends of the map.
Love this AI, well done! I may have a possible improvement. Averaging all cells to formulate your guess moves the average guess towards the middle, always. Your guess distribution will become guassian like with the center of the distribution being the middle. Example: the goal is washington, any contribution from any of the other states pulls the guess towards the center. The only way to guess an state on the edge of america correctly is when that is the only one having a probability. This is why your guesses are generally to far towards the center, not because of the pictures you've taken. I suspect that using the center of the max cell for a smaller grid has a better performance ceiling (if you would want to continue). Other than that great work and an inspiring video!
I was about to comment the exact same thing. You can also see that the prediction sometimes is in a box with a lower confidence score, even though there exists a box more to the edges of the map with a higher score. I assume this is caused by the same bias towards the middle of the map.
Another way to solve this is to include in learning set border parts of Canada and Mexico. When output is in the wrong country it will point to closest location in US, but probably it won't happen as you point out that it's always moving towards center. Problem is with states near ocean tho.
On top of that, this averaging also assumes that the distribution is a single gaussian. It's possible that an image depicts some part of the East Coast, but its features is similar to a region on the West Coast. The distribution is then disjoint, with the network predicting a high probability on both the East Coast and the West Coast, but averaging this distribution will incorrectly produce a guess at the center of the US.
@@zenith5180 well I think this argument is not as strong as one might first think, and I want to explain why: In such a case you suggest a good gest would be either east or west coast getting about a 50% chance to have either a good or a desasterous guess. The AI's guess would be bad in both cases but not desasterous, do you still have a 50% chance being better or worse than the AI's guess. Probably your strategy still is the better, but that is due to the reward function of Geoguessr, which I don't know anything about but probably decreases the reward points exponentially with distance from the target or sth. similar. Now imagine a different case, where you have this becomes even more trickier: you have 3 fields, each have the same likeliness of ⅓. These fields are disjoined and form a equilateral triangle. If you go full risk for one of these fields while the AI guesses for the center of the triangle, the AI has a better guess distance wise in ⅔ of the cases. What this means for the expected value of reward points depends even more on how Geoguessr's internal distance2target->points function. Therefore, I think the general idea of using not just the field with the highest likeliness but something, let's call it 'map awareness' is not wrong. However, a simple averaging is not the best idea, as explained in the previous comments. Personally, I think based on the AI's predicitions one could compute a final regularizes guess coordinate as the guess in a little bit more complicated way: one cross-relates the probability of a field with each of the other fields by considering also the distance between those fields. For example you have field A with a high likelihood and directly east of it field B, also with a high likelihood. This would be a clear cluster, and since the distance between the both fields is minimal the two high likelinesses would push each fields overall likeliness. If field A and B are from the east and west coast, the great distance between them would prevent, that they buff each others likelihood. And if field B has a not so high likelihood, it would buff the likelihood of field A just very weakly. Ofc, the likelihood values are not adding up to 1 anymore, that's why I'm not using the term probability here. However, the higher this value would be, the more likely is this region a good guess. I think that way, one would make improved guesses. Also, one can experiment if these values, likelihood and distance between different fields, should enter the problem just linearly or results improve, if the predicitions depend on these values in a squared/etc. fashion.
Your newly 6th sub here. I'd love to see a new model perform on Geoguessr's earth map :D Btw how did you collect your image data from Google's streetview? What tool did you use? Also do you recommend any other books/courses on CNNs? Keep it up man!
Thanks for the sub! I collected all of the images using Google's Street View Static API developers.google.com/maps/documentation/streetview/overview and collected images from random latitudes and longitudes within the US. The resource I linked in the description developers.google.com/machine-learning/practica/image-classification is probably the best resource I found for understanding the fundamentals of CNNs and feature extraction. Its exercises are also pretty useful for seeing how to implement CNNs using TensorFlow. For this project in particular, the Keras Applications documentation keras.io/api/applications was useful for finding and using a pre-trained model for feature extraction.
Since Google's Street View pictures are taken in a road, an improvement would be done if the prediction were moved to the nearest road in the direction of the highest confidence rectangle:)
@@mst8819 The AI will not learn to do so given enough time, in it's current setting anyway. It outputs confidence values for each of the boxes. The output of each box is just that, it won't be able to adjust towards roads just like in the animal example it will not suddenly output what part of an animal is in the picture, e.g. "ear of fox", unless you trained it that way in the first place
For world mode, trying to perform an "average" over the international dateline may prove to be difficult. But it would be so cool to see it done anyway! perhaps choose the center of the max probability box? is that better or worse? if you choose to go further and get more accurate, there is the problem of training data. If you can somehow make fine enough grids, you might be able to produce more accurate results, assuming more data also, if you take multiple photos of the same area, you could possibly get a better idea of the place - like take a few photos, then get the average of the outputs or something. i love it !!!
There is some mathematics ionvolved in picking the avertage of a set of points on a sphere. In particular, it is not necessarily unique. Think for example of the mean of the north and south pole, every point at the equator is equally reasonable as a mean, anyway the average of two opposite points does not make that much sense. A way around this problem would be first picking the continent which is most likely and then taking the average over this landmass, islands would still be a problem. So a better solution is to just take a region around the box with the highest probability and consider only boxes lying in that region. Also the shape of the boxes is not square anymore, so we should multiply the weight by the size of the box. The problem of training data can be adressed by automatization. Google street view should provide a suitable interface that can be used to automatically choose a random spot and generate some pictures. Using multiple pictures would have a great potential for improvement.
That's difficult, because what we train here is an image classifier and therefore we need predefined classes (our boxes) before (!) we start training. Since we want to use the same AI for each location we need the same boxes everywhere.
@@jakobm.4183 for a sphere, you could take the coordinates of a sphere, then turn them into a vector from the center of earth and average that - but since not all grid spaces are going to be used (for example, oceans), then there's going to be an implicit bias. Your method is likely better.
im sorry if this is dumb but couldnt you just use a slightly incorrect mapping? there would be bias towards some areas but you could just use a flat map of the earth then relate that point to the same point on the sphere
The problem with the square method is that, for example, the square that contains death valley also contains a number of ski lodges that are in a completely different climate
Some suggestions: 1. About pushing to the center, you can tweak weighted average algorithm giving it weight in power (eg instead of 0.5, give it 0.5^2=0.25). It will keep resulted dot closer to darker areas 2. Write script that will automatically parse geogessr page, predict location and answer using script. This will make everything easier for you 3. Give AI whole 360 image instead of small part of it, it will give it more data that AI can work on 4. Play with size of boxes, it is possible that smaller squares could get better results And thanks for the video, it was interesting to watch!
Such a fascinating video, it would be really cool to see the updated version in action taking on the whole world! Take as much time as you need with your next video, me and many others are looking forwards to it 😅
Oh wow, this was super interesting! I think the AI could definitely give GeoWizard a run for his money at his own no moving allowed 10-state challenge format. Maybe reach out to Tom and see if he's up for a head-to-head game?
From what I'm seeing, the AI's most confident guess on where the location is is quite accurate, opposed to the weigthed average calculation of the AI's guesses that you are doing.
Hahaha. The "official" United States map on Geoguessr is notoriously bad and those are the usual locations you get in the middle of nowhere. The "United States (no blurry)" map is so much better.
This is really cool. You could make the AI much more accurate by somehow plugging it into GeoGuesser, and letting it navigate the environment until it found an image/view which it could match at a very high confidence. I'm not sure how easy that would be unless GeoGuesser has an API, though .
It's pretty cool to see a video about some really complex AI. it's even cooler when the Geoguesser chooses a spot right in front of your old house at 12:43 and you can see what your house looked like 8 years ago. Those trees outside are long gone now. Very cool!
I think you can make the boxes smaller, use more training images. Then take 3 screenshots per location. Make AI predict per screenshot, and use the 3 predictions to triangulate.
it seems to me like using a weighted average of the various patches is maybe making a mistake? for an example of the sort of mistake that would encourage, imagine an image that is definitely inside a city, but you're not sure which city among a few candidates. averaging the longitude and latitude of each city quite possibly gives you some location in the middle of a field far away from anything urban. depending on the geogueser scoring system this is maybe still a score optimizing strategy, going for consistent but low score over a high risk but very high score if you actually get the city right? but it's just a question you've got me chewing on a bit, now, i suppose.
I had a similar thought, but for a different reason. Using averaging will negatively impact locations near the borders, since it can only average it away from the border. Would be interested to see how a "naive" AI (center of most probable square) would compare. Still a good video though!
I think the same. Average can't be used with terrain. Pictures of the coast would average to the middle of the country. Sides of a mountainous region would average to the mountainous region.
Using weighted average does make it statistically better than the average, though. Seeing as the lowest score possible would be to choose a point on the coast when the correct location is on the opposite coast, this possibility is eliminated when the AI will only choose a point closer to the center. Therefore, even when the AI is wrong, the way the point is calculated will still give it a better score on average. In conclusion, using a weighted average is a msitake because it's sorta like cheating, and makes it hard to see if the AI is actually doing well.
Very interesting conversation here! I think you all bring up reasonable points. Which method is preferred also depends on how the score in geoguessr is calculated. Take the opposite coast example: if the distance from the middle to one coast is too far to let geoguessr give you any points, you might as well make a guess between one of the two coasts and have a 50% chance for a lot it points, instead of 0% chance at any points when taking the average. What I keep thinking of is that fact that this question should actually be optimized *by the AI itself*. If you could train it based on the actual score it would get in a game, that would be awesome. Not sure how that would work technically, but from a theoretical view it would be really interesting!
The fact you only chose very rural areas with almost no defining features is very tough, but cresting a more advance AI it may be able to learn building styles, road signs and street names for urban areas
The fact that it didn’t even have access to the 360 images is impressive too, an AI that can accurately predict a location based on a still image is kinda spooky ngl
Soooooo. Your AI guessed pretty much exactly where I used to live on the one where it landed in Idaho. The on-screen coordinates showing on the video for this round were still for the previous round. However, I used to live in Idaho and even from the short glimpse on the map, I thought to myself... REXBURG!!! Then I noticed the Texas Slough on the map next to the AI waypoint and I knew, without a shadow of a doubt, this is very close to my hometown (South Madison Country, Rexburg, ID). I zeroed in on your coords with similar methods that you described in the video and landed on the coordinates of 43.727127, -111.829871. I then used that to calculate the distance to my original home address. The AI "thought" you were within 5 miles of where I used to live! What's even crazier, is that I moved to WA State which just happens to be within about 200 miles of where the ACTUAL location is. Trippy huh?
This is like one of the only videos I can find on someone creating an AI for GeoGuessr. I've had this similar idea ever since I first discovered GeoGuessr a few years back. Very cool stuff! Now I'm tempted to try doing this.
Great video! I think it would be interesting to retry this with a different output. You are doing multi-class classification and then weighting the outputs. Rather than making it a classification problem, you could make it a regression problem and directly predict latitude and longitude. Then make the loss the sum of the mean squared errors over both the lattitude and longitude. That would remove the “centralizing” bias you’re getting from weighting.
Cool vid! I think your square weighting method may be slightly flawed, because it means that the AI is very unlikely to guess correctly when the actual location is on the edge of the country. This is because the average of all squares without weighting would give a dot in the centre of the US, naturally drifting all guesses to the centre of the country.
It's not about the edge vs the center, it's in general that it cannot sensibly represent ideas such as Florida and California being likely and Texas unlikely. It assumes a uni-model distribution of confidence over the map.
I find it especially impressive considering the AI only has a single image to base its guess off of, while a human player can scroll along the road and turn the camera around
Great job my dude! I thought it was close to impossible to code an AI for GeoGuessr and yet you did it in a surprisingly straightforward fashion. I respect people who code AI and Neural Network stuff since I couldn't do that myself...
Really cool. You could create one for the new battle royale mode where you just need to guess the correct country. Could use road markings and road signs for the training data.
Really impressive stuff! Perhaps could you share more about your code? Like how long it took you, some snippets, and even share it! Loved the vid! Congrats on the great AI performance!
Yess, finally im early to see a underrated channel that is probably gonna grow so fast before it has 10k subs. Todays date is 03/08/2021, time is 16:37 and this channel 6,37k subscribers. This video has 416k wiews and the channel has only 2 videos. Other video has 95k wiews. im sure this channel is gonna earn at least 140k subs in the next 2 weeks. if he keeps uploading videos, this channel probably would be a lot bigger. Love your videos! Please keep uploading :)
Nice simplification of the classification task! I think a simple improvement could be, that the final prediction must always be inside the highest probability cell, but the precise position is moved by the weighted average.
I personally feel like the weight for the highest confidence square should have been higher for best results, maybe the conf level itself as a weight for all (or an equivalent system the result falls inside the top confidence squares)
Ideas for improvement (maybe they are feasible): Implementing periodic boundary conditions for the average confidence of the position. Using the whole 360° Image instead of the usage of the image selection. I just found your channel. Interesting videos, keep it up!
Really interesting - have another new sub! A few random thoughts that occur to me on this. "Extremising" the probability outputs might help improve the fit on the edges of the map in particular. For example, square the probabilities (then normalise) to skew the prediction towards higher confidence levels. For an all world system you'll probably have to drop or at least modify the averaging approach, as others have said. Maybe it picks the continent first? How did you choose the size of your boxes? Obviously smaller boxes would allow a finer prediction but it's more work to train. Given that human players can move around, perhaps you could give the AI more than 1 photo to base its prediction on.
I think taking the second power of numbers to determine the location would make this more accurate because it would make an ai more confident when he gets a higher percentage of chance for it to be in a specific state.
This is great. I want to see the ai play a diverse world map, country streaks, and if there is there a way to make the ai more accurate. Thanks for the nice video!
Super cool project! I can't wait to see what you do with it next, should you decide to continue. Once you get the average bias problem sorted out, it looks like you'll have a really powerful tool. With more training, the program could be used for so many applications. Well done!
Loved the video and really surprised by how well the AI did with so little information! Judging by the comments your video seems to be gaining some traction, that's well deserved ! Hoping more people see this !
I like how UA-camrs makes AI using the simplest method but getting good results while scientist and researchers using complex methods and yet not satisfied on their result
I am a bit interested in seeing a GitHub repo just so I can fill it with suggestions! Some ideas: * make the input an entire 360 degree strip. Something can be done to manipulate that street view I think. * make the output an greyscale image with bright dots corresponding directly to geo coordinates instead of a bunch of brackets. Or look to the object detection networks for other ways to encode the guessed location/bounding box
Hey as an avid Geoguessr fan and a Data Science student I loved this vid! I always joke to my friends about building an algo that could play for us but I never put any actual thought into it, but here you are haha. Although is there any way I could see your code; as a more in-depth and specific analysis of development would be really fun and interesting. Also, I'm sure it's been commented but using this classifier on country streak would be a lot of fun. Maybe that is something you could attempt.
May have been interesting to run the same photos again but with you choosing a different view/ direction and see if the AI came back with similar predictions. You choosing a view from each location may have represented a biased input to the AI. Great video and concept, clever bastard 😎
I think a critical note here is that the comparison to the pro's is not entirely fair. Your AI is trying to optomise score across all games, where as a pro knows to take risks and only keep the 1 lucky group of guesses. It would be cool to see a best attempt of a group of tests where the AI goes with its highest confidence box being much more heavily weighted.
If you were to do a world map version, it'd be interesting to have a tiered approach with the first AI identifying the general region, and then have regional AIs that specialize in a given area
you should try this with the entire world. i reckon the results will be better as the margin for error is slightly bigger. great video though i'll turn my notifications on for your next one/
I think that a good idea to help optimize the results would be to add major landmarks. Like add recognizable mountain structures or make sure that coasts are always on the coast. Also, maybe being able to cut out the sky so that the AI only bases its guesses off of the land, instead of the particular weather in the area at the time you trained it. Those are my suggestions, can't wait to see a new vid
Cool idea, seems rather good for simply plopping a standard image classifier into the problem. Getting a more precise position by weighted average seems a bit questionable, but it might work (i.e. improve results slightly) with some filtering. For example you might get two disjoint areas which are likely, say you can see the sea so you believe you are either on the east or west coast, but your prediction ends up right in the middle. Being right half of the time is likely going to give higher scores here, so just picking the center of the highest weighted square might actually be better. You should make some evaluation code to check automatically how well it performs, so you can try out different strategies and see what works best. Then a great next step would be to make it predict on several images and try to improve the prediction from that. You can use your existing model for this. If you can remove the human element of you picking an image that looks useful, that makes it a lot cooler. (Make a small macro that takes a screenshot, rotates the view by 90 deg, takes another screenshot, etc. for that extra A.I. feel.)
If you decide to do this in the world map, my suggestion would be to train the AI with pictures that are pointed to the street, since road markings, signs, language etc etc differ for almost all countries.
You honestly short changed the AI. Human players get waay more image data. You should have taken 4 images at each location and seen what it would come up with then. This is seriously impressive work.
This is really really cool, personally, think it would be funny to see if the AI can be trained to be able to read signage and pinpoint where it is from given distances to other cities. I'm a bit biased, but the Netherlands is a good country to do such a thing with, as there are signs indicating directions and distances nearly EVERYWHERE, mostly because these signs can also be found on many cycling routes.
Here's my advice. Instead of taking the average of the centers of every reddish square, you should only consider the center of the best square. I've noticed that distant squares, even if it is not very confident in them, bring lots of disturbance into your average calculation. By only considering the best one, you will avoid this problem and get closer results in my opinion.
Comment for the algorithm great job! Actually maybe you could double the weight of the boxes at the edge of the map as its very good at getting central boxes but slightly struggles at getting coastal boxes? Great work though!
Could you perhaps design code that uses video as input? Like, record the geoguessr screen and see the AI get progressively more confident as you look around. Amazing video I'm always amazed by the capabilities of neural learning.
That was a fun video! I thought there was no way for the AI to get anywhere close, but those scores are impressive. Maybe you could try different AI learning methods and compare them to see which is more accurate?
Chances are very real it's using meta strategies. There's entire websites on how you can narrow down where you are in geogeussr based on some characteristics of the footage such as errors in the stitching of the 3D image or dirt on the lens etc. Entire areas are covered by the same car/team so it could be possible that the AI has learned to recognize those artifacts as patches. The AI can't differentiate between picture artifacts and the rest of the photo, so it might very well just be recognizing that, for example, every street view image in a certain state or county has a certain scratch on the lens or whatever
Now make the AI say "Get in!" after an insane guess
hahaha yes!
Make another ai to confirm every 10 mins that he is recording
@@ryleypalmer Manke another AI that narrates from "the future" if the real AI isn't recording
LMAOO
Lmao this made my day
I think this would be interesting to see on country streak mode, since the guessing method in that mode is similar to how the AI works
Where do I know you from? Your name seems familiar
@@itzspiro6047 Do you watch Reid Captain?
@@NStripleseven yes sometimes
@@itzspiro6047 yea roaxial is a part of that community
@@NStripleseven I want to be a part of that community just because it looks gregarious, u guys know each other!
"However, some of the best geoguesser players can guess the location with only a single image and a couple seconds. This makes me wonder-" me: this shit makes me wonder if they could doxx me just taking a picture of a gas station near by-
watch GeoWizard's series "guessing the exact locations of my fans using one single photo"
his fans send him a photo of themselves and he tries to guess the exact location. it sometimes take 10 minutes, sometimes some hours, but he usually always gets it right..
but yeah, their photos are rarely a random house or something, so it helps. but still !
Yep, most likely, it isn't actually that hard to do but requires some work.
We can. I've found people by the view from their window.
@@Standplaats_Krakow It depends though. Not all pictures are foundable.
@@imutig3451 True, but there's often way more info than people think.
doing smaller countries could be interesting considering how a lot of the areas would look the same-ish it would be interesting to see how good the ai guesses the areas
Ireland
Vatican City
malta
Tunisia
@@lederp42 Given enough flags in the training set it would definitely be able to. Flags are basically rectangles with pastel colors, super easy to recognize for a computer ;)
UA-cam algorithm chose you, congrats!
I smell some nepotism.
@@Spoof10 what????? lmaooo do you know what nepotism means??? you think this guy has a friend at youtube who gave him views???
@@samcousins3204 You enjoy your time on the ground, while the joke is cruising at 10000 ft above you.
@@Spoof10 or the more virgin way, R/Whooosh
@@Spoof10 wooosh
Since this video has been getting a lot of views suddenly I figured I'd leave an update.
Firstly, thank you to everyone for the overwhelmingly positive support on this video, I never expected it to get this many views. I really enjoy reading all of the comments and seeing everyone's suggestions / improvements that could be made to this project. The only thing I've done with this project in the past year is that I expanded it to work on the entire world and played some battle royale games with it. The results were interesting, I just never got around to making a video for it. I've wanted to set aside time to make more videos, and seeing the support on this video is definitely motivation to do so. Thanks to everyone who has subscribed in spite of the fact that I haven't uploaded in almost a year, I'm hoping to make more videos soon.
Also, yeah I know the mic sucks in this video. I have a new one which should hopefully work better in the future.
First
Any chance you'll make your code available? I assume this is a tensorflow build?
Is there a tutorial for how you did this?, i'd love to do it but with a worldwide map but am unsure how I would get started with this
i personally would love to see a continuation of this video!!!!
@@austeyen5628 YES PLEASE
I love how this has just been shoved into the algorithm after nearly a year after publication
This vid deserves much more attention. Good job on this.
Have you considered placing the guess exclusively within the box the AI is most sure about, and then determining where you put the guess (still within that one box) based on the weighted average? Meaning the guess would always be in the most likely box. How do you think that would affect the results?
Guessing from the video results, I also had the impression that this would be more accurate. It seems that the unsure boxes are skewing the guess away from the predicted location. More so when the prediction is close to the edges of the country and there are a lot of unsure boxes to the other sides.
I'm also quite surprised that he didn't at least use a squared average or something
My thought exactly. As soon as he announced his guess averaging technique I was skeptical of that being a good choice due to the nature of coastal areas looking similar...but being on opposite ends of the map.
Love this AI, well done! I may have a possible improvement. Averaging all cells to formulate your guess moves the average guess towards the middle, always. Your guess distribution will become guassian like with the center of the distribution being the middle. Example: the goal is washington, any contribution from any of the other states pulls the guess towards the center. The only way to guess an state on the edge of america correctly is when that is the only one having a probability. This is why your guesses are generally to far towards the center, not because of the pictures you've taken. I suspect that using the center of the max cell for a smaller grid has a better performance ceiling (if you would want to continue). Other than that great work and an inspiring video!
I was about to comment the exact same thing. You can also see that the prediction sometimes is in a box with a lower confidence score, even though there exists a box more to the edges of the map with a higher score. I assume this is caused by the same bias towards the middle of the map.
Another way to solve this is to include in learning set border parts of Canada and Mexico. When output is in the wrong country it will point to closest location in US, but probably it won't happen as you point out that it's always moving towards center. Problem is with states near ocean tho.
On top of that, this averaging also assumes that the distribution is a single gaussian. It's possible that an image depicts some part of the East Coast, but its features is similar to a region on the West Coast. The distribution is then disjoint, with the network predicting a high probability on both the East Coast and the West Coast, but averaging this distribution will incorrectly produce a guess at the center of the US.
@@zenith5180 well I think this argument is not as strong as one might first think, and I want to explain why:
In such a case you suggest a good gest would be either east or west coast getting about a 50% chance to have either a good or a desasterous guess. The AI's guess would be bad in both cases but not desasterous, do you still have a 50% chance being better or worse than the AI's guess.
Probably your strategy still is the better, but that is due to the reward function of Geoguessr, which I don't know anything about but probably decreases the reward points exponentially with distance from the target or sth. similar.
Now imagine a different case, where you have this becomes even more trickier: you have 3 fields, each have the same likeliness of ⅓. These fields are disjoined and form a equilateral triangle. If you go full risk for one of these fields while the AI guesses for the center of the triangle, the AI has a better guess distance wise in ⅔ of the cases. What this means for the expected value of reward points depends even more on how Geoguessr's internal distance2target->points function.
Therefore, I think the general idea of using not just the field with the highest likeliness but something, let's call it 'map awareness' is not wrong. However, a simple averaging is not the best idea, as explained in the previous comments.
Personally, I think based on the AI's predicitions one could compute a final regularizes guess coordinate as the guess in a little bit more complicated way: one cross-relates the probability of a field with each of the other fields by considering also the distance between those fields. For example you have field A with a high likelihood and directly east of it field B, also with a high likelihood. This would be a clear cluster, and since the distance between the both fields is minimal the two high likelinesses would push each fields overall likeliness. If field A and B are from the east and west coast, the great distance between them would prevent, that they buff each others likelihood. And if field B has a not so high likelihood, it would buff the likelihood of field A just very weakly. Ofc, the likelihood values are not adding up to 1 anymore, that's why I'm not using the term probability here. However, the higher this value would be, the more likely is this region a good guess.
I think that way, one would make improved guesses. Also, one can experiment if these values, likelihood and distance between different fields, should enter the problem just linearly or results improve, if the predicitions depend on these values in a squared/etc. fashion.
+
for the 7:30 one you should've used more of that mountain, probablly would result in a higher cofidence
Your newly 6th sub here. I'd love to see a new model perform on Geoguessr's earth map :D
Btw how did you collect your image data from Google's streetview? What tool did you use?
Also do you recommend any other books/courses on CNNs? Keep it up man!
Thanks for the sub!
I collected all of the images using Google's Street View Static API developers.google.com/maps/documentation/streetview/overview and collected images from random latitudes and longitudes within the US.
The resource I linked in the description developers.google.com/machine-learning/practica/image-classification is probably the best resource I found for understanding the fundamentals of CNNs and feature extraction. Its exercises are also pretty useful for seeing how to implement CNNs using TensorFlow. For this project in particular, the Keras Applications documentation keras.io/api/applications was useful for finding and using a pre-trained model for feature extraction.
Now this is wholesome
@@adumb_codes Thank you for sharing!
Since Google's Street View pictures are taken in a road, an improvement would be done if the prediction were moved to the nearest road in the direction of the highest confidence rectangle:)
I think with more training data the AI should figure that out sometime
@@mst8819 The AI will not learn to do so given enough time, in it's current setting anyway. It outputs confidence values for each of the boxes. The output of each box is just that, it won't be able to adjust towards roads just like in the animal example it will not suddenly output what part of an animal is in the picture, e.g. "ear of fox", unless you trained it that way in the first place
Yo dude, really pog video, Ignoring the mic, I would've thought it was from a 100k+ subs youtuber
Me too, then I saw he had under 100 subs
Just need to scream nonsense while failing an easy video game: 1M subs.
For world mode, trying to perform an "average" over the international dateline may prove to be difficult.
But it would be so cool to see it done anyway!
perhaps choose the center of the max probability box? is that better or worse?
if you choose to go further and get more accurate, there is the problem of training data.
If you can somehow make fine enough grids, you might be able to produce more accurate results, assuming more data
also, if you take multiple photos of the same area, you could possibly get a better idea of the place - like take a few photos, then get the average of the outputs or something.
i love it !!!
There is some mathematics ionvolved in picking the avertage of a set of points on a sphere. In particular, it is not necessarily unique. Think for example of the mean of the north and south pole, every point at the equator is equally reasonable as a mean, anyway the average of two opposite points does not make that much sense. A way around this problem would be first picking the continent which is most likely and then taking the average over this landmass, islands would still be a problem. So a better solution is to just take a region around the box with the highest probability and consider only boxes lying in that region. Also the shape of the boxes is not square anymore, so we should multiply the weight by the size of the box.
The problem of training data can be adressed by automatization. Google street view should provide a suitable interface that can be used to automatically choose a random spot and generate some pictures. Using multiple pictures would have a great potential for improvement.
maybe some kind of repeated grid patterns in finer detail with the center of the most confident box would work well
That's difficult, because what we train here is an image classifier and therefore we need predefined classes (our boxes) before (!) we start training. Since we want to use the same AI for each location we need the same boxes everywhere.
@@jakobm.4183 for a sphere, you could take the coordinates of a sphere, then turn them into a vector from the center of earth and average that - but since not all grid spaces are going to be used (for example, oceans), then there's going to be an implicit bias. Your method is likely better.
im sorry if this is dumb but couldnt you just use a slightly incorrect mapping? there would be bias towards some areas but you could just use a flat map of the earth then relate that point to the same point on the sphere
The problem with the square method is that, for example, the square that contains death valley also contains a number of ski lodges that are in a completely different climate
don't worry those re-captchas will now never be click the sign questions. thank you adumb
the hero we didnt deserve
Unironically though: those recaptchas actually have grain added to them to confuse the AI, its some very specific grain too.
@@RubyPiec Good, good... as long as the humans are ahead of the AI then I am all for back to captchas.
Some suggestions:
1. About pushing to the center, you can tweak weighted average algorithm giving it weight in power (eg instead of 0.5, give it 0.5^2=0.25). It will keep resulted dot closer to darker areas
2. Write script that will automatically parse geogessr page, predict location and answer using script. This will make everything easier for you
3. Give AI whole 360 image instead of small part of it, it will give it more data that AI can work on
4. Play with size of boxes, it is possible that smaller squares could get better results
And thanks for the video, it was interesting to watch!
Such a fascinating video, it would be really cool to see the updated version in action taking on the whole world! Take as much time as you need with your next video, me and many others are looking forwards to it 😅
Your video might just blow up. I randomly got it in my feed today, and I'm sure I'm not the only one.
yeah I predict it'll get 100k - 1m views, give it a few weeks or days
Oh wow, this was super interesting! I think the AI could definitely give GeoWizard a run for his money at his own no moving allowed 10-state challenge format. Maybe reach out to Tom and see if he's up for a head-to-head game?
this is the most perfect video I will probably see this year
From what I'm seeing, the AI's most confident guess on where the location is is quite accurate, opposed to the weigthed average calculation of the AI's guesses that you are doing.
So what I've learned is that 90% of America is dry, flat terrain with a road.
lmao
Similar to Poland with less greenery and more occasional mountains
Hahaha. The "official" United States map on Geoguessr is notoriously bad and those are the usual locations you get in the middle of nowhere. The "United States (no blurry)" map is so much better.
This seems really cool. You could do it again with more training data and maybe smaller squares.
This is really cool. You could make the AI much more accurate by somehow plugging it into GeoGuesser, and letting it navigate the environment until it found an image/view which it could match at a very high confidence. I'm not sure how easy that would be unless GeoGuesser has an API, though .
I'm guessing they don't have an API that can do it for that exact reason lol. You could use google maps API but afaik that costs money.
Just feed the ai like 50 images where ur navigating the environment, then u can create a weighted average of it's guesses for them.
@@matthewbadger8685 Well yeah but how many pictures would you need for that ._.
@@prav2568 50
@@chippyconqueror bruh i Don't know about Geosquare but as I assume it includes a whole lot of area , you would need more than that
I wonder if you could train a model to take CLIP embeddings and predict lat+lon from that
Your final challenge: Create a geoguessr world bot that gets it down to the street more than 50% of the time.
Can you make deep dreams for each of the states? It would be cool to see how the network thinks each state looks.
It's pretty cool to see a video about some really complex AI. it's even cooler when the Geoguesser chooses a spot right in front of your old house at 12:43 and you can see what your house looked like 8 years ago. Those trees outside are long gone now. Very cool!
I think you can make the boxes smaller, use more training images. Then take 3 screenshots per location. Make AI predict per screenshot, and use the 3 predictions to triangulate.
it seems to me like using a weighted average of the various patches is maybe making a mistake? for an example of the sort of mistake that would encourage, imagine an image that is definitely inside a city, but you're not sure which city among a few candidates. averaging the longitude and latitude of each city quite possibly gives you some location in the middle of a field far away from anything urban. depending on the geogueser scoring system this is maybe still a score optimizing strategy, going for consistent but low score over a high risk but very high score if you actually get the city right? but it's just a question you've got me chewing on a bit, now, i suppose.
I had a similar thought, but for a different reason. Using averaging will negatively impact locations near the borders, since it can only average it away from the border. Would be interested to see how a "naive" AI (center of most probable square) would compare. Still a good video though!
I think the same. Average can't be used with terrain. Pictures of the coast would average to the middle of the country. Sides of a mountainous region would average to the mountainous region.
Using weighted average does make it statistically better than the average, though. Seeing as the lowest score possible would be to choose a point on the coast when the correct location is on the opposite coast, this possibility is eliminated when the AI will only choose a point closer to the center. Therefore, even when the AI is wrong, the way the point is calculated will still give it a better score on average.
In conclusion, using a weighted average is a msitake because it's sorta like cheating, and makes it hard to see if the AI is actually doing well.
And won't using this average method having a result always in the 8 middle squares ?
Very interesting conversation here! I think you all bring up reasonable points. Which method is preferred also depends on how the score in geoguessr is calculated.
Take the opposite coast example: if the distance from the middle to one coast is too far to let geoguessr give you any points, you might as well make a guess between one of the two coasts and have a 50% chance for a lot it points, instead of 0% chance at any points when taking the average.
What I keep thinking of is that fact that this question should actually be optimized *by the AI itself*. If you could train it based on the actual score it would get in a game, that would be awesome. Not sure how that would work technically, but from a theoretical view it would be really interesting!
Wow, this is a really well made video! You seriously need more attention, this is awesome!
The fact you only chose very rural areas with almost no defining features is very tough, but cresting a more advance AI it may be able to learn building styles, road signs and street names for urban areas
The fact that it didn’t even have access to the 360 images is impressive too, an AI that can accurately predict a location based on a still image is kinda spooky ngl
Soooooo. Your AI guessed pretty much exactly where I used to live on the one where it landed in Idaho. The on-screen coordinates showing on the video for this round were still for the previous round. However, I used to live in Idaho and even from the short glimpse on the map, I thought to myself... REXBURG!!! Then I noticed the Texas Slough on the map next to the AI waypoint and I knew, without a shadow of a doubt, this is very close to my hometown (South Madison Country, Rexburg, ID). I zeroed in on your coords with similar methods that you described in the video and landed on the coordinates of 43.727127, -111.829871. I then used that to calculate the distance to my original home address. The AI "thought" you were within 5 miles of where I used to live! What's even crazier, is that I moved to WA State which just happens to be within about 200 miles of where the ACTUAL location is. Trippy huh?
This is like one of the only videos I can find on someone creating an AI for GeoGuessr. I've had this similar idea ever since I first discovered GeoGuessr a few years back. Very cool stuff! Now I'm tempted to try doing this.
Great video! I think it would be interesting to retry this with a different output. You are doing multi-class classification and then weighting the outputs. Rather than making it a classification problem, you could make it a regression problem and directly predict latitude and longitude. Then make the loss the sum of the mean squared errors over both the lattitude and longitude. That would remove the “centralizing” bias you’re getting from weighting.
Cool vid! I think your square weighting method may be slightly flawed, because it means that the AI is very unlikely to guess correctly when the actual location is on the edge of the country. This is because the average of all squares without weighting would give a dot in the centre of the US, naturally drifting all guesses to the centre of the country.
It's not about the edge vs the center, it's in general that it cannot sensibly represent ideas such as Florida and California being likely and Texas unlikely. It assumes a uni-model distribution of confidence over the map.
I find it especially impressive considering the AI only has a single image to base its guess off of, while a human player can scroll along the road and turn the camera around
Please make more videos on this, this is amazing. Im sure people like Geo, jack, etc would even be on board to help too, so cool!
Great job my dude! I thought it was close to impossible to code an AI for GeoGuessr and yet you did it in a surprisingly straightforward fashion. I respect people who code AI and Neural Network stuff since I couldn't do that myself...
Really cool. You could create one for the new battle royale mode where you just need to guess the correct country. Could use road markings and road signs for the training data.
Really impressive stuff! Perhaps could you share more about your code? Like how long it took you, some snippets, and even share it! Loved the vid! Congrats on the great AI performance!
Yess, finally im early to see a underrated channel that is probably gonna grow so fast before it has 10k subs. Todays date is 03/08/2021, time is 16:37 and this channel 6,37k subscribers. This video has 416k wiews and the channel has only 2 videos. Other video has 95k wiews. im sure this channel is gonna earn at least 140k subs in the next 2 weeks. if he keeps uploading videos, this channel probably would be a lot bigger. Love your videos! Please keep uploading :)
Nice simplification of the classification task!
I think a simple improvement could be, that the final prediction must always be inside the highest probability cell, but the precise position is moved by the weighted average.
I got this video recommended today.
I do not regret it
Fantastic content
Congrats on your video blowing up! Nicely done.
Now that Geoguessr has a U.S Streak mode, you should try it again :)
I personally feel like the weight for the highest confidence square should have been higher for best results, maybe the conf level itself as a weight for all (or an equivalent system the result falls inside the top confidence squares)
geowizzard; are u challengeing me?
I saw the the answer was near to squares that were most dark, maybe use a power law distribution for the confidence
Ideas for improvement (maybe they are feasible):
Implementing periodic boundary conditions for the average confidence of the position.
Using the whole 360° Image instead of the usage of the image selection.
I just found your channel. Interesting videos, keep it up!
It would be really interesting to see this done for the Urban World map, also Europe. Great vIdeo!
Really interesting - have another new sub! A few random thoughts that occur to me on this.
"Extremising" the probability outputs might help improve the fit on the edges of the map in particular. For example, square the probabilities (then normalise) to skew the prediction towards higher confidence levels.
For an all world system you'll probably have to drop or at least modify the averaging approach, as others have said. Maybe it picks the continent first?
How did you choose the size of your boxes? Obviously smaller boxes would allow a finer prediction but it's more work to train.
Given that human players can move around, perhaps you could give the AI more than 1 photo to base its prediction on.
I love the idea of this video. Seeing the AI guess in the world map would be interesting. You got yourself another sub :)
Great video. I would love to see it guess for the world map!!
Yes, moar geoguessr please!
0:33 WTF That's literally the village in Switzerland where I live - I even recognize exactly that street 😱
Is that a village?
@@ALLINONETECH6 It's a road leading to this town (about 20k inhabitants), there's a garage and petrol station there that I pass weekly 🤫
@@chavem oh nice👍
I think taking the second power of numbers to determine the location would make this more accurate because it would make an ai more confident when he gets a higher percentage of chance for it to be in a specific state.
Great video CodeBullet, big fan since last 3 years
This would be a really interesting RL problem. Allowing the AI to explore the area and all.
This is great. I want to see the ai play a diverse world map, country streaks, and if there is there a way to make the ai more accurate. Thanks for the nice video!
Would love to see a video of some Battle Royale attempts!
This is very interesting, glad that the algorithm has chosen you, so maybe we can see what happend in world map
It would be interesting to use some topology to allow guesses to be pushed to the edges
Super cool project! I can't wait to see what you do with it next, should you decide to continue. Once you get the average bias problem sorted out, it looks like you'll have a really powerful tool. With more training, the program could be used for so many applications. Well done!
great video! with the algorithm blowing it up now, i think its time to try this again for a country streak or state streak game!
The weighted average needs a lot more stock in the darker boxes. This is pretty good
Loved the video and really surprised by how well the AI did with so little information! Judging by the comments your video seems to be gaining some traction, that's well deserved ! Hoping more people see this !
Dang I thought there was no chance for the AI. Very interesting and well done video, good job!
Also - thanks YT algorithm
I like how UA-camrs makes AI using the simplest method but getting good results while scientist and researchers using complex methods and yet not satisfied on their result
I am a bit interested in seeing a GitHub repo just so I can fill it with suggestions! Some ideas:
* make the input an entire 360 degree strip. Something can be done to manipulate that street view I think.
* make the output an greyscale image with bright dots corresponding directly to geo coordinates instead of a bunch of brackets. Or look to the object detection networks for other ways to encode the guessed location/bounding box
Hey as an avid Geoguessr fan and a Data Science student I loved this vid! I always joke to my friends about building an algo that could play for us but I never put any actual thought into it, but here you are haha. Although is there any way I could see your code; as a more in-depth and specific analysis of development would be really fun and interesting. Also, I'm sure it's been commented but using this classifier on country streak would be a lot of fun. Maybe that is something you could attempt.
UA-cam gods are not wrong again :) very nice presentation man and looking for the follow up video
May have been interesting to run the same photos again but with you choosing a different view/ direction and see if the AI came back with similar predictions. You choosing a view from each location may have represented a biased input to the AI.
Great video and concept, clever bastard 😎
I think a critical note here is that the comparison to the pro's is not entirely fair. Your AI is trying to optomise score across all games, where as a pro knows to take risks and only keep the 1 lucky group of guesses.
It would be cool to see a best attempt of a group of tests where the AI goes with its highest confidence box being much more heavily weighted.
If you were to do a world map version, it'd be interesting to have a tiered approach with the first AI identifying the general region, and then have regional AIs that specialize in a given area
you should try this with the entire world. i reckon the results will be better as the margin for error is slightly bigger. great video though i'll turn my notifications on for your next one/
Cool project! I would be interested in how you acquired the data. I think you most definitely did in an automated fashion.
Finally, a channel like Codebullet!
Edit: you know, if you made that project open-source, you can improve the AI's image database as well...
"Each 'square' is approximately 340 miles tall and 360 miles wide" :D
I didn't realise that this is a 245 subscribers channel, it is over 1m subscriber quality! keep it up (I subbed :) )
i never realised your sub count untill the end of the video, good stuff keep it up
There’s a ton of things you could do to make this better, and it still did pretty good.
That’s so cool! Would love to see more in that direction
I think that a good idea to help optimize the results would be to add major landmarks. Like add recognizable mountain structures or make sure that coasts are always on the coast. Also, maybe being able to cut out the sky so that the AI only bases its guesses off of the land, instead of the particular weather in the area at the time you trained it. Those are my suggestions, can't wait to see a new vid
This was really well done, I hope you do more like this
Cool idea, seems rather good for simply plopping a standard image classifier into the problem. Getting a more precise position by weighted average seems a bit questionable, but it might work (i.e. improve results slightly) with some filtering. For example you might get two disjoint areas which are likely, say you can see the sea so you believe you are either on the east or west coast, but your prediction ends up right in the middle. Being right half of the time is likely going to give higher scores here, so just picking the center of the highest weighted square might actually be better.
You should make some evaluation code to check automatically how well it performs, so you can try out different strategies and see what works best. Then a great next step would be to make it predict on several images and try to improve the prediction from that. You can use your existing model for this. If you can remove the human element of you picking an image that looks useful, that makes it a lot cooler. (Make a small macro that takes a screenshot, rotates the view by 90 deg, takes another screenshot, etc. for that extra A.I. feel.)
If you decide to do this in the world map, my suggestion would be to train the AI with pictures that are pointed to the street, since road markings, signs, language etc etc differ for almost all countries.
you choosing the pic to give it makes it almost more about your choice than the AI IMO
I'd say that it's totally totally fair to pan the camera, though the moving around is a lil funky
You honestly short changed the AI. Human players get waay more image data. You should have taken 4 images at each location and seen what it would come up with then. This is seriously impressive work.
This is really really cool, personally, think it would be funny to see if the AI can be trained to be able to read signage and pinpoint where it is from given distances to other cities. I'm a bit biased, but the Netherlands is a good country to do such a thing with, as there are signs indicating directions and distances nearly EVERYWHERE, mostly because these signs can also be found on many cycling routes.
Here's my advice. Instead of taking the average of the centers of every reddish square, you should only consider the center of the best square. I've noticed that distant squares, even if it is not very confident in them, bring lots of disturbance into your average calculation. By only considering the best one, you will avoid this problem and get closer results in my opinion.
Comment for the algorithm great job! Actually maybe you could double the weight of the boxes at the edge of the map as its very good at getting central boxes but slightly struggles at getting coastal boxes? Great work though!
Could you perhaps design code that uses video as input? Like, record the geoguessr screen and see the AI get progressively more confident as you look around. Amazing video I'm always amazed by the capabilities of neural learning.
That was a fun video! I thought there was no way for the AI to get anywhere close, but those scores are impressive. Maybe you could try different AI learning methods and compare them to see which is more accurate?
now make it with more boxes, also make it so you can enter in multiple images and find the average placement of all of the images
Idk why I’m just getting this in my recommended but great video 👍
Chances are very real it's using meta strategies. There's entire websites on how you can narrow down where you are in geogeussr based on some characteristics of the footage such as errors in the stitching of the 3D image or dirt on the lens etc. Entire areas are covered by the same car/team so it could be possible that the AI has learned to recognize those artifacts as patches.
The AI can't differentiate between picture artifacts and the rest of the photo, so it might very well just be recognizing that, for example, every street view image in a certain state or county has a certain scratch on the lens or whatever
interesting how the winter picture from colorado threw the bot off
especially since its one of the area it actually did good at