AI vs Human Route Setting - Can You Tell The Difference?
Вставка
- Опубліковано 22 сер 2023
- AI is a topic of discussion globally right now, but we wondered if there was any practical application when it comes to route setting. Creativity is at the heart of what it means to be a route setter, and we were curious to see if AI could produce anything that matches or surpasses what the human mind can do. Tune in to find out...
Want to get WEEKLY on-the-wall training from Louis Parkinson and off-the-wall training from Neil Gresham? Join our channel and become a Catalyst Online Squad Member!
You also get access to all past Livestreams including Q+A's with amazing people from the climbing world, past Live Climbing Gym Follow-Along sessions and our Saturday Workout Lives!
We also have our own Discord server where you can chat with fellow COS members around the world, arrange climbs at local gyms (or outdoors!), chat about competitions or even post your cat sending a home V13!
You also have 24/7 access to our 3 resident professionals: Emma Woods for Mindset, Tom Herbert for Nutrition and Cristiano Costa for Physiotherapy and Injury queries.
Join our channel and become a Catalyst Online Squad Member today!:
/ @catalystclimbing - Розваги
To be fair to Sam on that third boulder, ChatGPT didn’t really give him much to work with. He basically just got “set some technical moves on holds” and had to figure it out. That boulder isn’t set by AI, it’s set by a first-time routesetter
I think Sam could have "prompted" the AI a bit more. It's possible to say "can you rephrase that into (whatever format)". But that wouldn't necessarily have made for good content...
Chat GPT is really emulating a lazy employes- tells the other one vague tasks to do and fucks off
I agree, I think they should do it again where a proffessional setter goes against another proffessional setter but one of them goes off the ai
Using the advanced data analysis feature in chatgpt to depict the boulder on a graph would probably make more sense than trying to interpret what chatgpt means through a natural language description. I think you need to pay $20 for the subscription for that though.
and he stopped trying midway on the second bolder so this entier video is kinda pointless
Very fun idea, but as a suggestion for if there’s a follow up: ChatGPT does great with follow-up questions which ask for more detail when it’s vague or has made mistakes. The second boulder would almost certainly have been a lot better with some back and forth dialogue with ChatGPT :)
My thoughts exactly, people need to learn how to properly use AI
Agreed! And they could test the boulder and tell ChatGPT how each move felt and ask whether or not to adjust it. Would make for some really interesting content. Hope they do it again some time.
That would be banger content @@kadycampbell3302 I would like to see bot ha beginner and a routesetter use AI to create something they think is really really good. Prompting is usually a good way for pros to get out of writer's block anyways
The problem is ChatGPT can't set the boulder anyway) It can't say where to place holds, distances etc... vague ideas don't really help here)
There are extensions that enable it to analyze pictures. If they do it again I hope they use images like the wall, the human set route, and the holds and prompt it for more information. They can have both setters use chatgpt as well
Strong Hermione Granger vibes from Imi as she struggles not to correct Sam throughout the setting of his routes. Love it.
Is there more Imi content? She just seems like a joy to watch and get perspective on her setter mindset. So cool watching her work.
She posts a bit about routesetting on insta (and several gyms around London always have lots of Imi's setting which i suppose is real-life content 😅)
now do another one where Imi sets according to AI and Sam sets on his own, would be interesting to see the comparison then, as pro route setter would bring more life into AI idea.
no point.. ChatGPT just gives some vague ideas, you can do almost anything based on them
so they will be her boulders
@@RomanGuro well applying this logic, this video had no point either.
amazing setting by Imi, but i think it would be cool to see boulders set by her and AI on the same wall keeping the styles simple to make it harder to guess :)
Indeed! We need a redo
no point.. ChatGPT just gives some vague ideas, you can do almost anything based on them
or! have chatGPT vs an experienced route setter write out instructions, and have imi set them and have both Imi and Louis guess which of the climbs were AI. This way it’s double blind, and Imi’s route setting experience has equal impact on both.
Very cool idea! Would love to see an AI trained for this, or an image based ai that going be more specific about hold location
I really took routesetters for granted, it’s absolutely beautiful to see louis coast through some well thought out moves like poetry
id love to watch Sam (or any other first time rout setter) set a route while Louis, Imi, and other veteran setters actively talk about the good and bad things and help teach the setter how to fix any mistakes they make. it could be really cool to vocalize the mindset of professional setters and see some good dos and donts
Best part of the video right here:
14:10
You're welcome
Something to consider is that ChatGPT is basically a baby AI and only knows what holds and climbing are via text. Once AI can understand and produce images and video, things like setting and coaching should improve dramatically. Someone will train an AI on the countless hours of videos of people climbing, give it the schematics of plastic holds and 3d scans of holds on rock, characteristics and metrics of climbers (like the Lattice dataset), and suddenly we'll have AI that will be bananas at analyzing movement and providing feedback. All probably within the next 5-10 years.
But
only if we provide it with this information and instruct it accordingly.
ChatGPT 4 is already capable of ingesting images & video feeds. The future is closer than you think!
@@jaredloffler2951 can it do this without us motivating it?
8:08 If you're not happy with the AI output, simply prompt more instructions and it will do what you want. The resulting route might not be good, but the instruction format you get will be as requested.
When Imi said she had no inspiration yet, THAT'S exactly where you can use AI as a professional to just... inspire. Don't follow it to the letter, but use your experience combined with its idea's and make something nice.
Also try and use model 4 not 3.5, it's a lot better, and there are free to use sites online that even focus the application on certain topics :)
Either way, always love me a setting vid.
I use boulderbot on my homewall and it works pretty well once you've set it up right. It makes you point out holds (location, style, direction and difficulty) on a picture of the wall and some wall dimensions. Then there's a button that just generates a boulder for you, and some sliders for difficulty, number of moves etc. They're similar to what a very new setter would make, but they're a good and extremely quick start. With some experience and tweaks they turn into good problems that are outside of my own style. And every once in a while there's just some outlandish move that I just wouldn't have thought of.
I'm an NLP dev. Things I would want to try:
- Create a formal grammar for text representation of routes. Some sort of grid probably makes the most sense. Each node being a drill-hole. Perhaps different letters for slope gradients. Different numbers for different types of holds.
- You can use many-shot prompting to teach it or you can use fine tuning. GPT3.5Turbo can now be fine tuned.
- Why not start with a canvas board? The app probably already has a standardised grid representation that you can feed into it. And if you can export those, then you have all the data you might want for fine-tuning.
- Some tree-of-thought reasoning prompts or something similar.
In my opinion, a global database of successful human-set boulders would be useful, showing wall angles, distances, hold manufacturers, etc. Online Observation channel here in Japan uses great 3-D rendering, but there could be a simpler template system too.
As long as ai doesn’t start designing holds since it has no idea what our hands are supposed to look like
You'd have to have a large enough hold to fit all 12 fingers, 24 if you plan to use both hands.
I see what you did there.
This was lovely, i would want to see a Nikken vs ai setting session as well
Let’s switch the roles for Sam and Emmy now! Let Sam do whatever he feels and force Emmy to try and follow very specific AI instructions
I would like to see this done again except with someone who is experienced with setting, but letting chat gpt fully guide them, because of sams lack of experience he wasn't able to fully extract from chat gtp I feel, super cool idea, might see if we can try something like this at the gym I help the setters at!, might be a fun experiment!!!
Lots of potential with AI augmented route setting. Would be really cool to see what a professional route setter can do with AI assistance; especially if using a visually trained model to supplement the text.
that was fun to watch :) love your videos
Cool idea!!! Surprisingly worked decent
I had chat GPT set some kilter climbs a few months ago. It doesn’t fully understand how the human body works (too many dynamic moves to crimps), but there was some cool stuff. No new or creative moves (except one time it told me to do a downwards dyno to an undercling… on a kilter board at 40 degrees), but some pretty okay boulders
I work in machine learning, specifically RL, it’s awesome to see brainstorming/collaboration between human and AI work out so well. Awesome video ❤
I think it would be better to use something like dall-e 2 to generate pictures of a route that you can then try to copy.
I hope you revisit this idea sometime in the future because there are some things that would make it much more interesting. The thing about chat-gpt is you have to be very specific about all the details you want it to produce, and you should specify how it should approach its output and what factors it should keep in mind. That way it sort of gets closer to outputting something that would follow a coherent thought process rather than something that just sounds kind of right. That will produce a more coherent baseline prompt, which you then tweak based on any problems with it's answer. Once the problems are minimal you can ask follow up questions to further refine any small details. I played around with this a bit and got much more coherent boulders. That being said, it still isn't on par with a good human route setter, so if you do this again in the future maybe pair the route setter with the ai, and the newbie with their own creativity.
VIDEO IDEA:
If you want to make another AI routesetting video, take a picture of an empty climbing wall and make Dall-e fill it in with a boulder problem. Then you try to set that boulder
Louie is the Guthrie Govan of climbing. Subscribed!
super fun idea!!!
fun video!!! also what's the trousers Sam is wearing, great the color and design!
The AI boulders were more like "enterpreted AI", as the program did not draw a boulder or gave very specifik instructions...
Very interesting idea for a video, though. 👍👍🌞
At least tell ChatGTP to be more specific and also give distances and angles. 😭
right, he has to use chatgpt better. Zero follow up questions here.
This was real wholesome.
amazing video idea
very good enjoyable video with lovely people 👌🥳
great video!
I usually watch while exercising. Today I couldn't finish sets because I was dying laughing. xD
Where is my boy ink?🥺
Make AI generate a picture of a V7 boulder, based on the aestethics instead! This was just a new route-setter trying to decipher an AI's message type boulder.
for beginners or very little experience, i think that the AI system is on point. very detailed etc. And would certainly help improvement
If you’re not balanced on an climb, including and overhang, how do you stay on? If you’re unbalanced the forces will throw you off.
You could try that with a Kilterboard, it has pre set holds
Anyways the idea is kinda cool
I think it would have been a bit tougher to determine if none of the climbs had used any volumes. That and give the AI only one hold family to set from so that part of aesthetic looks good for both too.
So you could say that the AI's boulders were just... bad imi-tations 😎👉👉
More Imi content is needed.
I don’t understand why in these videos they don’t ask Chatgpt for further advice or more specificity when they don’t understand soemthing. Chatgpt doesn’t work w just one prompt, you’re supposed to ask it an initial question and then dive deeper into the matter until everything is clear and AI can give you truly helpful advice,
try doing some work with ian browns ai setting i think ive seen him doing some stuff like this but more specific
I mean, it would actually be pretty interesting to create an ai made for routesetting.
It could end the discrepancy between gym grades and outdoor grades
@@paulgaras2606literally how? Sorry that comes off as rude, but AI that we have now is just basically pattern recognition and interpretation, so it would base its route off our systems, not make its own.
Actually, people are doing this with the kilterboard. The number of parameters is much more limited and there is a huge training set.
@@frozenturtl827 Yeah you could train an AI to make an educated guess but it obviously wouldn't always get it right
I thought about doing this for my dissertation coz I thought it would be cool, then I gave it some serious thought and it would be next to impossible for one person. It's an incredible amount of work to gather the training data alone whether you want 3d models of climbs or images of them, and then actually having it make interesting climbs would be hard as that's subjective. People have made non-AI algorithms that place the next reachable hold but even that is subjective.
Maybe have a beginner route setter set against an AI. That way they’re both new and know less techniques. May be more of an equal playing field :)
I believe an AI that uses computer vision could build more objectively graded routes. It could also explain to climbers through an app interface movement suggeations and subjective route feel based on force and power factors
they should have done a boulder based on ai with imi's help
We need a ChatRobo-T to set the boulders.
I think it would be a cool idea to have Imi use chat GPT as a tool for route-setting. Not blindly following it but more as a guideline, almost like a collaboration between her and the AI. Maybe it could turn out to be a useful tool for her
I had to try and see if chat gpt could set problems on the (mini)moonboard. It can't :/
Which gym is this?
While this video idea is very fun, I don't think this was a fair comparison. I think this mostly just shows amateur route setter vs professional route setter (different families of holds, no sense of aseshtetics, etc) rather than the AI making "bad" boulders.
I think Imi would've set a different static route with the AI's instructions and it would've been much more human-like (aka good).
I wonder what 2 good routesetters, one human one ai, would make of it
it wouldve been more interesting if you tried to replicate boulders generated by AI art generators rather than ChatGPT
Did Imi just invent the double fist bump? 😝😝
More Imi content!!!
They could have used an image AI instead, I think that would be more effective
He said that the looks are bad cus of the positioning and holds but ai didn’t choose that one they should have asked a image create ai to make it
13:40
He has a point and it’s scary
The moment he set the red one I knew it was way too easy for Louie -_-
Wheres Magnus?
Chat GPT didnt use any volumes
It was not really a conversation with ChatGPT. You could have gotten much more information out of it by asking it to provide specifics.
I wish you took it seriously. If you do it again, ask the ai for one move at a time with a back and forth conversation to refine the individual moves. Also provide it what makes a good move for you. It doesn't know what makes a move good, so have louis write up a paragraph or two on cool moves or sequences. GPT learns well from example. First and foremost it is a language model. It is good with words, not rock climbing. Just asking it for a whole thing and throwing your hands up will give you terrible results by default.
Overhung climbing is just slab, but upside down
Don't know if you'll see this but I'd love to see this become a (perhaps yearly) series. As others have pointed out this is the worst it'll ever be, and within our lifetimes I'm positive there'll be an AI route setter that's as good, if not "better", than the best human setters. Even only a few months later there's ways to build on the challenge. GPT-4-V is quite good at understanding images, so you could give it more context about the wall, volumes and holds (especiaily for holds, you could provide images and keys for each hold so it can choose itself. You could even overlay a coordinate grid on the img of the wall so it could tell you precisely which holds and where. Within the decade a robot will be able to set problems autonomously.
I'm a natural language processing engineer - ChatGPT is not a very good way to do this 😅
What would be amazing would be to see an AI trained to move a 3D person to complete the routes. Then you could create an optimisation algorithm to move the holds and the 3D person can test the route. You could also teach the AI what different levels of competency look like that way you would get an idea of what grade the route you've created it. You could also experiment with different body types.
They should just use midjourney to design them a route and try copy it 😂
ChatGPT just fools around)) it does not set any bolder problem, just gives some very abstract and vague ideas ..
Soon a robot will set the whole boulder... 😅😅
.
the toe is entrancing and the camera man knows the best angles
🐫
I appreciate that this is a completely unreasonable thing to be bothered by, but..... the way louie tucks his t-shirt so snugly into his shorts that it pulls the seams on his shoulders forward making the t-shirt sit wrong/weird is a cannot unsee pet peeve.
What if the pro route setter is using AI and the novice is using human brain? Do that to do a more realistic view on AI.
Leave the human decisions to the humans that actually climb... maybe... 🤔
The biggest difference between AI and Human Intelligence is creativity. AI can only take what exists. It cannot create something truly new. The only bits out of the AI routes that were good were the creative, human interpretations of the route setting. Both could have been boring "ladders" according to the AI instructions.
"Creativity, something AI cannot do"..... actually, it's one of the things AI is spectacularly good at!
nuh uh
@@dylandark30wow, I'm floored by this human creativity on display! Obviously you have proven me wrong with your detailed and well researched argument. I retire from the battlefield, my head hung in shame.
As a computer scientist I can tell you AI is the opposite of creative lol everything it says is using pattern recognition based on data it was selectively trained on. It doesn't think or know things, it just builds sentences by following the trends it can find in given data
@@ZombieChicken-X ..and gives amzingly creative results. I use AI in my day to day job for image generation and scripting as well as many other non-generative tasks such asbackground removal etc. The best thing about all of the tools I use is their creativity, the ability to generate dozens of new ideas. I don't care how it does it. I care about the results.
and Im telling you its not creative, its just a machine that spits out amalgamations based on what its been given. If a model was not trained on any images of fire hydrants and you asked for one itd probably give you a messy blob of ink. I use Midjourney for concept art generation, the 'creativity' you have is limited by your prompts and the training data. You made a statement and Im just telling you why youre wrong@@chelfyn
Chat gpt isn't designed for this. Kind of dumb.
not gonna watch videos about AI
Not a very good idea for a video honestly. ChatGPT is text-based so this is quite uninteresting.
I would like to see some more back and forth with the AI another more polished video of this would be neat