Was Sail cheating? How you can tell for yourself.
Вставка
- Опубліковано 4 вер 2024
- A suspicious-looking video popped up on Reddit with a clip from a Fox player named Sail. Were they cheating? I'm on the case.
Original Reddit post:
/ they_played_the_entire...
Remember, always fuzz your macro's input values and timings with appropriately-distributed noise functions.
dont say that
@@jeremyie lol anyone with basic knowledge of this stuff could figure out that this is something that could be done to evade being caught
The true takeaway.
If I was cheating, I would just have the macro be recorded by myself in training mode first, then it’s not weird
@@zachstarattack7320 the problem with that is that someone more intelligent will see that you do the same exact inputs in the same exact timing, you would have to record yourself several times so each time is unique
So sail is just high tech, mid skill.
Yep
I like this terminology. I think I'm a low tech, mid skill kind of player.
I would not even call it high tech, it seems everyone just thought he was cheating due to spamming super small dash dances.
Never saw another clip with anything really extraordinary.
I’m low tech, low skill
Kinda like relno
Nah, Sail won because the opponent was trying to summon Satan with his inputs. Evil never wins.
Evil is winning as we speak. Research the occult
I knew someone was gonna say that! XD ahh, made me laugh.
Greatest thing I have read all week.
I don't know, let's not get ahead of ourselves here. Evil wins quite frequently.
@@jimtroeltsch5998 Not in the long run.
i think when fizzi finishes ranked there will be people who actually cheat a lot more often. these metrics will probably become a lot more important
I think a report button that analyses the replay would be effective
@@Luper1billion 100%. It should check these metrics and post it to Reddit for community analysis.
the arms race begins. inb4 midgame captchas.
I wonder if Slippi team will be able to automate detecting metrics like "button duration, inputs per minute, and analog burst windows" to automatically flag people for review. Kind of how chess sites do it.
@@J_Davi People who create cheats are usually pretty creative. Wouldn't it be within the realm of believability that those programmers would put in place randomizable algorithms that emulate human inconsistencies and error? And even within those inconsistencies a consistent pattern of inconsistencies to establish a pattern of play from session to session?
I'm so shit at smash, but I love the communities dedication to directing this stuff. Very interesting
Ayy Moliminous, I used to watch some of your stuff a long while ago! Didn't expect to see you here lol
get good
Ok
The obvious question is:
Where does Wizzy's SDI rank on that analog burst IPM chart?
perfect sdi is an input per frame so, very high
@@triplestomp1186 Most likely Wizzy is wank SDIing once per 2-3 frames.
@@FortWhenTeaThyme i'm just pointing out that with wank it is very possible to get one input per frame
@@triplestomp1186 Is wank SDI using the analog and the c stick? Or is that something else? :o
@@Jardonius I think "wank" SDI refers to moving the controller while you keep the joystick thumb somewhat stationary. Kinda looks like you're wanking when you do that.
this is all pretty reasonable, love the way you outlined the information. I'm sure that a player will make a macro that will have button press duration fit a normal distribution and carefully limit the burst IPM.
You don't understand how macros work do you?
@@thesaucypickle9866 have you looked at the code for smashbot?
@@fmakofmako he wouldn't know what he's looking at based on his first comment
@@thesaucypickle9866 you can have macros include logic. This logic could easily include a random distribution of input length that still results in the macro having the expected outcome. There are still ways to detect it but it becomes a lot harder.
Hell, you could have macros that check game state and change/abort if it's not going to work. For example, a wave shine macro that changes wavedash direction or self-aborts if they happen to wiggle out. Since you can monitor game info in real time, it's just a matter of having a solid setup.
Yeah, I remember back at the release of one of the WoW expansion packs, I used a macro that would press a key every few minutes to keep myself online even when I wasn't there (because logging out would put you in a huge login queue) and I specifically made the macro to have both:
1. slightly random button press durations
2. slightly random intervals between each button press
So, even though I was using an automated macro that would press a button for me, I made it LOOK like it was human input by adding in small amounts of randomness to the macro, so Blizzard wouldn't be able to detect that I was using a macro. (even something as simple as an anti-afk macro that presses a key every 5 minutes is a bannable offense because it's "automation" and thus botting)
I'd imagine it wouldn't be all that difficult to do something like this for Melee macros.
this just showed up on my recommended wtf
hi sail
@@euphoricenjoyer hello jealous
Sail just looks like an Osu player who picked up a digital controller
280 bpm deathdasher
Free my mans Sail
i have no idea who u are but congrats on 100k
All he did was smokeless dash dance!
He aint do nothin
#freeSail release my boy from those chains ⛓ 😭
Your graph at 9:20 is more of a gamma distribution (which would make sense if you think of it from a maximum entropy perspective. and yeah for modeling purposes normal is prolly good enough) Also Sail could still definitely have a micro for that dash dance.
(I should add that this whole thing is amazing and you're all doing a great job)
You in grad school?
@@richardfeir935 i love how (science?) grad students can all sense each other by the presence of statistics
9:00 see, but now I'm trying to figure out why wizzy is holding jump for almost 60 frames
That particular chart appears to be in milliseconds, you can hear that the values he is verbally saying at that timestamp do not match the values on the chart at all either.
im wondering if it includes tap jumps. Easy to hold up for a while after one. Or sometimes if you're gonna full jump up aerial someone and you claw jump, you end up holding jump for a while
got hit while jumping at a high% so he just held the button down
This is great info. Knowing the tools we have to spot cheaters hopefully deters cheaters from entering tournaments in the first place and helpsto increase people's faith in legalizing box style and other third party controller mods that make the game more accessible and play better.
It also gives cheaters even more info to work off of to make their cheating less detectable and more discreet, so unfortubately videos like this just gives them ammo and will make cheating worse. There's a reason why most developers who make anti-cheats say absolutely NOTHING about how they work.
I'm surprised the investigation even started, it's very easy to do that dash dance on a kb or box so I would've dismissed it as that. Nice video though, so I'm glad you did do it.
wouldn’t be impossible to add something to the macro asking for extra jump frames in accordance with normal distribution frequency but I agree it’s unlikely, absolutely great video love to see it
If you're ever so cracked at melee that people start accusing you of being a bot, it is the highest form of compliment
Great vid, very interesting stuff, only critique is that it would be good to have the axes labelled. Loved the vid keep up the good work!
glad to know my gut feeling that he was just a gaming freak was correct :D
Yeah I had strong Boxx vibes from that dash dance
I had no idea how many levels of analysis they used to tell if people were cheating. Its awesome having the “curtain pulled back” on what happens behind the scenes of games. Subscribed. Keep up the work, very interesting watch.
alternatively viewed, this is a tutorial on how to create macros and bots that are difficult or impossible to detect. hopefully you understand this and have some way planned to nullify that.
AltF4 left a comment on reddit basically talking about this, but because of polling errors, a macro/bot that effectively disguises these things and one that allows you to do crazy superhuman shit are basically mutually exclusive.
@@miloferrara19 it's not really about doing crazy superhuman shit. Imagine if you never SD'd or never missed an L cancel. Shit determines whole tournaments yo
@@XxuplmxX at least as far as I understand, macros don’t work like that. A macro can press buttons for you but a smart one that reads the game state is a whole different animal. How would a macro make you never SD? That’s so broad and would most likely interfere with your gameplay. At that point, you’re basically writing an (incredibly advanced) AI.
You could write a macro that SHFFLs an aerial, sure, because that’s a consistent timing, but how would it make you never miss an L-cancel?
@@miloferrara19 you're right, I used bad examples that completely ignored the definition of macro. The point I was getting at was maybe better suited for something like a shinegrab or ledgedash. Just never missing some simple inputs due to spaghetti.
@@XxuplmxX but that comes down to what started this thread-either the macro is easily detectable because you’re hitting a ledge dash with the exact same angle every single time, or you make it worse and harder to detect.
What if I code my bot to vary the input times? For example, if pressing forward air is the "correct" move, then I code that they should delay any input from 0-5 frames, but I also set a mean/average for a Poisson distribution around 1.5 frames. You can already kinda do this with a SSBU mod (Training Mod Pack, select multiple "delay" options), but I think they are using a flat distribution for the randomness rather than a Gaussian or Poissonian process.
You'd have to make a function based on creating purposeful inconsistency.
It would have to completely drop the input sometimes without doing it, sometimes do it properly but at the slightly wrong angle, and come up with a rand that increases or decreases the min and max windows by a few integers after successful inputs. 0-5 frames is too consistent
theres a paradox when it comes to cheating, the more you try to hide your cheat the worse it becomes, as hiding it = replicating the imperfection that makes human players suboptimal. the only way for your cheat to truly be undetectable is for it to perfectly replicate the abilities of a human, in which case, you could just play.
in other words, you could get good instead. but it does sound like a fun project
i think its amazing how far melee players have come that even if a cheater managed to make an unnoticeable macro, theyd still be losing to top players because of the realistic frame delay. top players WILL put someone into a combo within those 5 mere frames of delay. absolutely fucking wild
as someone who played cs 1.2 through cs go, i can tell you one thing about cheaters. ~half of them turn the cheats on and off, and looking at his data, this MIGHT be reflected in that, instead of full on cheating the whole time
Love the John Avon basics in the back!
i didn't have any fucking idea who sail was but this analysis was so well put together
hey funk 👋 loving ur rewriting homestuck series! keep up the good work!
Very interesting, although this is kind of the perfect outline for how botters can go about making harder to detect bots.
Was thinking this same thing. If I was to code a bot, I would also create tests to make sure it follows the patterns illustrated in this video
@@tylerjones9197 if you made a bot perform like a human then it wouldn't be as precise and scary as a bot
@@312bigbeanburrito
It's like they haven't watched any of the SmashBot content. Changing it to make 1/70 inputs a "bad" input completely changes the difficulty. Top players actually took stocks off of SmashBot because of that 1/70 input, compared to it being almost impossible to land a single hit when it was perfect.
So if make a bot or macro that has so much randomness and intentional errors that it's undetectable, then it's not going to perform well, and even slightly above average players will probably beat it.
@@fumoffu_l exactly. If you're going to make a bot with imperfections to make it less detectable as a bot, you're losing the competitive advantage of making a bot in the first place
slippi detective blessing us with another upload 🙏
I never usually comment on videos but the format and styling of this video is very refreshing! Simple and concise!
and then Sail comes out with a video called "how my machine learning bot passed as a human player"
I've never really got into melee in any sort of way but how thorough and detailed you were with this investigation was impressive. I enjoyed the video style and pace. Great content keep it up...
Okay, but that read had me dying.
Sail must’ve came from clone hero community with that dash dance consistency lmao
this is an awesome analysis and very well presented too. glad you’re helping with ranked
Huh, since there were so few analog positions (and that suggests they’re using a b0xx), I’m surprised you didn’t look at the specific values and see if they compare to what’s possible to make with a b0xx
"Burst inputs per minute" is done on a scale where it's actually inputs per second. If it were inputs per minute then it would range form 0 to 3600 not 0 to 60.
video was cool and i learned some new things but mannnn those prints behind you are too cool, obviously the mario ones were cool af but i love the darker ones to your right
Dude playing against Sail had 666 unique inputs? My boy Sail is being framed, other player is in cahoots with unholy forces. MonkaW
That's a sick Advantage 2 keyboard you got there, great for ergonomics. Great video as always!
That was an absolutely superb explanation, I've never dived into this sort of thing before but I had no confusion whatsoever with your clear and concsice wording, thanks! ♥
Sail's analog burst IPM is probably similar to a lot of other B0XX players though, right? It should be easier to mash two buttons than to waggle a control stick, that's one of the main advantages of the B0XX.
Those super fast dash dances happened only when opponent dead (spamming with two hands left and right buttons). Those dash dances in game were much slower (one hand). It seems.
no
@@sail6573are you the player in the video? are you dashdancing with a ring and index finger? if so, what other games have you played to get such muscle memory? I can't even stream that fast in osu
tldr: not "cheating", just a box player doing something completely impossible on GCC
How does an SLP file handle "random" occurrences like Peach's down-B and G&W's side-B? Are they not actually random?
They are not random, they're pseudorandom. They follow a complex equation that changes each frame or for each random event, which is derived from a value at the start of each match and the inputs of the players.
@@magneticflux- Ah, got it. So if the game is replayed frame-by-frame the result of those moves will always be the same. Thanks for the info.
@@superilikeeggsyo If you're interested, fizzi wrote an article on the subject, called "Fighting Desyncs in Melee Replays
". It was for the replay software, but his finding applies to rollback. It's also why there's no music online (reading music, or loading a stage transformation, require a disc access which can cause desynch for both players online)
What about phantom hits? Are they handled the same way?
@@alexwood9941 phantom hits are not rng at all
They happen when your hitbox just baaaarely graze your opponent's hitbox. The game considers that it's not enough to give you a full hit but grants a small amount of damage anyway
Good video, but I feel like you missed an important, and possibly most important, set of inputs to check durations for, the dash dancing. Normally, pretty tricky since it's an analogue input, but because he's using a digital analogue, it should still be easily measurable.
What I'm asking is, when he's dash dancing, is there any variance in the durations he's pressing left/right, or does it look more like he's, say, holding a button that hits left/right every other frame?
nah dw guys he's an amiibo
2:53 I don't understand. Diagonal inputs are used all the time to wavedash and DI and such so how come there are none here?
Love those full art lands on the wall 😍
I haven't played smash since the one on n64 and only at a friends house and this video has been fascinating. I could definitely see someone hitting left and right that quickly with lots of practice on a keyboard when I think of people that play guitar hero and how quickly they hit buttons.
I don't even play Smash, but the technical information here is _awe-inspiring_ to see. Knowing how to break down a replay and all these reports to determine a player's legitimacy is one of the _coolest_ applications of math!
Some pretty unhinged art you got there
27 distinct inputs to 666....i think we're investigating the wrong guy
I'm curious why the Smashbot graph had a few longer-duration jump presses. Is there ever a benefit to holding the button for longer than necessary for a full hop, or does the bot just not let go of inputs unless it has a reason to? (I don't play Melee myself so I'm not sure if there's some obscure reason I'm not aware of.)
Unless it's doing something to buffer (which melee has very very very very *very* few of), there is no benefit to it
this video was super informative, very well put together and entertaining
nice job
It’s nice that you distinguished humans from bots using histograms and normal distributions
"they're definitely not using a GameCube controller...but that doesn't necessarily mean that they're cheating"
WRONG!!!!!
As a digital controller player for other fighting games, I always love freaking out enemies with super fast wave dashes just by holding the left button and mashing the right button.
What keyboard are you using? It looks super funky from the side shots
Question about SLP replay files. You say they store exact inputs frame by frame, but how do they handle RNG and RNG seeds? Does it take the seed at the start of the game and with the TAS precision of the replay, it advances the exact number it would on exact frames in-game? Or do SLP files take things like outcomes as well, like seeing where a luigi winds up after using side-b and applies that appropriately? I feel like the first option is a lot more likely but also they're both very possible to mess up just by the nature of RNG itself.
ua-cam.com/video/rVqRrjZ-8yE/v-deo.html
@@Nekolo6 Thank you!
From this data we can conclude... Damn Sail is REALLY good at pressing jump for as short a time as possible!
Because it's a bot.
If you want to hide the fact that you are using a bot, you just make it mess up here and there.
And if you REALLY want to hide the fact you are using a bot, you make a UA-cam video explaining how it's not a bot.
@@justinmadrid8712 and if you have no evidence for your claims, you don't present any
@@Harry-mf6rq Evidence is common sense. If you create a bot and it is perfect in every way, you will get caught. Meaning...
Meaning, that showing a few clips of the bot making mistakes and claiming that as evidence that it isn't a bot, is silly.
@@justinmadrid8712 you presented neither evidence or common sense. You just made up a story based on nothing.
>Reddit
Discarded
Hey I saw your comment on reddit!
Not saying this is the case, as I don't know how hard bot-busting is enforced on this platform, but there is something to be said regarding programed inefficiency. Done the same with RuneScape bots to keep em from drawing too much attention while still performing their task efficiently enough.
This is a good video, but knowing exactly how cheaters are caught might help them find ways to evade detection. Certainly I imagine it wouldn't be quite as simple as fuzzing your macros, but I do hope that there's some more methods you and the rest of the team has up their sleeve.
You'd be surprised at how difficult it is to generate random noise. Even commonly used RNG functions aren't true random and have patterns associated with them.
I can even fathom how you dashdance to that level of speed. At my fastest speed you still can see fox moving left and right but when Sail does it, it's almost like he's standing still.. 🤯 actually ridiculous.
Box controllers make dash dancing way smoother.
Press adadadadadadadad on your keyboard as fast as you can. You will see it's pretty damn fast, and that is with zero practice and without whatever crack head energy fuels this madman.
"they're definitely not using a gcc but that doesn't necessarily mean they're cheating" ~well hold on there
It seems like most of the data here was aggregate over the entire match, which is reasonable for statistics like distributions of the various mentioned metrics over the entire match. But would it make sense to look for anomalous distributions of these metrics during certain sections of the match, such as during the sections that seem most suspicious to having been performed with a macro? It seems reasonable that Sail might not have been using a macro the entire time, just toggling it on for certain situations and then switching it back off to avoid detection.
Yeah, this is what I was thinking. Plus, couldn't one just program a macro to give some variation in the button press duration and other stats that people might look for to tell if someone is cheating?
The problem with that is you're picking and choosing.
Some people are worse at certain parts of the game, their performance drops and raises. You run into these what ifs but fail to ask the "Wouldn't they accidentally turn on the Macro?" since a single misread moment does that giving it away.
9:30 are you sure that's a normal distribution? That looks to me like a skewed distribution although it could also be log normal? I could totally be wrong though, it's been a while since I took statistics.
Fuego content my guy
Macro writers:
_write that down, write that down!_
unrelated but I like your mtg land art prints dude!
Would it be doable to make a macro/bot whose inputs conform to the distributions displayed in this video? I haven't ever coded anything like a macro/bot for a game, but with my limited knowledge I would expect a dedicated cheater to be able to add some random jitter to analog input values and input durations and such, so as to make it pass these tests.
It's possible, but detectible rather easily. You'd either have a uniform distribution or a very distinct split normal distribution.
MtG full arts.
Wait - how would he have a high analog burst IPM, comparable or even better than players wiggling the analog stick as fast as they can, if he's using a box?
where did you get those prints of the John Avon Lands? :O
Really interesting video!
Looking forward to more videos like this
How to make a bot thats undetectable:
Let’s watch him play in person.
It would be interesting seeing the number of frames jump is held for peach players
Next level basement Fox
Ayeee I like the full art land pictures on your wall, very pretty
>sail was using a digital controller
so he was cheating :3
couldnt you have multiple macros for the same action? where the effect is the same but the timing of the inputs are slightly different?
i guess maybe you couldnt get a normal distribution that way.
ifni was sail cheating, I would make fake inputs in between bot inputs to show flaw
Very interesting video but I worry the more you post about your methodologies for detecting cheaters the more discrete they'll be with how they cheat.
Really interesting look into things, and extra cool to see a video that actually proves somebody playing perfectly normally.
pls can we get this for LoL? I get accused of 'scripting' any time I pin down a predictable opp.
Poor guy was just trynna play.
i can smell something fishy
but nobody was cheating
-They are human and not using macros- If they are cheating, the tests applied here didn't detect it.
Not to imply that I have strong feelings that they are cheating. Claims should be made with evidence after all, and it's probably right to move forward with the assumption that they didn't cheat if no evidence is presented.
Looks like digital controllers just make movement look a little weird because the game was made with the assumption of an analog stick.
sorry if i dont make sense, but i have a question
would it be possible for the stats on a slippi replay to be misled by another third party thing like a cheat engine or number changer?
It's possible to doctor a replay file, sure. But this was recorded and given to me by Sail's _opponent_. So there's nothing Sail could have done to tamper with it.
@@2600AltF4 oh absolutely, i didnt mean to make it seem like your help in this search for truth was doing anything bad
i was simply wondering if in a nutshell it was possible
thanks for the reply! and for improving melee!!
You can record macros with random variation timings, this is a feature in razer synapse. You can also record your own mouse and keyboard input then just have that playback.
So, someone lost, got butt hurt, and took it to reddit? Classic.
if you click on the reddit post instead of being braindead, you can see that he never even mentions that he thought he was cheating
i sometimes play with a smashbox, and im glad im not that great at melee to not be identified as a bot for my digital inputs
This is how boxx players think they play
So he actually does have a really good gaming chair
I hope our national security is as integral as the smash community
what do you use to generate your graphs?
Might be wrong, but this looks like matplotlib, a library for python, which is also, I bet, how he’s reading the replays.
so randomize button holds a bit, keep (burst) ipm low and make your bot a bitt of an asshole and you're good to go with cheating :D
Bruh Digital controllers are cheating in themselves 😂 how is it fair they get the same input every time.
I appreciate the analysis dedicated to sift out bots... but isn't the explanation of the analysis just going to teach the bot creators to better hide the suspicious activity??
If the bots are coded well enough to be indistinguishable from human players, then does it really matter? In order to not be detected as a bot, it would have to make player-like mistakes - at which point a player could presumably win?
@@TestSubject06 That's a good point. However, bots coded well enough to be human is still a problem bc it is easily replicated... If the bot can play like a top 100 player it isn't impossible for good players to win, but code can be copied quite easily.
So the player base could become flooded with extremely skillful bots. Thus making Slippi very difficult for lower level players.
Generally I have more faith in the Melee community, yet still a concern.