There is a problem with the German text, the official conversion of special characters is: Ä = AE Ö = OE Ü = UE ß = ss But you used Ä = A Ö = O Ü = U ß = ss
No! The ß cannot easily replaced with SS. Only if the vowel of a word before ß is fast/short spoken it got replaced with ss and the word as a whole is written in capitals, the SS is used. There is technically a capital for ß, but nobody uses it. There are many words you have to write with ß
Cookie Builder an excessively strict interpretation. you use ss instead of ß in crosswords for example, and there's nothing technically barring you from writing it there
Well, the alternative is to call "Yoyle" Japanese, which I think justifies the 85% confidence. Also, realize that all German words are stripped of their accents, so they're only using the basic 26 letters.
do the experiment on yourself, guess which language it is for 200 words and compare the results to see if you can beat the network in a 200 iteration run.
Cassidy is english. Even if its a name its an english name. A 'y' at the end is not only uncommon, but extremely rare. I cant even think of a german word that ends with y...
Cary, you should derive the step size from an equation where the input is the % of the last 200 correct and the output is the step size. That way the machine can make more improvement early on, then make teeny tiny adjustments as it becomes better, and it is far less likely to over correct.
I can't wait to see the results of all 12 "languages" (Since Random and Key Mashing aren't really languages...) running overnight! WAIT 𝓦𝓱𝓪𝓽 𝓲𝓯 𝔂𝓸𝓾 𝓵𝓲𝓿𝓮𝓼𝓽𝓻𝓮𝓪𝓶𝓮𝓭 𝓲𝓽???
if you can't write german vowel mutations like äöü you have to rewrite them as: ä (pronounced "e" as in "_e_nemy") = ae ö (pronounced like "e" as in "her") = oe ü (pronounced like "ee" as in "see" with rounded lips) = ue so words like "nächste" become "naechste" rather than "nachste" which doesn't mean anything
Basically, the neural network runs the calculations, and outputs something between 0 and 1 for each option. The closer an output is to 1, the higher confidence it has. The program then checks all outputs, and reads out the one with the highest confidence.
I saw this video in my sidebar and thought this might be an interesting thing to bring up: Remember when Skype used to have the little "typing" animation? (It might still do...) Well, they had an animation specifically for when you key-mashed that was a cat walking across a keyboard as opposed to the usual human-typing animation. So essentially it could work out when your cat was walking across the keyboard...
Darc Gibson AFAIK it would keep a running average of time between keypresses, and if it were to drastically shorten it would assume you were key mashing
I would just like to say that as a viewer who came across your channel for the machine learning stuff, TWOW seems really awesome and I'm super excited to follow season 2.
Have you considered making the step size larger and smaller depending on HOW wrong the neural net is? So it being 90 percent sure of something it's guessed wrong would yield a larger adjustment than barely being sure at all. This might increase the accuracy of the algorithm without having to drastically change the net.
It's right here! Though some people have told me it doesn't work... drive.google.com/drive/folders/0B98DkR2AsGZaTnQ1aUFILU85VTg?pageId=100617964454119728920
In order to make it work I copied and pasted the brain code into the neuralnetworklanguagedetection code. Then I had to disable the font (for some reason it wouldn't allow the font). Any amount of languages less than 3 worked, but over there would be an error reading a string eventually.
Detect language based on sentences is a lot more effective, because there lot of word collisions between languages and gramma patterns are a lot easier to pick up the sequences of letters. Japanese have specific set of silabels so Japanese can be easily dismissed if invalid set of letters occurs, like "FT" in word "AUFTRAG", also japanese words will always with vowel or letter "N" because of that, this is pattern that your neural network should pick up
>...japanese words will always with vowel or letter "N" for whoever reads this in the future: Japanese words actually can start with (almost) any Japanese syllable (some of which start with consonants) except N :p but after a consonant is always either * the same consonant (tt for example) * y (ry for example) * s or h (for ts(u) and ch respectively) and all syllables end with a vowel (or are N)
Don't stop your channel! It is the best that i have ever seen! It's amazing to meet a person with the same interests as me! (I'm Brazilian so don't worry about my mistakes)
*"You thought you could just throw away all the parts of human life that was just too much stress on your poor little body, but you couldn't and those human urges stuck with you. They are there forever and you can never escape.- oh and look at that the prediction rate if over 80% now!"*
Cary, can you QUIT with the Boredom Repellant™ please? Anybody watching your videos is certainly not the kind of person to be bored by the things you talk about. Instead, the Boredom Repellant™ just becomes more of a distraction. 😱
For shorter words you could try to vary the starting position of the words in the 15input nodes, to prevent biasing of shorter vs longer words. This way the lower nodes in the hidden layer can also be updated more regularly
'Waren' means 'goods' or the past tense of 'where', which is a more common usage so most Dutch people probably thought of that meaning without any context. Also, there are multiple dutch words with three e's in a row, so I'm not surprised. Except the Dutch e is pronounced more often as the English a, so take from that what you want.
“We have the Swahili people going AAAAA” “The Dutch people going EEEEE” “The Spanish people going OOOOO” (And the racist people going “Hi guys, Carykh here”)
I suggest an easy way to immediately see the keymash pattern: Try it on Dvorak keyboard. I'll do a keymash, on a Dvorak keyboard: ha,ktboesu;itnhaucra,. etkuta,hr,ac.uhasntoeuh uchatnueobtuauhcr,a.uhcr See how much pattern it has? The qwerty keyboard keymash has just as much pattern as a Dvorak keyboard pattern. The middle row is used a lot more often, and some letters often appear close to each other, and it often alternates between left hand keys and right hand keys. It's just that on Dvorak, the left middle row is all vowels and the right middle row is all the most common consonants, so the pattern is immediately clear because the keymash is almost English-like. Now you can see the Dvorak pattern clearly because it fits the English pattern. For the network it's as clear as the qwerty pattern (since it doesn't know English, it can see one pattern as easily as the other), so it has no problem seeing that.
Is it too crazy to flip the order of letters before it enters the neural net? I mean, since every language has words beginning with all the letters of the alphabet the ending of a word is probably more representative of a certain language than the letter it begins with. It would be a fair improvement in the results with so little effort. Great job, anyway! (PD. I know nothing about neural nets, I'm just thinking out loud)
Im very thankful that you provide the source code in the description, i dont have the skill or knowledge to program this myself, but would like to do some experimenting, thank you so much
Yeah, I could do that! The only problem is that Processing isn't designed for machine learning, so the code I wrote is slower and clunkier than it could be. I made this project as a teaching experience to myself how machine learning worked.
Using äöüß would be quite a give away and also making the Code more complex as he'd need to add every letter with an accent on top of the 26 existing ones.
Actually, he wouldn't need to... His system could take in the entire Unicode lexicon... He does after all have 26 inputs per node. All he really would need to do is translate each symbol to binary through the Unicode codex. The function is already there, he just need to add the index for the characters to the software.
ADD ANOTHER LAYER!!!! More complex patterns should easily be picked up like Japanese is mostly combinations of syllables, and suffix -ness would be picked up no matter where it is.
Its so satisfing watching your AI training montages, I wish I knew enough python to make my own AI also, I am so surprised that you actually recognised Esperanto as a langugage hurray.
First of all, there's some *fantastic* higher-level frameworks that you can use to make constructing and training neural networks much easier. I use Keras, which is a Python-based framework that runs on top of TensorFlow or Theano (other frameworks for accelerating the kind of mathematical calculations used in neural networks.) It's very easy to use! Go here for info, including links to the actual site and docs: github.com/fchollet/keras-resources Second of all, check out these articles - they're good introductions (Especially Karpathy's!), and interesting in their own right: karpathy.github.io/2015/05/21/rnn-effectiveness/ (A good introduction to RNNs, and also super super cool) karpathy.github.io/neuralnets/ adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/ colah.github.io/posts/2014-07-Conv-Nets-Modular/ karpathy.github.io/2016/05/31/rl/ (Learning to play Pong from *raw pixels*!) Third, install whatever deep-learning framework works for you (I recommend Keras), and just start slapping layers together and seeing if something cool happens. Maybe write a MNIST classifier first, that's like the 'Hello World' of deep learning. Oh, and here's some great blogs - blog.otoro.net/ (Messing about with neural networks to do Cool Stuff for fun and art) karpathy.github.io/ (Not many posts, but a fantastic explainer - check out the Deep Reinforcement Learning article) colah.github.io/ And the r/MachineLearning subreddit is good for discussion and interesting new papers.
I think the NN would perform much better, if you and more layers to it. Then it takes longer to learn, but it will better understand the rules of languages.
Keep it up, love learning more about neural networks from your vids. One small question I have though, is the source code for your math evolution simulator on the Internet?
Cary, did you know that you can reverse the neural network? aka You can make it produce words that it thinks are a certain language! (I'm not sure how to though, so you'll have to look up how. :\ ) I just know that it's possible to do it.
The same way you train it, actually! Instead of finding the gradient of the connection weights with respect to the output, you can find the gradient of the *inputs* with respect to the output, and use gradient descent to optimize the input with respect to how german the network thinks it is. The way the input is structured, though - as one-hot vector encodings, where each character is equivalent to one of the nodes being on and all others off - would be very poorly suited to this, however, since I think you'd be incredibly unlikely to end up with anything close to a valid character. But that's how things like DeepDream and neural style transfer work - you back-propagate the gradients all the way through the network to find the derivative of the inputs with respect to the output, and optimize the input image to correspond to the desired output activations. (Style transfer - which lets you take photographs and classical paintings and wind up with selfies painted in the style of Starry Night - works by optimizing the target image, as seen by an image-recognition network, on two different levels - you try to match it to the low-level colors-and-textures features of the style image as seen by the earlier layers of the neural net, and the high-level abstract-content-and-semantics features of the source image. )
With the exceptions of n/m and sh, Japanese typically follows an alternating consonant to vowel structure. This is something that human readers would be able to pick up on pretty quickly, but your algorithm is incapable of seeing patterns like that.
Keymash has a pseudo-structure derived from the used keyboard's layout, while random has a different structure (that sometimes produces a word in the former's 'dictionary').
keymash and random will of course be completely different, because random will be exactly that, random, with no discernable letter pattern. Out of a near infinite series of 1-15 letter "words" that are randomly selected, all the letters will come up about equally, meaning no discernable pattern. your neural net says that if x letter is in y position then it is more likely to be z language, and multiple neurons come together to point to one language, so it can tell from sequential letter patterns, where the same letters arise in the same places, which is more likely. keymash is actually a language, letters that are closer together will be more likely to be pressed, so while its not consciously understanding that, it is more likely that thbvf will come before or after a g than z would. of course im a spatial reasoner so it would make sense that i would immediately understand the spatial groupings of sequential letter combinations in a synapse network.
after seeing this, I got thinking. what if someone made a bot that could be taught to beat bfdia 5b levels using the same method as this. so it could then be able to beat levels it hasn't seen before. the only problem is there arent enough levels for it to be able to learn. *wink* *wink*. but seriously, do you think this might be possible?
No. I have no idea what is going on in this video... All I saw was, "AI Sees when you're key mashing." Not sure what the relation is here. What AI? and what is being taught languages? Why is it being taught languages? Why is this significant? Is this one of those games based around being a programmer? Is this a game at all? Why was there a cube bouncing balls on its top surface and suggested it was juggling issues?
His AI. He wrote a neural network (which is a type of AI) designed to distinguish between words from different languages. As for the other stuff... Yeah I'm still trying to work that out as well. xD (As a side note I ended up on this video from the first part, which I ended up on from a video on the Unity engine....)
A book called "Predicting the unpredictable" has a good chapter on it about random strings of numbers. When creating 'random' numbers, each number does not have an equal chance of being picked, as you would expect if it was random. For instance, if you are picking two digits, '37' is more likely from a human than '66' (or another number were both digits are equal), as people believe they shouldn't repeat digits yet it has an equal chance of being picked randomly. When writing out large strings (100+ digits) people tend to repeat certain strings (eg '6482') much more often than should appear. This kind of demonstrates how terrible the brain is at creating random numbers. I think the same can be applied to the key mash/random, where, like Cary said, letters close together are much more likely. I would expect letters on the edges of the keyboard (P,Q,L) to be picked less frequently than 1/26, and also possibly some strings of letters are common based on normal keyboard resting style, for example: Gamers might have a higher chance of putting WASD than non-gamers. Would love to analyze the data for this. In summary, Predicting the unpredictable is a great book well worth reading, and humans are terrible at randomness.
I wonder how this algorithm would function if you didn't represent the letters as just 1 true and 25 falses. If you instead characterized each letter by its classification and "sound"*. *So the letter B would have a 1 in a "consonent" value; a 1 in "voiced" value, a 1 in "semilabial" (maybe also a 1 in labial, as it falls under both; though semi-labial is the more specific), and a 1 in a "stop" value. For more on what all that means: en.wikipedia.org/wiki/Consonant#Features The idea being that perhaps it would be able to pick out specific sounds and higher level patterns and easily associate them with a specific language. Like right of the bat; if a word is formed by a bunch of consonent-vowel pairings, it might have a higher chance of being japanese, vs a word with tones of consonants in a row it would have a higher chance of being german.
Key Mash is different because in Key Mash all letters seem to be around two common points per string, or a continuous line on the QWERTY setup, because humans generate it and they seem to either punch it over and over in one place per hand, or drag it across the keyboard. However, random is generated by a computer, therefore no pattern occurs.
The problem with German and Japanese is that Japanese mainly uses syllables consisting of either only a vowel (a,e,i,o,u and n) or one to two consonants followed by one or two vocals, but never more than three letters in a syllable. It usually should be fairly simple to detect. Also your German set is flawed because the mutated vowels / umlauts have simply been stripped of their diaeresis instead of transforming them into their transformed version with an appended 'e', so ä->ae, ö->oe, ü->ue and very important as well: ß->ss.
Essentially the entire thing that makes neural networks interesting and powerful, and almost the whole motivation behind the recent wave of deep learning success, is that if you structure your network right and provide enough training data, *you shouldn't have to do that*. Every past approach to AI required carefully hand-engineered features that required almost as much effort to develop and create as it would have taken to just solve the problem yourself. Deep neural networks, however, can actually use the lower layers to learn their own features, which the higher layers can process to do stuff like classification. It's actually possible for neural networks to learn how to play videogames using only raw pixel inputs! The trick is, though, you've got to actually be moderately clever in structuring your network, so he'd have to use a recurrent neural network or a 1D convolutional neural network or something else that incorporates the sequential, spatial structure of the problem.
I died at Swahili screaming the loudest
the second time me too
germans dont scream
FaeLuvZelda we say Haaaaaaaargh xD
said the swahili in the swahili coast
"AAAAAAAAAAAAAAA" - the Swahili on the Swahili coast
There is a problem with the German text, the official conversion of special characters is:
Ä = AE
Ö = OE
Ü = UE
ß = ss
But you used
Ä = A
Ö = O
Ü = U
ß = ss
Actually, new grammar rules in German say that the Eszett can be replaced with ss. However, ü must be replaced with ue.
No! The ß cannot easily replaced with SS. Only if the vowel of a word before ß is fast/short spoken it got replaced with ss and the word as a whole is written in capitals, the SS is used. There is technically a capital for ß, but nobody uses it. There are many words you have to write with ß
+HiiragiTia but when the ß character is not available for whatever reason (as in this video), the preferred replacement is indeed ss
Syzygy314159 When available never replace the ß, only if you can't type it you are allowed to replace
Cookie Builder an excessively strict interpretation. you use ss instead of ß in crosswords for example, and there's nothing technically barring you from writing it there
Unexpected Screaming Swahili
cpase
a wild siiva fan
i must be wary
lest i accidentally make a high quality rip joke
oh well off to see my grand dad
ARGH
Wonder if Todokete is more LOUD or Japanese...
uuAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAUuaghh...hhhUUUAAAAAAAAAaaghhhahahahahh.... hUUUUUUUUGHgh
Niko Baby I died laughing at that part
Cube_Splitter I know, right?
'You guys seemed to like the boredom repellent last time, so here's more' CARY I SWEAR TO GOD
Gio D | Ikr. The only reason we're here is because it's interesting.
Same profile pics
Gio D I love your avatar
Thanks, it used to be link (Twilight Princess) Put I changed it to this to match my other profile pics.
Ah, the pain in the ass wich are the Numbers on Google Keyboard. Also im dissapointed with your new Profile Pic. Zelda's better.
5:40
the reason it guessed "cassidy" is japanese is because it is definitely not a german word!
uh.... okay?
i heard thayt for the first time and refuse to believe that that is german, more englisch
Kaßidi
I am from Germany and it is not a German word at all:-)
BaumeisterJac k ich weiß ist ja meine Aussage
This is a weird episode of TWOW 20B.
Meester Tweester yeah... something seems off...:/
I can't believe, that despite a 90% accuracy, the machine predicted, that yoyle would be german, when german is almost void of Ys.
tgamagedon to be fare there is no letter L in Japanese.
The japanese R does get romanized as L sometimes though, specially in foreign words where L was the original letter (e.g. laifu, lakki, littoru...)
kikones34 there are no consonant clusters though (excluding "n" coming before other consonants occasionally.)
Japanese: (Is good at screaming)
Swahili: *I'm about to end this whole man's career*
A little racist, but I get what your trying to say
Letting it run on all 12 languages w/o any restraints like letter length for like 2 days even could make a good video just sayin
None of the Words predicted as German beeing Japanese looked any German from a German perspective.
espesally yoyle witch makes the confidence of 85% questionable ;D
Well, the alternative is to call "Yoyle" Japanese, which I think justifies the 85% confidence. Also, realize that all German words are stripped of their accents, so they're only using the basic 26 letters.
well using öäüß would be quite a give away.
carykh the weird thing is, the letter y is almost never used in german
...and it appears like it's used a lot in Japanese. I guess this project didn't work as well as I thought it would.
One of the test words for Dutch at about 15:16 is Hewlett-Packard. (It's easier to see if you go at 1/4 speed)
do the experiment on yourself, guess which language it is for 200 words and compare the results to see if you can beat the network in a 200 iteration run.
keep it up man, love your videos.
LOL
MIXED SIGNALS!! AAAAAAAAAAAAAAAAAAAAA
*screams in Swahili*
Lmao
6:35 No love for german letters, horen is actually hören
Cassidy is english. Even if its a name its an english name. A 'y' at the end is not only uncommon, but extremely rare. I cant even think of a german word that ends with y...
I checked it: the only words with y at the ends are words taken from english
Make a "Word Maker" so a reverse algorithm
Cary, you should derive the step size from an equation where the input is the % of the last 200 correct and the output is the step size. That way the machine can make more improvement early on, then make teeny tiny adjustments as it becomes better, and it is far less likely to over correct.
this is better than TWOW
TWOW is a completely different thing and form of media
:(
dynaboyj L. :(
dynaboyj L. That's my boy
I can't wait to see the results of all 12 "languages" (Since Random and Key Mashing aren't really languages...) running overnight!
WAIT
𝓦𝓱𝓪𝓽 𝓲𝓯 𝔂𝓸𝓾 𝓵𝓲𝓿𝓮𝓼𝓽𝓻𝓮𝓪𝓶𝓮𝓭 𝓲𝓽???
if you can't write german vowel mutations like äöü you have to rewrite them as:
ä (pronounced "e" as in "_e_nemy") = ae
ö (pronounced like "e" as in "her") = oe
ü (pronounced like "ee" as in "see" with rounded lips) = ue
so words like "nächste" become "naechste" rather than "nachste" which doesn't mean anything
æ œ ue
Fyi, 'entertainment' is also a Dutch word, so the net was corret, not just close.
🎶The AI Sees When You're Key Mashing🎶
🎶It knows when you're awake🎶
🎶It knows if you've been rand or not🎶
🎶So be rand for SkyNet's sake.🎶
What _is_ "rand"? 🤔
-emily
@@rebeccaangeles7310 hey are you single, i got a friend named cole thats looking for an (over 18 years or age) girlfriend
@@EpicGamer-te6cm who asked
@@foxfoxy5490 me, I am asking her
Do your peers or professors know about your channel?
They do not! Perhaps I'll tell them at some point, but only once my projects become worthy of those sacred eyes
I wouldn't be surprised if they know :) Your videos are great!
yo perhaps i missed it, but did you explain elsewhere how the confidence for any output is generated?
they already detected you with a neural network and your test answers as training data.
Basically, the neural network runs the calculations, and outputs something between 0 and 1 for each option. The closer an output is to 1, the higher confidence it has. The program then checks all outputs, and reads out the one with the highest confidence.
Actually it's "hören" not "horen" - and you really should take out "cassidy" from the german list. It's neither a word nor a common name.
indeed
doesn't mean nothing in german, sound more like a name
congrats, you found one person of millions
MEGACORP Domain I've made a search for you. There are about 15 People in Germany named Cassidy and none of them is born here.
Phoenix D. Black - König der miserablen Wortwitze Horen is dutch (means the same as in german)
Cassidy?!?? Not a german word! Maybe a name? Pretty sure it's Englisch tho...
It's Gaelic.
German detected.
confidence: 100%
As a hobby-linguist and a programmer these videos interest me highly. This is pretty neat!
It has a confidence in blank being swahili because your swahili database has blank spaces between some words..
"waren" is also a german word, so german wouldn't have been wrong.
I saw this video in my sidebar and thought this might be an interesting thing to bring up: Remember when Skype used to have the little "typing" animation? (It might still do...) Well, they had an animation specifically for when you key-mashed that was a cat walking across a keyboard as opposed to the usual human-typing animation. So essentially it could work out when your cat was walking across the keyboard...
Darc Gibson AFAIK it would keep a running average of time between keypresses, and if it were to drastically shorten it would assume you were key mashing
I remember there was an app locking the screen when it detected a cat walking across the keyboard.
Darc Gibson have you ever had a guinea pig walk across your keyboard? It's the same problem, but you can do something about it!
Since suffixes are more telling about a language why not reverse the word so that suffixes will always sit on the same node
Great idea. Easy to code and could have a significant effect.
Good idea
Genius
***** Would be interesting. No idea how to actually create such a thing in reality, i'm pretty new to this stuff.
I would just like to say that as a viewer who came across your channel for the machine learning stuff, TWOW seems really awesome and I'm super excited to follow season 2.
I'm a linguist, and "key mash so good" is correct
Conlang Critic You key mash so well
Is correct too
what that makes you is a fraud.
And you forgot a period.
OMG!!!!!
Have you considered making the step size larger and smaller depending on HOW wrong the neural net is? So it being 90 percent sure of something it's guessed wrong would yield a larger adjustment than barely being sure at all.
This might increase the accuracy of the algorithm without having to drastically change the net.
Make a download so we can mess around :)
It's right here! Though some people have told me it doesn't work...
drive.google.com/drive/folders/0B98DkR2AsGZaTnQ1aUFILU85VTg?pageId=100617964454119728920
In order to make it work I copied and pasted the brain code into the neuralnetworklanguagedetection code. Then I had to disable the font (for some reason it wouldn't allow the font). Any amount of languages less than 3 worked, but over there would be an error reading a string eventually.
Thanks i will try
Can you also provide the wordlists? Or am I missing something carykh
word lists are in the data folder
17:45 adding the 'o' made its confidence in spanish go up 10% though
Detect language based on sentences is a lot more effective, because there lot of word collisions between languages and gramma patterns are a lot easier to pick up the sequences of letters. Japanese have specific set of silabels so Japanese can be easily dismissed if invalid set of letters occurs, like "FT" in word "AUFTRAG", also japanese words will always with vowel or letter "N" because of that, this is pattern that your neural network should pick up
Shhhh... Cary is too lazy.
Uh... What's a silabel?
Oh look, it's a wild Shadowriver :P
KITATUS lol
>...japanese words will always with vowel or letter "N"
for whoever reads this in the future: Japanese words actually can start with (almost) any Japanese syllable (some of which start with consonants) except N :p
but after a consonant is always either
* the same consonant (tt for example)
* y (ry for example)
* s or h (for ts(u) and ch respectively)
and all syllables end with a vowel (or are N)
Use the neural network as a fitness function to generate words for specific languages with a genetic algorithm.
github.com/karpathy/char-rnn
github.com/shariq/burgundy
Don't stop your channel!
It is the best that i have ever seen!
It's amazing to meet a person with the same interests as me!
(I'm Brazilian so don't worry about my mistakes)
*"You thought you could just throw away all the parts of human life that was just too much stress on your poor little body, but you couldn't and those human urges stuck with you. They are there forever and you can never escape.- oh and look at that the prediction rate if over 80% now!"*
Keep it up, Cary! Love your videos!
yeah me neither
Photonic Wow! I am so good at making puns. It was PUNtastic!
Cary, can you QUIT with the Boredom Repellant™ please? Anybody watching your videos is certainly not the kind of person to be bored by the things you talk about. Instead, the Boredom Repellant™ just becomes more of a distraction. 😱
jackofallspades98 tm
™
Bru, it's a joke.
How u do the tm
Yeah, more like Fractal Migraine Nightmare™.
For shorter words you could try to vary the starting position of the words in the 15input nodes, to prevent biasing of shorter vs longer words. This way the lower nodes in the hidden layer can also be updated more regularly
'Waren' means 'goods' or the past tense of 'where', which is a more common usage so most Dutch people probably thought of that meaning without any context. Also, there are multiple dutch words with three e's in a row, so I'm not surprised. Except the Dutch e is pronounced more often as the English a, so take from that what you want.
well, if you used the umlaute, it would have been easier ;) hören would stand out right away.
“We have the Swahili people going AAAAA”
“The Dutch people going EEEEE”
“The Spanish people going OOOOO”
(And the racist people going “Hi guys, Carykh here”)
I suggest an easy way to immediately see the keymash pattern: Try it on Dvorak keyboard.
I'll do a keymash, on a Dvorak keyboard:
ha,ktboesu;itnhaucra,.
etkuta,hr,ac.uhasntoeuh
uchatnueobtuauhcr,a.uhcr
See how much pattern it has? The qwerty keyboard keymash has just as much pattern as a Dvorak keyboard pattern. The middle row is used a lot more often, and some letters often appear close to each other, and it often alternates between left hand keys and right hand keys.
It's just that on Dvorak, the left middle row is all vowels and the right middle row is all the most common consonants, so the pattern is immediately clear because the keymash is almost English-like.
Now you can see the Dvorak pattern clearly because it fits the English pattern. For the network it's as clear as the qwerty pattern (since it doesn't know English, it can see one pattern as easily as the other), so it has no problem seeing that.
I like how you can still tell it's you even through the voice effects.
cassidy is not a german word :D
Is it too crazy to flip the order of letters before it enters the neural net?
I mean, since every language has words beginning with all the letters of the alphabet the ending of a word is probably more representative of a certain language than the letter it begins with.
It would be a fair improvement in the results with so little effort.
Great job, anyway!
(PD. I know nothing about neural nets, I'm just thinking out loud)
Wow, I love these kinds of videos quite a bit more than the TWOW stuff, so I'm surprised more people wanted TWOW
Im very thankful that you provide the source code in the description, i dont have the skill or knowledge to program this myself, but would like to do some experimenting, thank you so much
Can you do processing tutorials on machine learning in the future?
Yeah, I could do that! The only problem is that Processing isn't designed for machine learning, so the code I wrote is slower and clunkier than it could be. I made this project as a teaching experience to myself how machine learning worked.
6:28 It's actually written 'hören'. Ö replaces 'oe'
Zum Glück gibt es ö
Using äöüß would be quite a give away and also making the Code more complex as he'd need to add every letter with an accent on top of the 26 existing ones.
That's true, I just wanted to point it out.
Actually, he wouldn't need to...
His system could take in the entire Unicode lexicon...
He does after all have 26 inputs per node. All he really would need to do is translate each symbol to binary through the Unicode codex.
The function is already there, he just need to add the index for the characters to the software.
The Major but with the curtent system it would make the code bigger.
I love the machine videos much more than the others
I have little understanding in this, but I'm thrilled by your videos, and it's inspiring me to go mess with cool projects including nural networking
ADD ANOTHER LAYER!!!!
More complex patterns should easily be picked up like Japanese is mostly combinations of syllables, and suffix -ness would be picked up no matter where it is.
was expecting bill wurtz style weirdness :(
The Swahili confidence was probably from the spaces in the data set
Its so satisfing watching your AI training montages, I wish I knew enough python to make my own AI also, I am so surprised that you actually recognised Esperanto as a langugage hurray.
Funfakt Waren is a German word meaning goods same in Dutch quite similar languages i supose;D
Waren is were in dutch too
wow yes, i didnt even realise that it also meant goods! :)
in German for 1. person plural aswell
TheXenonXe German and Dutch have the same origin.
Plot twist, it was wären😜
By the way, I think you'll have MUCH better results if you did this by syllable instead of by letter.
+Gamesaucer How would you detect syllables?
+SirCutRy The answer is obvious isn't it?: A neural network!
netforce 10 Nooooo, it's turtles all the way down
SirCutRy No, I mean that you enter syllables instead of letters.So you yourself would be the one to detect them.
Gamesaucer There are thousands of words for every language.
Just got an add for "Grammarly" GOOGLE KNOWS
So that's why talk-talk-mashble was open
I'm really interested in AI, neural networks and deep learning, any suggestions where I can start? (I'm a programmer already)
First of all, there's some *fantastic* higher-level frameworks that you can use to make constructing and training neural networks much easier. I use Keras, which is a Python-based framework that runs on top of TensorFlow or Theano (other frameworks for accelerating the kind of mathematical calculations used in neural networks.) It's very easy to use! Go here for info, including links to the actual site and docs: github.com/fchollet/keras-resources
Second of all, check out these articles - they're good introductions (Especially Karpathy's!), and interesting in their own right:
karpathy.github.io/2015/05/21/rnn-effectiveness/ (A good introduction to RNNs, and also super super cool)
karpathy.github.io/neuralnets/
adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/
colah.github.io/posts/2014-07-Conv-Nets-Modular/
karpathy.github.io/2016/05/31/rl/ (Learning to play Pong from *raw pixels*!)
Third, install whatever deep-learning framework works for you (I recommend Keras), and just start slapping layers together and seeing if something cool happens. Maybe write a MNIST classifier first, that's like the 'Hello World' of deep learning.
Oh, and here's some great blogs -
blog.otoro.net/ (Messing about with neural networks to do Cool Stuff for fun and art)
karpathy.github.io/ (Not many posts, but a fantastic explainer - check out the Deep Reinforcement Learning article)
colah.github.io/
And the r/MachineLearning subreddit is good for discussion and interesting new papers.
Eldritch Wow, thanks a lot for getting me started on this, I really appreciate that amount of information.
I think the NN would perform much better, if you and more layers to it. Then it takes longer to learn, but it will better understand the rules of languages.
The beauty is achieving something complex with simplest means.
These videos are hilarious.... I love that someone's using ANNs in a humorous way.
I love the Boredom Repellant, it works every time! I think every channel should use it.
Keep it up, love learning more about neural networks from your vids. One small question I have though, is the source code for your math evolution simulator on the Internet?
Thanks! Also, I think this might be what you're talking about: www.openprocessing.org/sketch/377698
Cary, did you know that you can reverse the neural network? aka You can make it produce words that it thinks are a certain language! (I'm not sure how to though, so you'll have to look up how. :\ ) I just know that it's possible to do it.
Thanks! This is exactly what I was looking for :)
The same way you train it, actually! Instead of finding the gradient of the connection weights with respect to the output, you can find the gradient of the *inputs* with respect to the output, and use gradient descent to optimize the input with respect to how german the network thinks it is.
The way the input is structured, though - as one-hot vector encodings, where each character is equivalent to one of the nodes being on and all others off - would be very poorly suited to this, however, since I think you'd be incredibly unlikely to end up with anything close to a valid character.
But that's how things like DeepDream and neural style transfer work - you back-propagate the gradients all the way through the network to find the derivative of the inputs with respect to the output, and optimize the input image to correspond to the desired output activations. (Style transfer - which lets you take photographs and classical paintings and wind up with selfies painted in the style of Starry Night - works by optimizing the target image, as seen by an image-recognition network, on two different levels - you try to match it to the low-level colors-and-textures features of the style image as seen by the earlier layers of the neural net, and the high-level abstract-content-and-semantics features of the source image. )
Cool! That was a little hard to understand (for me at least), but after reading it a few times, I get what you mean!
i don't think the computer recognized the key mash butt rather the pattern of the random generator
BUY OUR BOREDOM REPPELENT NOW! ITS A GLASS PANE WITH A ZOOMING IN FRACTAL!!
With the exceptions of n/m and sh, Japanese typically follows an alternating consonant to vowel structure. This is something that human readers would be able to pick up on pretty quickly, but your algorithm is incapable of seeing patterns like that.
I hate TWOW, I watch everything BUT twow on your channel.
This new TWOW episode is weird.
Keymash has a pseudo-structure derived from the used keyboard's layout, while random has a different structure (that sometimes produces a word in the former's 'dictionary').
keymash and random will of course be completely different, because random will be exactly that, random, with no discernable letter pattern. Out of a near infinite series of 1-15 letter "words" that are randomly selected, all the letters will come up about equally, meaning no discernable pattern.
your neural net says that if x letter is in y position then it is more likely to be z language, and multiple neurons come together to point to one language, so it can tell from sequential letter patterns, where the same letters arise in the same places, which is more likely.
keymash is actually a language, letters that are closer together will be more likely to be pressed, so while its not consciously understanding that, it is more likely that thbvf will come before or after a g than z would.
of course im a spatial reasoner so it would make sense that i would immediately understand the spatial groupings of sequential letter combinations in a synapse network.
also, i apparently paused the video exactly one second before you talked about letter groupings being more likely.
where is download?
jjbinks2 it isn't.
I just posted it online, it's here!
drive.google.com/drive/folders/0B98DkR2AsGZaTnQ1aUFILU85VTg?usp=sharing
Read the readme though.
I'm dutch and I can't really come up with any words ending in "ment".
rendement
***** mentos
***** I don't really care where he considers a brand-name a word. I'm not even sure I consider a brand-name a word.
***** ow ofcourse
Probably on loan from the French.
key mash rarely has 2 letters next to each other making it a good way of ditinguishing them
Laughed my head off at the crazy Swahili moment!
Another brilliant video 👍
after seeing this, I got thinking. what if someone made a bot that could be taught to beat bfdia 5b levels using the same method as this. so it could then be able to beat levels it hasn't seen before.
the only problem is there arent enough levels for it to be able to learn. *wink* *wink*.
but seriously, do you think this might be possible?
It is possible! See sethbling's video of MARI/O
am i the only one here that didn't understand anything in that video and got it in my recommended list?
No. I have no idea what is going on in this video... All I saw was, "AI Sees when you're key mashing." Not sure what the relation is here. What AI? and what is being taught languages? Why is it being taught languages? Why is this significant? Is this one of those games based around being a programmer? Is this a game at all? Why was there a cube bouncing balls on its top surface and suggested it was juggling issues?
His AI. He wrote a neural network (which is a type of AI) designed to distinguish between words from different languages.
As for the other stuff... Yeah I'm still trying to work that out as well. xD
(As a side note I ended up on this video from the first part, which I ended up on from a video on the Unity engine....)
alright guys, thanks for the replies. you're awesome :D
"...and you can never escapeee....... Oh, look at that!"
You can remake Captcha by telling people to mash their keyboards and then tell the difference between random generated and mash :D
What is this and why is it recommended to me?
Do you still accept key mashes?
no
iruyhtiouwvyrhweyo9cqmorcwiungchp9amorogqywea?
Bolgêg indeed.
I don't know, can you tell a key mash from an average UA-cam comment? Perhaps he still does, he just doesn't know it yet.
Ndndhwhiasbbakaosbfbfvejsknsnsnd
That site does have a random password generator and random string generator which generate random letter strings, if you were wondering.
17:02 fun fact: the letter E is the most commonly used letter in dutch, and this fact was even used to break a code once.
what is TWOW?
Ten Words Of Wisdom, carykh's (ir)regular series.
So cringey "the neural network can talk"
A book called "Predicting the unpredictable" has a good chapter on it about random strings of numbers. When creating 'random' numbers, each number does not have an equal chance of being picked, as you would expect if it was random. For instance, if you are picking two digits, '37' is more likely from a human than '66' (or another number were both digits are equal), as people believe they shouldn't repeat digits yet it has an equal chance of being picked randomly. When writing out large strings (100+ digits) people tend to repeat certain strings (eg '6482') much more often than should appear. This kind of demonstrates how terrible the brain is at creating random numbers.
I think the same can be applied to the key mash/random, where, like Cary said, letters close together are much more likely. I would expect letters on the edges of the keyboard (P,Q,L) to be picked less frequently than 1/26, and also possibly some strings of letters are common based on normal keyboard resting style, for example: Gamers might have a higher chance of putting WASD than non-gamers. Would love to analyze the data for this.
In summary, Predicting the unpredictable is a great book well worth reading, and humans are terrible at randomness.
I like how "libra" an actual word showed up in your keymash.
Fifth (not a nazi)
jaska mattila might as well change your pfp. I know it's that peace symbol or whatever, but
you have a bad profile pic if you have to add a warning to a random comment
jaska mattila | but your profile picture?
Just write NOT A NAZI in the image.
Its not a peace symbol either, it's actually the Finnish air force roundel from before 1945.
I wonder how this algorithm would function if you didn't represent the letters as just 1 true and 25 falses. If you instead characterized each letter by its classification and "sound"*.
*So the letter B would have a 1 in a "consonent" value; a 1 in "voiced" value, a 1 in "semilabial" (maybe also a 1 in labial, as it falls under both; though semi-labial is the more specific), and a 1 in a "stop" value.
For more on what all that means: en.wikipedia.org/wiki/Consonant#Features
The idea being that perhaps it would be able to pick out specific sounds and higher level patterns and easily associate them with a specific language. Like right of the bat; if a word is formed by a bunch of consonent-vowel pairings, it might have a higher chance of being japanese, vs a word with tones of consonants in a row it would have a higher chance of being german.
Gotta love zooming in on fractals.
Key Mash is different because in Key Mash all letters seem to be around two common points per string, or a continuous line on the QWERTY setup, because humans generate it and they seem to either punch it over and over in one place per hand, or drag it across the keyboard. However, random is generated by a computer, therefore no pattern occurs.
The problem with German and Japanese is that Japanese mainly uses syllables consisting of either only a vowel (a,e,i,o,u and n) or one to two consonants followed by one or two vocals, but never more than three letters in a syllable. It usually should be fairly simple to detect.
Also your German set is flawed because the mutated vowels / umlauts have simply been stripped of their diaeresis instead of transforming them into their transformed version with an appended 'e', so ä->ae, ö->oe, ü->ue and very important as well: ß->ss.
TWOW, this is amazing, I want to see it guessing all the languages at the same time.
I think that extra nodes providing statistics like % of vowels followed my consonants or similar might help get rid of the word-length problem.
Essentially the entire thing that makes neural networks interesting and powerful, and almost the whole motivation behind the recent wave of deep learning success, is that if you structure your network right and provide enough training data, *you shouldn't have to do that*.
Every past approach to AI required carefully hand-engineered features that required almost as much effort to develop and create as it would have taken to just solve the problem yourself. Deep neural networks, however, can actually use the lower layers to learn their own features, which the higher layers can process to do stuff like classification. It's actually possible for neural networks to learn how to play videogames using only raw pixel inputs!
The trick is, though, you've got to actually be moderately clever in structuring your network, so he'd have to use a recurrent neural network or a 1D convolutional neural network or something else that incorporates the sequential, spatial structure of the problem.
I see you've got an Esperanto option. I like the idea of a universal (or even European) language, so I approve.