1:23 *#1:* Playing audio is easy 3:45 *#2:* The audio mix is hard 8:12 *#3:* Learn to work with sound designers [9:36 Conversion] volume = 10^(dB/20) 11:35 *#4:* Middleware makes bootstrapping quick 16:31 *#5:* Sound is always the first to get the shaft 18:43 *#6:* Listen! 19:23 Post-mortem 21:56 Audio programming is fun! + Q&A
I love how he looked around in the audience to make sure everyone finished taking a photo hahaha. Very clever man, seems like a true master of the craft.
Thats funny about his offhand comment abount stream limits on the xbox 360. I totally remember that instead of the gameplay lagging it would be the audio that would go first, especially remember this when it would have to load new areas, or chunks near player. Happened a lot in Fable 2.
@@nintendude794 Dora the Explorer: Dora's World Adventure (not the other GBA Dora Game)... seen one XM file floating around, not sure how to get the 5 others...
As a guy who was interested and practicing in music producing for several years now but started programming in the web field ~2-3 years ago, only now I decided that it's time for me to merge my two favourite things which is audio and programming and thus focus on audio programming more.
@@earvingallardo1391 been thinking of doing this too. going to be a long road for me (trying to self teach) but this seems like something i really want to do haha
incase somebody missed the code or coming back to get it that he posted on the screen at 9:36 : float dBToVolume (float dB) { return powf(10.0f, 0.05f * dB); } float VolumeTodB (float volume) { return 20.0f * log10f(volume); }
@@sheboltaev Well, you're right, it should be. The problem is that the computer can't really handle infinities. (Well, technically, floating points have the NaN (Not a Number) for anything undefined, but I don't think it woud be really helpful in this case). The solution Charles Parker gives seems good to me (I'm far from being an expert tho). You could also take care of that beforehand (You never need to stream a sound at -inf dB, so maybe you could cut it if it's under a threshold or something, Idk...)
@@ohhnyx9229 Maybe I misunderstood what Charles said, but I was under impression that volumeTodB would already return -Inf for 0 even without any check. Haven't tested though...
@@sheboltaev What I think he meant is that it mathematically would, obviously. The thing is, you *don't* want your function to be like "Yea This goes to negative infinity. I'll return a Not-a-Number and everyone will be happy!", Because it could break things in a lot of situations (if it doesn't throw an error at you in the first place). The simplest example is this algorithm: You have a float A corresponding to a volume. You set A to VolumeTodB(A) You set A to dBToVolume(A) Now, try to find what happens when A starts as 0 : VolumeTodB(A) returns NaN, since it goes to -inf. (or it may throw an error directly.I don't know exactly how C and family handle that, so let's assume it doesn't for the sake of the example) So now, A equals to NaN DbToVolume(A) then returns NaN too, because the log to something undefined is obviously undefined. You end up with A = NaN In other words, you start with a volume at 0, then convert it, deconvert it, and get an undefined volume. Not ideal, right? x) That was for the long explanation. Since you specified that you haven't tested the code, I'm guessing you program at least a little, so you may have understood my second explanation without the example. Was still worth putting it there, just in case someone with no programming knowledge reads this. If you still have questions (or you figure out that I'm totally wrong, could always happen), don't hesitate !
It's funny thinking back to the Sound card wars in the mid to late 90's, all the ads in PC Gamer etc. for Aureal, Creative etc. sound cards/technologies, Vortex etc. some interesting things with 3D Spatial sound and now we've pretty much stagnated.
What a god tier talk. Well done. Lesson #6 is meta af. And I feel like a game such as league of legends has utterly mastered sound mixing within radius
Careful with the 6dB = double. That is true for a voltage or current , but things change for power signals. 3dB is double the power. The ear perceives power, not voltage.
Actually the +6db = Double is kinda tricky subject, depends on what youre talkin about, sound pressure level is indeed +6db, but sound Power level is +3db and loudness level is +10db... Sound designer here Sorry ahhahahahha
Playing sounds badly is nearly free. Playing sounds well is incredibly resource intensive to the point that the nobody even tries anymore. There is only one sound source and only two ears, so how does that generate complexity? The sound can reflect, refract around and transmit through objects and it's all frequency dependent. You've got doppler shifts, you've got the propagation delay before the sound reaches the listener which is different in different media, you have head related transfer functions that simulate the way the shape of the head and ear filters the sound. It gets very complex very quickly as soon as you leave the comfort zone of modern games with bog-standard stereo, or perhaps an inferior quality HRTF and some simple echo filters. Even just a very simple and incomplete implementation of partial wave tracing and with HRTFs for directional implementation, which was done 20 years ago with the aureal vortex 2 chipset, is better than anything we have today. This is a very sad fact, and I hope VR will revive the quest for accurate physical simulation of audio.
Well... they're not *the* shit, but they are shit, as is everyone else since the early 2000's when creative patent trolled Aureal to death and developers stopped trying to do correct sound rendering.
Oh boy, I just searched Aureal Vortex 2 on YT and you're serious. That stuff sounds pretty awesome compared to stuff we got today (not even considering how old the Vortex 2 is). Obviously these kind of virtual 3d simulations have their flaws too (everything sounds "too close") or things like the 3d projection on your screen not matching the location where you actually hear it. Not sure how feasible it would be to get around that, but if those peolple in CG can do awesome sh!t, the audio people should be able to do that too.
I haven't worked on a game in my life but was happy with a lot of things he pointed out. I found them a bit obvious every now and then, but I know most people are very poor at listening. I find this both as a musician and as a language learner. Doing sound well is hard and a lot of people will not consciously notice. It's also the cause of the loudness wars in pop music.
Start with understanding basic audio engineering concepts, like sample rate, bit-depth, channel formats, audio codecs, etc, and also get an understanding of audio mixing concepts like signal buss, effects sends/chains, and various common effects like reverb, stereo delay, dynamic compression, and maybe some type of 'distortion' if your feeling gutsy. As for programming concepts, there will be lots of buffers and queues, so understanding various ways to implement these will help. You will probably need a very reliable method of thread synchronization. Because audio will almost always require streaming from disk, and because you don't want to risk file IO blocking on the audio thread, you will want some sort of lock-free (or at least non-blocking) queue to separate file IO form the audio thread. If you let the audio thread stall, then you will probably get horrible sounding stutters, which are generally considered more annoying than frame-spikes. You will also need a way of interfacing with the sound hardware... this could be done through your operating systems API (SDL, ALSA, etc), or from various other libraries like OpenAL, FMOD, etc. (Though, the higher level library you rely on, the less control you have, and the more of it's issues you are stuck with.) Once that works, then you could move on to some of the more advanced math subjects, like Nyquist Theorem and Fast-Fourier-Fransforms. These concepts will help in implementing effects like pitch shift, doppler, EQ, etc... Good luck!
@@stephenborntrager6542 It's obligatory having a strong calculus background to learn about Fourier transforms and trying to implement those myself? I'm actually in first year of computer engineering but that stuff is being taught in like 4th year or 5th year. Is it realistic for me to try going for those subjects?
@@Caglarcomposes If you're a sound designer/composer and learning on your own, I'd recommend Wwise - their tutorials are better. Once you learn one, you'll be able to pick up the other without much trouble. It's an oversimplification, but Wwise has kept up with more advanced features for the sound designer - stuff like an integrated audio occlusion/spatialization system that scans your 3D meshes, better 1st party plugins, more flexible parameterization and 3D options - you can tweak and randomize things like detailed 3D positioning without messing with your game engine, for example. Basically you can do more with less code. The pricing options are different as well. FMOD is more generous in this regard if your budget is less than $500k US.
In another video I saw a similar comment and the speaker replied that people were laughing, the mic is just set up to only pick up the speaker’s voice.
So like I have experience in sound design and mixing but I want to get a job in this field. What course should I take to merge code and music/sound design?
I'm just bad at audio stuff to where I'm fairly certain this just isn't a talk for me. Like the other day a DJ said that spotify had bad audio quality, and I still don't hear it. A shaky camera and I'm done though because I'm more visually artistic. HAHAHA this is a job? HAHAHA I'm lost.
Paputsza it’s a hard task and most people don’t even realise it’s important until it’s too late and we save their asse(t)s near the finish line of their project.
trust me i am a sounddesigner and 90% of sound designers, that imply they hear the minutiae of sound are full of shit. There is plenty of blind test that have shown, that anything above 192kbit/s audio quality is almost inaudible to an uncompromised audio - now translated: spotify usually plays at 192 or 320 kbit/s, while uncompressed audio plays at 1,4 million kbit/s. So if an sound designer tells you spotify sounds like "shit", well... they are full of shit. They just insecure about their hearing capabilites and must flex on others, but i guarantee you if you do blind testing on spotify premium vs cd at same volume and lufs, the difference will be inaudible. I challenge every audio engineer to this test, they will fail (cuz it has been done before and even the best couldnt diff. 320kbit vs 1,4mil kbit)
You dont hear anything. Then great because they're spending too much money to really care whats really matter like buy flac 24 bit 192 kHz for little improvement is just ridiculus
Thought I'd learn about audio programming, how hard can it be right? Now I'm sitting here looking at differently colored letters, symbols and numbers and I don't know what the hell is going on. Guess this isn't the day I learn code.
Learning the grammar of a programming language is easy, doing something useful with it takes research and experience as well as a logical mind. It's the difference between being able to sketch some doodles with a pencil and being able to sketch a landscape scene. This is a guy who has spent over a decade learning/building experience in audio programming, you're not going to get to his level (or any other professional) without effort.
Just saying... don't ever say the words "NO" to a creative... especially an audio engineer. They know what they want when they ask for it. Nos will always create tension between the two of you. Word choices are very important. Us audio people are very stubborn, "NOs" get under our skin almost immediately. Maybe instead, ask them "Would there be something else we could create for you to use to make this task better/easier than *insert current solution here*?"
1:23 *#1:* Playing audio is easy
3:45 *#2:* The audio mix is hard
8:12 *#3:* Learn to work with sound designers
[9:36 Conversion] volume = 10^(dB/20)
11:35 *#4:* Middleware makes bootstrapping quick
16:31 *#5:* Sound is always the first to get the shaft
18:43 *#6:* Listen!
19:23 Post-mortem
21:56 Audio programming is fun! + Q&A
I love how he looked around in the audience to make sure everyone finished taking a photo hahaha.
Very clever man, seems like a true master of the craft.
Oh
I like the pacing of this presentation. Very efficient delivery.
Thats funny about his offhand comment abount stream limits on the xbox 360. I totally remember that instead of the gameplay lagging it would be the audio that would go first, especially remember this when it would have to load new areas, or chunks near player. Happened a lot in Fable 2.
I was sold on watching this since he showed he worked on Dora the Explorer.
Is that the one with a Virt soundtrack with songs in XM format? :3
@@boptillyouflop now I want to hear this
@@nintendude794 Dora the Explorer: Dora's World Adventure (not the other GBA Dora Game)... seen one XM file floating around, not sure how to get the 5 others...
As a guy who was interested and practicing in music producing for several years now but started programming in the web field ~2-3 years ago, only now I decided that it's time for me to merge my two favourite things which is audio and programming and thus focus on audio programming more.
Where are you in the transition now?
floating in the vibe
How is it going, I'm thinking of doing that in college.
I just started doing this, been the happiest I’ve ever been.
@@earvingallardo1391 been thinking of doing this too. going to be a long road for me (trying to self teach) but this seems like something i really want to do haha
10:22 - *NO* -. This answer solves almost every problems.
incase somebody missed the code or coming back to get it that he posted on the screen at 9:36 :
float dBToVolume (float dB)
{
return powf(10.0f, 0.05f * dB);
}
float VolumeTodB (float volume)
{
return 20.0f * log10f(volume);
}
Oldsiren I would add a check for volume==0 in the volume to db function, because this will be -Inf
@@charlesparker6167 But it should be -Inf in that case, shouldn't it? I don't understand.
@@sheboltaev Well, you're right, it should be. The problem is that the computer can't really handle infinities. (Well, technically, floating points have the NaN (Not a Number) for anything undefined, but I don't think it woud be really helpful in this case).
The solution Charles Parker gives seems good to me (I'm far from being an expert tho). You could also take care of that beforehand (You never need to stream a sound at -inf dB, so maybe you could cut it if it's under a threshold or something, Idk...)
@@ohhnyx9229 Maybe I misunderstood what Charles said, but I was under impression that volumeTodB would already return -Inf for 0 even without any check. Haven't tested though...
@@sheboltaev What I think he meant is that it mathematically would, obviously. The thing is, you *don't* want your function to be like "Yea This goes to negative infinity. I'll return a Not-a-Number and everyone will be happy!", Because it could break things in a lot of situations (if it doesn't throw an error at you in the first place).
The simplest example is this algorithm:
You have a float A corresponding to a volume.
You set A to VolumeTodB(A)
You set A to dBToVolume(A)
Now, try to find what happens when A starts as 0 :
VolumeTodB(A) returns NaN, since it goes to -inf. (or it may throw an error directly.I don't know exactly how C and family handle that, so let's assume it doesn't for the sake of the example)
So now, A equals to NaN
DbToVolume(A) then returns NaN too, because the log to something undefined is obviously undefined.
You end up with A = NaN
In other words, you start with a volume at 0, then convert it, deconvert it, and get an undefined volume. Not ideal, right? x)
That was for the long explanation. Since you specified that you haven't tested the code, I'm guessing you program at least a little, so you may have understood my second explanation without the example. Was still worth putting it there, just in case someone with no programming knowledge reads this. If you still have questions (or you figure out that I'm totally wrong, could always happen), don't hesitate !
1:25 "Playing audio is easy"
* me who was introduced into audio programming back in 1994 *
Rrrrrrrrigh
It's funny thinking back to the Sound card wars in the mid to late 90's, all the ads in PC Gamer etc. for Aureal, Creative etc. sound cards/technologies, Vortex etc. some interesting things with 3D Spatial sound and now we've pretty much stagnated.
What a god tier talk. Well done. Lesson #6 is meta af. And I feel like a game such as league of legends has utterly mastered sound mixing within radius
Shout out to the one guy in the audience that learned audio programming lmao
Careful with the 6dB = double.
That is true for a voltage or current , but things change for power signals. 3dB is double the power. The ear perceives power, not voltage.
haha yeah he seemed like he knew he was slipping up slightly there
usually the magnet is stationary and the coil is attached to the cone
Actually the +6db = Double is kinda tricky subject, depends on what youre talkin about, sound pressure level is indeed +6db, but sound Power level is +3db and loudness level is +10db... Sound designer here Sorry ahhahahahha
>Sound designer here sorry haha
It's time for you to stop.
@@benjiusofficial nah let him laugh like supervillain, it's enjoyable
i love how the talk about sound has the worst audio problem
that's your crappy phone speakers ;P
Its the unspoken law of youtube
the classic "hairdresser with bad haircut" type problem, guess the audio fellas are too busy giving the talks rather than setting them up hahaha
Playing sounds badly is nearly free. Playing sounds well is incredibly resource intensive to the point that the nobody even tries anymore.
There is only one sound source and only two ears, so how does that generate complexity? The sound can reflect, refract around and transmit through objects and it's all frequency dependent. You've got doppler shifts, you've got the propagation delay before the sound reaches the listener which is different in different media, you have head related transfer functions that simulate the way the shape of the head and ear filters the sound. It gets very complex very quickly as soon as you leave the comfort zone of modern games with bog-standard stereo, or perhaps an inferior quality HRTF and some simple echo filters.
Even just a very simple and incomplete implementation of partial wave tracing and with HRTFs for directional implementation, which was done 20 years ago with the aureal vortex 2 chipset, is better than anything we have today.
This is a very sad fact, and I hope VR will revive the quest for accurate physical simulation of audio.
Honestly, if you want to hear good sound, Nintendo games on the WiiU are the shit.
Well... they're not *the* shit, but they are shit, as is everyone else since the early 2000's when creative patent trolled Aureal to death and developers stopped trying to do correct sound rendering.
Do you even know what I'm talking out?
Oh boy, I just searched Aureal Vortex 2 on YT and you're serious. That stuff sounds pretty awesome compared to stuff we got today (not even considering how old the Vortex 2 is). Obviously these kind of virtual 3d simulations have their flaws too (everything sounds "too close") or things like the 3d projection on your screen not matching the location where you actually hear it. Not sure how feasible it would be to get around that, but if those peolple in CG can do awesome sh!t, the audio people should be able to do that too.
@@theIpatix I suspect that how well 3d audio matches depends on geometry of head+ears and the HRTF used in audio engine.
I was pulling my hair out until 13.10 (initialization of 3D sound causes POP sound). THANK YOU!
I haven't worked on a game in my life but was happy with a lot of things he pointed out. I found them a bit obvious every now and then, but I know most people are very poor at listening.
I find this both as a musician and as a language learner.
Doing sound well is hard and a lot of people will not consciously notice. It's also the cause of the loudness wars in pop music.
feels good to know both the languages 8:45
So now a question: If I want to get into engine development, and am tired of practicing graphics code, what should I learn for audio programming?
Start with understanding basic audio engineering concepts, like sample rate, bit-depth, channel formats, audio codecs, etc, and also get an understanding of audio mixing concepts like signal buss, effects sends/chains, and various common effects like reverb, stereo delay, dynamic compression, and maybe some type of 'distortion' if your feeling gutsy.
As for programming concepts, there will be lots of buffers and queues, so understanding various ways to implement these will help.
You will probably need a very reliable method of thread synchronization. Because audio will almost always require streaming from disk, and because you don't want to risk file IO blocking on the audio thread, you will want some sort of lock-free (or at least non-blocking) queue to separate file IO form the audio thread. If you let the audio thread stall, then you will probably get horrible sounding stutters, which are generally considered more annoying than frame-spikes.
You will also need a way of interfacing with the sound hardware... this could be done through your operating systems API (SDL, ALSA, etc), or from various other libraries like OpenAL, FMOD, etc. (Though, the higher level library you rely on, the less control you have, and the more of it's issues you are stuck with.)
Once that works, then you could move on to some of the more advanced math subjects, like Nyquist Theorem and Fast-Fourier-Fransforms.
These concepts will help in implementing effects like pitch shift, doppler, EQ, etc...
Good luck!
For me, what did it was writing some VST plugins (and Buzz plugins back where Buzz was a thing).
@@stephenborntrager6542 It's obligatory having a strong calculus background to learn about Fourier transforms and trying to implement those myself?
I'm actually in first year of computer engineering but that stuff is being taught in like 4th year or 5th year. Is it realistic for me to try going for those subjects?
09:57 is what you are looking for
Thanks man for this vid! :) Very helpful
Lesson learned: Use FMOD
Wwise is better ;)
@@DukeJon1969 Why is Wwise better in your opinion? I am trying to prioritize learning one of them, so if you'd explain, I'd be happy
@@Caglarcomposes
If you're a sound designer/composer and learning on your own, I'd recommend Wwise - their tutorials are better. Once you learn one, you'll be able to pick up the other without much trouble.
It's an oversimplification, but Wwise has kept up with more advanced features for the sound designer - stuff like an integrated audio occlusion/spatialization system that scans your 3D meshes, better 1st party plugins, more flexible parameterization and 3D options - you can tweak and randomize things like detailed 3D positioning without messing with your game engine, for example. Basically you can do more with less code.
The pricing options are different as well. FMOD is more generous in this regard if your budget is less than $500k US.
very thorough overview of audio programming, might be true not only for a game audio engine...but what seriously happened with that hand at 18:08
why does it sound like your voice is being parallel processed by an aliased spectral resynthesis engine
great talk!
This dude cracking jokes all the time and nobody laughs smh...
In another video I saw a similar comment and the speaker replied that people were laughing, the mic is just set up to only pick up the speaker’s voice.
Audience isn’t mic’ed up
This is a video about audio production tools, but it looks like some of the audience might not understand how microphones work.
The mood is somber-g
What a legend.
So like I have experience in sound design and mixing but I want to get a job in this field. What course should I take to merge code and music/sound design?
the coil is fixed to the cone, suspending inside the magnet which is fixed to the speaker housing.
it's the coil that moves the cone, not the magnet.
13:00
Programmer time and priorities => Sound < AI < Gameplay < Graphics :(
aloluk in terms of immediate noticeability, this is also the order.
Sound < AI < Gameplay < Graphics < Lootbox Roulette Screen
is this C
Same goes for theme parks and experiences.
Big budgets for the rendering machines and projectors. Scraps for the audio system.
Hey Guy
I'm just bad at audio stuff to where I'm fairly certain this just isn't a talk for me. Like the other day a DJ said that spotify had bad audio quality, and I still don't hear it. A shaky camera and I'm done though because I'm more visually artistic.
HAHAHA this is a job? HAHAHA I'm lost.
Paputsza it’s a hard task and most people don’t even realise it’s important until it’s too late and we save their asse(t)s near the finish line of their project.
Audio coder is a job and I've done it. And yeah, I can do the thing he talks about where you can figure out the type of sound bug by ear.
trust me i am a sounddesigner and 90% of sound designers, that imply they hear the minutiae of sound are full of shit. There is plenty of blind test that have shown, that anything above 192kbit/s audio quality is almost inaudible to an uncompromised audio - now translated: spotify usually plays at 192 or 320 kbit/s, while uncompressed audio plays at 1,4 million kbit/s. So if an sound designer tells you spotify sounds like "shit", well... they are full of shit. They just insecure about their hearing capabilites and must flex on others, but i guarantee you if you do blind testing on spotify premium vs cd at same volume and lufs, the difference will be inaudible. I challenge every audio engineer to this test, they will fail (cuz it has been done before and even the best couldnt diff. 320kbit vs 1,4mil kbit)
You dont hear anything. Then great because they're spending too much money to really care whats really matter like buy flac 24 bit 192 kHz for little improvement is just ridiculus
awesome
1000% TRUE
i.e. use FMOD
Damn this guy lost his job
why doghead
@@SuperTrisset Telltale shut down like a month ago
He left Telltale in 2015.
Haven't we all?
Teach us how to make a online mastering robot
Thought I'd learn about audio programming, how hard can it be right? Now I'm sitting here looking at differently colored letters, symbols and numbers and I don't know what the hell is going on. Guess this isn't the day I learn code.
Learning the grammar of a programming language is easy, doing something useful with it takes research and experience as well as a logical mind. It's the difference between being able to sketch some doodles with a pencil and being able to sketch a landscape scene. This is a guy who has spent over a decade learning/building experience in audio programming, you're not going to get to his level (or any other professional) without effort.
So 12 years of working on 300 lines of code.
Lol, no. That was just the minimum he ever shipped. He talked about a bunch of other things that could be done
I'm sure he's knowledgeable, but what's the takeaway from this talk?
Just saying... don't ever say the words "NO" to a creative... especially an audio engineer. They know what they want when they ask for it. Nos will always create tension between the two of you. Word choices are very important. Us audio people are very stubborn, "NOs" get under our skin almost immediately. Maybe instead, ask them "Would there be something else we could create for you to use to make this task better/easier than *insert current solution here*?"
What are you, four years old? You’re a grown fucking adult, you can be told no especially if you’re asking for something.
@@chonchjohnch You missed the point. By a long shot. Like, I'm confident you didn't read between the lines in anything he said.
Tell them no and give them trash?
Leaving.
what
what
underwhelming