Sonification & The Problem with Making Music from Data
Вставка
- Опубліковано 14 чер 2024
- This video takes a brief look at the subject of Sonification (AKA Parameter mapping or Audification) and how it is used for better or worse by scientists and musicians around the world. Some people (including those at NASA and CERN) have created sonifications for Mars, Jupiter, Saturn, Space. Others have looked inwards, focussing on things like social causes and sonifications of well known events. Some of it is good. Some of it is awful!
My Patreon page:
/ tantacrul
Here's a Journal Article that makes a similar point about the weakness of converting data to music without any insight or interpretation:
www.james-saunders.com/2016/05...
Higgs Boson Sonification Video
• Listening to data from...
Using the Sun to ‘Make Music’
• Using the Sun to Make ...
cse.ssl.berkeley.edu/stereo_so...
Further Reading:
sonification.de/
sonification.de/projects/eegso...
www.scientificamerican.com/ar...
cse.ssl.berkeley.edu/stereo_so...
www-pw.physics.uiowa.edu/voyag...
www.revealnews.org/blog/a-son...
www.nasa.gov/mission_pages/ca...
saturn.jpl.nasa.gov/
EDIT: During my description of how radio waves from the Cassini spacecraft were translated into sound, I said that the 'wavelengths were shortened'. This was a mistake where I mixed up wavelength and frequency. It was the frequency that was reduced (by a factor of 44), which would have the effect of actually increasing the wavelength. Apologies.
This video gave me gose bump.
Was about to trigger at this. Then I got to 6:30 :).. Well done, Sir!
You are a big joke that's my description to you
you’re another disgusting American
Hahaha😉
Tantacrul You're another disgusting gose bump, that's my big description to you. You should joke yourself.
I synthesized some data without transforming it and the results were very good.
The data was Beethoven's Fifth Symphony.
😀
I tried the same with some guy called Justin Buiber or something like that. Did not turn out well. No regrets though. It's important to experiment!
@@georgebrown1807 I actually did a piece sonifying the Fibonacci Sequence in midi, each bar a number in the sequence, four digits in base 12. soundcloud.com/sleuth-slime/fibonacci-sequence
@@georgebrown1807 Woah that's really cool
@@jeffoneto278xd @Kelly Kaufman Isn't it? I love the idea that at some future point in history, long after the apocalypse when all current forms of media are long gone, some survivor discovers a record of this long numerical sequence and somehow manages to reconstruct The Fifth from it. Okay, a fair few logical leaps there, but it's a cool idea for a story. :)
I made a live stream a while ago titled "The Sounds of Space." It was just a picture of Saturn with no sound.
it eventually got taken down by youtube
Well, you can't steal John Cage's music.
better not have lasted 4 and about a half minutes
edgy af
@@ShirubaGin sorry fellas silence doesn't enter the public domain until 2062.
not the hero we deserve
There's a great description of real sonification in Dirk Gently's Holistic Detective Agency (the original novel by Douglas Adams). Richard MacDuff feeds a bunch of data, about the migration and aerodynamics of swallows, into his conversion program:
"The result was a short burst of the most hideous cacophony, and he stopped it.
He ran the conversion program again, this time instructing it to force-map the pitch values into G minor. This was a utility he was determined in the end to get rid of, because he regarded it as cheating. If there was any basis to his firmly held belief that the rhythms and harmonies of music which he found most satisfying could be found in, or at least derived from, the rhythms and harmonies of naturally occurring phenomena, then satisfying forms of modality and intonation should emerge naturally as well, rather than being forced. For the moment, though, he forced it.
The result was a short burst of the most hideous cacophony in G minor."
Douglas Adams was a very very bright beacon. He was also close friends with Richard Dawkins and appeared in his Christmas Lecture!
Reading the 2nd Dirk Gently right now (:
And then the aliens made Bach music out of data.
I loved that book, and Adams' writing style in general. I used to hate writing like that in middle school, but I changed my tastes after a while.
I still celebrate towel day, years later.
Which is strange because it should technically work. The examples at 10:13 and 12:09 make use of what is known as pan-diatonicism, which takes into account the fact that constructing harmony using only diatonic notes is generally pleasing to the ear, even if these notes turn out to be randomly selected. From a musical standpoint it's a clever way to obscure data, but without any context you wouldn't know that such music holds any significance to the data to begin with.
@@warbler4954 that's not really how music works, you can set it up with a synth to sound like ambiant noise, but if you try and make any sort of melodic music, randomly selecting notes in a key will still sound as bad as randomly playing white keys on a piano (which is technically playing perfectly in C major)
Could the sonification help you find out if a swallow would be able to carry a coconut, though?
I don't think that CERN researcher wanted to be giving that talk very much, she seemed to recognize that there was nothing quantifiable valuable to gain from it and seemed like she wanted to just come out and say "we made a noise with some numbers from our big machine, it doesnt mean anything.
I am impressed by the astronomer though, I've always thought that observational astronomy would be harder for blind people because everything is measured in terms of light essentially.
I felt bad for her ngl
What she was doing actually wasn't observational astronomy (you're correct in thinking that observational astronomy would be more difficult for blind people, because it would be near-impossible). Modern astronomy tends to a lot more often be looking at data from very very far off celestial objects and inferring things about the objects from those data; as discussed in the video, this is something that sonification excels at helping with.
This is every TED talk.
Really feels like she was the one who drew the short straw and had to get up on stage to say "here is a thing we did that really isn't very noteworthy". She very much looked like she didn't want to be there, although it was probably stagefright. That was a big room and it was pretty packed, she had probably never given a presentation in front of so many people. I'd have been nervous as hell.
I once worked at an academic institution, that built a large wave tank, primarily for the purpose of studying......big waves......I think.....
A bit unkind - it was used for various marine biology courses and the study of tidal mechanics on shorelines. Really interesting stuff.
There was however, one particularly egregious use of research funding - a mathematics PhD used research funding from a computer music research group to produce a piece of music using pressure sensors as midi triggers, and lining the wave tank with them, the idea being that it would of course produce audible patterns, and a "piece of music" at the end. No hypothesis was tested, no new information gained, and the end result, while challenging to the definition of what we consider to be artistic and musical intent, was......nonsense.
Real, tangible research money that could have been used to study the effects of music on synaptic density in brain damage sufferers.....was used to create something even less intentionally musical, and more tediously mathematical, than serialism.
Fuck.
> "Boson sound" plays
> Unimpressed audience
This made me laugh
Should've made a thumbnail with massive text "this is NOT the sound of Saturn" with red circles and arrows everywhere.
Lol. Might do!
that sonification of the higgs talk, omg, that must be the saddest thing I've seen in a long while.
lol.
Didn't she mention, that this is paid from our taxes? (I live in the EU).
Like, she knows she's got nothing useful.
most TED talks are pointless and sad. "i did a thing, heres what it was... ok... thats all... bye." They used up all the interesting presenters pretty early on in their run.
@@fakiirification Ted X is like that at least. I haven't seen an actual Ted video in ages so I don't even know. Seems every ted talk these days is rubbish tedx nonsense
I think a lot of this has to do with the way "pop-science" tries to make science cool and interesting for the layman, and as a result, ends up losing everything that makes science valuable.
People with absolutely no curiosity, being tricked into thinking they know a thing about anything because they like the pretty pictures and sounds that show up on their basic-bitch pop-sci blogs.
Sounds like almost every TED Talk, or the social mediafication of science. People want to feeel smart, without actually having to study or read for decades.
@@the81kid Isn't that what society is for, though? Some people want to live in a house without going to carpentry school.
@@thegardenofeatin5965
They don't live in a house built by someone else, and feeel they are a carpenter.
All three of you have aliases that start with 't'.
On the other hand about science, some of it *is* really cool, at both basic and deep levels. I remember reading in depth about shape memory alloys and polymers. Absolutely fascinating. I even started to assist with some experiments on them at the local university while I was in high school. Unfortunately my transportation was very spotty and I wasn't able to keep helping.
@@adamofblastworks1517 Thank you AdamofBlastWorks, very cool!
I really enjoyed your reimagined heart rate monitor as a sawtooth synth. I think it really drove your point home in one soundbite.
When you lazily recreate a sound - someone, somewhere will know exactly what you've done :)
You mean Importing an image of the Mandelbrot Set into Metasynth and playing it doesn’t make me a genius?
I mean, does it sound good?
@@baronvonbeandip If it sounds good then it aint dumb
Doesn't say anything about the intelligence of the composer though
Congrats,
You are now aphex twin.
Now put it in a song.
Wait I actually want to try that now, just as an expirement
If you actually wrote the software metasynth, yes.
This is so on point. I frequently use sonification for monitoring digital radio traffic, because the data makes more sense when represented as audio. Inevitably I get the questions though "What does WiFi sound like?", "What does 5G sound like?", "What does Bluetooth sound like?". The respective answers are: random clicking + white noise, white noise, and random clicking. But then everyone wants to try to "dress it up" as music, when that defeats the point of both sonification, and music.
I remember starting out with amateur radio, and was astonished how "musical" things could be (when demodulated by product detector like in every amateur radio and fed to audio out). I get some dopamine when I flip the coax switch, the power meter wents up, and the characteristic noise (filtered white) comes out of speakers. Quickly learned to distinguish popular emissions like SSTV, RTTY (which is just 2FSK) and even I know the characteristic whistle of PSK31. As much as the community is divided about FT8, since it's MFSK it makes beautifull sounds.
But ye, you are right, with high symbol rate and packet structure, there is something like massive FM synthesis going on (if we stick to audio terms), so it sounds like noise.
The CERN presenter looks awkward when she says arbitrary. It’s like she knows she selling you a dodgy car.
And when she says it was paid for by your tax.
They should have hired a Yank, not a Brit. Would have had no problem lying.
@Star Trek Theory I mean they are creating a time travel device to create a single government world order so...
@@whereammy Just stopping by to point out that this comment is great.
Yeah, if you've spent most of the TED talk hyping up your cool research, it's going to be awkward when you admit that it has few real-world applications. It'd be like if the car salesman had to follow up their pitch with saying they are legally required to tell you that the vehicle you are about to purchase does not meet the official standards of what constitutes a "car"
My tinnitus kicked in at the exact moment the Jupiter sound came on. It took me embarrassingly long to seperate the two.
Ouch
7:29 Congratulations! You have died!
Perfect game over sound.
XD
I've been laughing at a dead guy for a whole minute now.
Congratulation
You have cancer
r/cursedcomments
I’d like to see a dozen different artists independently create sonifications of a data set like the rings of Saturn one to prove a point on how arbitrary it truly is.
Somebody get Rings Of Saturn on the phone
Actually, don't. Lucas Mann is a hack.
actually it doesn't matter how arbitrary the sonification is - if you got a dozen artists to plot the data visually you might see a dozen different plots. The point with sonification is that it has the potential to reveal something in complex noisy data that just plotting it might not reveal. To get there you might have to assign parameters in a dozen different ways and if you're lucky you might hear in one of them that one surprising blip that leads you to ask a new question of the data, just as when plotting data you might see nothing until you try log scaling , filtering outliers and whatever else you might throw at it. The issue with sonification is not that the assignment of parameters is arbitrary, it's that the results, however cool they might sound, must never be made available to journalists.
If you can sonify climate data, then you can climatize sound, too. Imagine the possibilities for musical scoring! Instead of hard to read ledger lines, we could just print the year number. Sibelius should add a menu for this.
Sonification has saved lives. There was a nuclear reactor, where they took all the signals from the reactor and assigned it audio properties, played it out the speakers, and people got used to how the reactor was “supposed to sound”. When there was an issue, they didn’t notice it on their visual representations, but they could hear the difference, performed extra checks, and found issues.
This is only kinda true, as they played the actual sound from the reactor into the control room.
That makes a lot of sense to me. I work as a cashier, and I have the whole checkout process down to a routine that I follow without thinking. We have to be really careful when people pay with credit or debit cards, because the machine to do that is entirely separate. Sometimes I breeze through the checkout process so quickly, that I don't see when something goes slightly wrong. But I DO pause and feel like something is wrong, only to realize a moment later that it made a different sound and that means I need to go back and change something.
Another example is with putting in phone numbers, in the days before we all had our contacts in smartphones. When you type in someone's number, you'd know if you'd done it wrong because it would just sound off. That's the kind of thing that would be easy to miss if you just glance over the written number, but the sound will absolutely make you double take.
Sound is so helpful in so many ways, it's a wonderful indicator when something is slightly out of the norm and you should take a second look and check to be sure.
Source?
@@charliefranklin8523 The only similar story I've heard was in the fusion experiments at JET. Here's the tom scott video I learned that from: ua-cam.com/video/IrtGp8hv-0Y/v-deo.html
Oh! That car engine is sounding rough.
That's not sonification.
Before the Internet, in the late 70s or early 80s, the book _Our Universe_ included an additional activity kit with a record pressed on a sheet of plastic with the frequencies of the planetary orbits sonified (back then we still had nine planets), as well as the magnetic fields of Earth and Jupiter, solar prominences, and the solar wind, among other things. But they were honest and said that the music was an artistic interpretation.
First things first: Sorry for my english, my basic skills are a serious limitation in expressing what I have in mind...
As a researcher which sometimes works with sonification I have to say that you made some good point in this video, but by saying "Sonification: The Problem with Making Music from Data" you're actually messing things up in the same way people you criticize does. I mean, sonification and music should be treated separately, or at least it should be pointed out clearly if one is using music as a technique of sonification (someone calls this "musification") or he is using data as inspiration for a composition (which has nothing to do with sonification at all), because those are very different things!
In other words: your title sounds like "Histograms: the problem of making paintings with data". Basically you are talking about two topics: "Sonification" (the problem of conveying information with sound and sometimes music) and - let's call it - "Data driven composition" (the problem of making music with data).
Anyway, very nice video, it is the first time I see someone outside the field that tries to convey the topic correctly...
and yes, I also want to cry each time I read things like "this is the sound of ###" :D
I see what you mean although I can imagine electro-acoustic composers taking issue :) I'm personally happy enough with using the blanket term 'sonification'. Everything else: data mapping, audification, etc. seem to me to be sub-categories. The use you put it to determines whether it's music or simply data representation. to be honest, I don't mind. I'll just bend with the wind on this one.
> "Histograms: the problem of making paintings with data"
Dunno, but check some examples out here, some are quite impressive! www.reddit.com/r/dataisbeautiful
This was helpful! #EASL
Well said. I’ve been talking to a professor who is using acoustic energy to map blood vessels and control targeted treatment absorption. Actual audio data is important to his work, I don’t think he would much like it if people took the data and when “the music of our blood” or some garbage like that. Honestly, the media has no idea how to read scientific journals.
When I go to ICAD, I often get the sense that everyone is interested in making sound, nobody is interested in listening - it drives me nuts a lot of the time.
It's like saying you are making scientific diagrams but instead wildly spray paint onto canvasses, ending up with some mixture of colors and a story, without achieving any transparency on the source data whatsoever.
Often, it already starts with the sound system being really poorly set up in the presentation room as if nobody actually cared about that part.
A part of the problem is of course that this aspect of sonification (listener involvement) is really difficult to peer review.
BTW I really hate that CERN "Beethoven's 5th" video, because I KNOW you could make sonifications of the CERN data that are more than just "funny sound with a story". I hate it when scientists say sonificaton is "arbitrary". It's just that they don't invest enough effort into optimizing the specific way sound synthesis and listening are applied.
IT IS COMPLETELY ARBITRARY, the sounds you get out........ really
Well of course the *sound* is arbitrary...
Raw data doesn't have inherent, shall I say... *timbre* ...
He wrote "timbre" but he meant "tamber"
@@wedaringu667 its spelled timbre
i think "using data to make music that doesn't really express the data" is fine as long as you aren't trying to express the data and just wanan make a cool sound
@@stooge389 lol
then don't advertise that it IS expressing the data LMFAO
Your comedic timing and editing is on point.
Probably rythmic education.
TANTACRUL! you're one of the (if not THE ONLY) youtube educators that routinely has me laughing out loud alone in my apartment. Keep up the good work. :)
My master's degree thesis was on producing a sonification tool and testing how well users could understand it compared to existing visualizations of the same data. I think you've hit the nail on the head on where sonification goes wrong and what it's potential is. I don't think it's _quite_ as bad as you make it seem, though, the sonifications that are about making data understandable instead of tHe SoUnDs Of ThE UnIvErSe are a thing that scientists are working on too.
It's too bad that the good uses of sonification don't make it into the news :/
Sonification definitely has potential, i know that one of the coolest moments of my EE undergrad was when we used matlab to play the results of all our math to the speakers. (This is because all the math we did was intentionally already in the range of human hearing)
And i think that something like shifting the frequencies of radiowaves to human hearing has value, atleast in the same sense that false color maps have value. Yes that isn't the exact thing you are looking at but it's a lot easier to interpret, just be honest that this isn't art or even exactly what you started with, just a new way of "visualizing" the data to make interpretation easier.
the natural frequency of the universe is 432 Hz
what a meme
I'm with 440 gang, tho both make me nauseous.
I just like 432 because it sounds more pleasant , but really I don't care all that much , despite being somewhat of a snob (which I'm not really proud/ashamed of so don't eat me alive)
pi is exactly 3
What's that shit at 432.1hz though?
Personally, one of the best uses of sonification is Jake Chudow's - hydrogen (Vsauce music). It uses a sound based on the emission spectrum of hydrogen and builds around it, not just uses it in isolation
its basically a jake chudnow song but with a sonification sound effect nothing wrong with that case tbh
hey vsauce michael here
I think the problem is thinking of this as scientifically useful in the ways these are done, but I think there's artistic validity to these types of projects
I agree. To the extent that the critiques in this video are aimed at certain uses of sonification ("the sun sounds like...") relative to scientific inquiry, then these critiques seem reasonable. But as to artistic merit, that's a different story. It sounds like the video is saying that the artistic examples he shows are shallow (with the exception of the gun death one). That could be, but that's more a difference in taste rather than "the problem" with making music from data. I'd love to see a video by artists who can weigh in with their own expertise in the arts.
I think people don't realize how arbitrary it is in data visualization which we took for granted.
allenloves: except data visualisations aren't passed off as actual representations of the things they're measuring, as is the case here. I'm not saying arbitrariness is automatically bad, I'm saying it's bad if you're presenting it as non-arbitrary. Bear in mind that I'm not attacking the discipline of sonification, I'm criticising its misuse.
Unfortunately when we look at a pi chart or bar chart, we don't even think twice about this is a presentation of a data set. It is not only passed of but considered by default as actual representations of the things they are measuring. However it is not less arbitrary than mapping a data into pitch or duration. It is the same when people look at a run chart saying this is what a stock market looks like, and listen to a crappy sound and say this is what Higgs boson sounds like.
Well, that's not accurate though is it? There's a very obvious emotional difference in the mind of the observer between a graph and a series of abstract sounds. If a normal educated person is presented with a graph representing, say, the properties of Jupiter, they're going to understand that what they're looking at is an abstract representation. If they're shown a series of abstract sounds - without any context surrounding how the sounds were created - they could be led to believe that they're actually 'listening' to the thing being represented. Furthermore, if you improve the visual quality of a pie chart, it's still going to be understood as a pie chart. If you add spatial effects and reverb to data relating to a planet, you're mimicking it's imagined properties and obscuring the functional purpose of sonification. And this is easily illustrated by looking at how the 'sounds' of space are commonly presented to the media. I hear this comparison of visual graphs and sonification all the time. Sometimes it makes sense to do so but not in this case. When Carla Scaletti made the comparison, she was arguing that people require education on how the process works in order to understand it. She wasn't saying visual data is the same as audio data in all circumstances.
@@Tantacrul You could say that our sound perception is arbitrary since it doesn't even capture the whole spectrum and is biased. You can hear part of the thing, but not necessarily all of what's there to hear. For now we're bending data to sonic properties, but if we become cyborgs some day perhaps we would be able to experience data in more raw way.
True, but if public is presented with a picture Jupiter, they might take for it granted, and not think how it is color-corrected, edited, etc. Same goes for images of nebulas. They are artistic representation of them, and I think they have value as such...@@Tantacrul
there was a team of scientists who tried using sonification (with the help of CRISPR) to help detect genetic disorders. they assigned some piano notes to each amino acid (or nucleotide. or stop/start codon. idk, it’s been a few years), and although this method was neither the fastest nor the most efficient, it was interesting to listen to how the sequence flowed. at first, you‘d hear a variety of notes that simply existed without trying to form a ‘proper’ melody. and then you’d hear the dissonance. one jarring chord kept repeating itself. three sharp piano notes on their own. it sounded so... wrong. at that point, my biology teacher explained that within a gene, excessively repetitive sequences may lead to diseases (such as Huntington’s disease). kinda blew my mind, how DNA can be represented by piano music. wish i knew where to find that research paper...
(btw, greetings from Malaysia. i really like your channel, even though i know nothing about music composition. only clicked because i saw shostakovich, but i suppose i’m gonna binge all these videos now 😂)
reminds me of that video of Paint represented by sound
Sonification is the musical equivalent of staring at clouds until you’re convinced that you see Snoopy’s face in them. What annoys me about this method of composition is the same thing that annoys me about total serialism: the composers who use sonification seem to put more effort into the means of composition than the composition itself. All that work creating an elaborate system of compositional order and the music still sounds the same as if I were to pick a bunch of notes at random and order them together.
I think thats fair, and probably a good explanation as to why total serialism fell out of favor this century. However, I don't necessarily think exploring an idea for the sake of exploring it is necessarily a bad thing. Because how else would we have known what it sounded like?
I only get annoyed when entitlement accompanies these hypercomplex methods. You are not automatically revolutionizing music composition by trying something new or different - just put it out there and let history decide how important it is or is not.
@@ThrenMusic The thing is that it's usually not really complex at all when compared to other complex methods and systems of composition, but most importantly, if you try to turn data into music, you are almst never trying anything new anymore, people have been doing that for 50+ years.
No... But it is Snoop Dog in the cloud. He is way up in the sky and telling me to be chizzle because I am his homie
Means of composition over quality of composition seems to be all the rage these days
Douglas Adams' "Dirk Gently's Holistic Detective Agency" is a good read for this topic (and that's from way back in the 1980s, mind you): some arbitrary measured data is fed into an E-MU keyboard via MIDI (which was quite the cutting edge tech back then) and the result is - as one might expect - the "most hideous cacaphony". So then the protagonist enables an option saying "force C minor", and the result is - _surprise_ ! - the "most hideous cacaphony in C minor". I agree with your findings. Sonification can be helpful, but recent examples border on finding hidden messages in the Bible by applying some random algorithm for picking individual letters. With the amount of data to choose from, you are _bound_ to end up with something _seemingly_ meaningful if you just cherry-pick the samples you happen to like.
I must give it a read! Thanks for the comment - it made me laugh.
@@Tantacrul oh please do. It's wonderful
I'd forgotten about this excellent bit in the book! I was actually studying music (jazz performance, Piano) when I first read it in the late nineties, and one of my classes was digital composition (the institute I attended had a MIDI lab with Yamaha SY85s and Atari STs, outdated by that time but still fine for teaching the basics). We'd been learning introductory stuff like setting the key of a project and click-and-drag transposition only a week or so prior, which made it particularly hilarious to me. :)
Considering that say, Jupiter is just a giant ball of gas with rather high speed winds, wouldn't its sound just be a sort of constant deafening roar?
Plus some thunder from the lightning, and pitch alteration from the atmospheric makeup wherever you're measuring.
actually what I learnt in school is that sonification is mapping data to sounds but doing it in a composer like way, so you actually try to make a composition out of it. audification, on the other hand, is mapping data to sounds without trying to make it into a composition
The terminology is a problem. You also have 'parameter mapping' in the mix too. I think all of them should fall under the umbrella of Sonification.
Had a class in college about presenting scientific findings and information through art that felt entirely like this pretentious bullshit. I hated every moment of it and regretted taking the class.
7:28 was genious, imaginge in hospitals when the heart monitor stops messuring heart beats it plays this jingle
There's a fantastic section in a Douglas Adams book (possibly The Long Dark Teatime of the Soul) where someone has developed sonification software and is using it to generate company anthems absed on their finances.
Might be Dirk Gently I think
Radiation sounds like a bunch of clicks and pops. Chilling, it's the individual uraniam parmitipicles being radioacticitively exploded...
You can sonify molecules with nuclear magnetic resonance, which yields a spectrum that is literally audible before Fourier transforming. While it yields relatively boring sounds, it is literally what you are speaking of here, so it might be interesting for you to look into.
Ever see the NMR Carol of the bells?
7:35
I feel bad for her now.
She looks defeated after saying "It's all arbitrary"
Same
this channel is so fantastic. Absolutely love all your content - the effort behind each video, the well-suited graphics and editing techniques, intriguing concepts and genuinely funny voiceover. Keep it up buddy!!
I think sonified data is harder for most people to interpret because hearing tends to be a much more relational sense than vision. Unless you have a trained ear or a natural hability, when you listen to a sound you tend to pay more attention to the contrast between the sounds (the harmony, dynamics...) than to the sounds themselves (the exact pitch, volume...).
You need to have perfect pitch and rhythm to do it.
But even an untrained ear can easily puck out disonance that doesn't belong, so depending on the sonification scheme you could use it to identify timestamps that feel off to locate irregularities in the raw data for further review. (The whole thing may sound awful but eventually it becomes white noise and the deviations from the pattern will be dissonant and stick out)
Did you know some deaf people, when they got their hearing fixed, were surprised the sun didn't make a sound?
7:26 actually had me struggling to breath from a laughing (then subsequent coughing) fit. Really good way of demonstrating the point.
6:04 I mean they were so bored by their own work in the 90s, that they went and invented World Wide Web. They also had a musical band consisting of female singers back then and so the edited photo of the band became the very first photo on the Internet.
2:31
I actually like this unironically, just apply it with some repetition and you'd get music (or use it as a sample).
"producing an explosion of particle debris and a simultaneous explosion of dilettantes trying to make a name for themselves" that's fucking funny, man! You're going places.
Your tone (pun in tended) and personality are delightful and the edit is very well done...I really enjoyed this.
Honestly, the best part of the Sonification of planets is that they make GREAT droning noises for Call of Cthulhu campaigns. After all, there's no better noise to give the effect of the vast uncaring void of space, than the vast uncaring void of space.
7:11 was the best joke and best possible example for that point
As someone who has worked with new music composers for years, this video really hit home. Thank you for making it.
By the way, when music is programmatic in some way (as these sciences pieces are by definition), I really wish the basis for them would be explained as you have explained them here: we took this as our starting point, manipulated the hell out of it so it would sound 'nice' to the average listener, and now the work must stand on its own and really has no relationship to where it started.
This is a great video - and good to see the appropriate scorn actually being put to use! Nice to see James Saunders making a cameo in the description, I was reading his PhD dissertation recently and thumbing through the scores to his modular pieces.
Another really enjoyable and informative video. Good work big Tan.
dang, i love this channel because it like actually displays a lot of shower thoughts i’ve had before. great job!
For a while I thought you were being a little harsh, but then I remembered how angry I get when people talk total bollocks about the golden ratio in the visual arts, after which it all seemed a bit more measured and proportionate
Music based on mathematics I fun. Because some sequences actually have patterns which are also found in music. The online database of integer sequences has a function to sonify any sequence into midi. There is also a tag to some sequences that sound great.
Dude, you transformed the exact thoughts into a video that formed in my head while researching sonification. Really an impressive way of videofication
My dude I'm a long time subscriber -- this is your best video! Wouldn't be surprised if it gets a lot of traction. That said, it's pretty disappointing that you're willing to name and shame some undergrad from Minnesota (printing his name, face and Uni on the screen!) and then you won't even mention the big names like Steve Reich for fear of blowback. We don't need some specific student's face and life story in order to laugh at his music, and no one's going to lynch you for saying it can be cheap to use world tragedy to add punch to compositions.
That's a fair comment. This criticism occurred to me and it does make me wince a little. Let give you my reasoning and you can tell me whether you think I was off the mark. First off, I don't discriminate when it comes to naming and shaming and not I'm worried about the power of more established musicians. I think that statement is backed up by my previous videos (Bono, Coldplay, Springsteen, Ed Sheeran, Eurovision, etc.) and I actually hate people who take easy pot shots at amateurs (like Pop Idol does). However, this subject presented a problem: I openly criticised everyone I could but drew the line at artists who referenced extremely emotive or divisive topics. To give a flavour: think war atrocities or religious conflicts between nations. In those cases, where the author took a strong (and usually liberal) position, if I was to criticise the work, I could easily be accused of taking the opposite position and being a bigot or worse. I hate being handcuffed like this, so I made a more general criticism. I thought it was the most valuable comment I made in the video. In the case of Dan Crawford, I felt featuring him was fair because he's repeatedly posted his work on UA-cam and has ability of a far higher level than plenty of famous musicians. The topic of climate change isn't explosively divisive so I felt secure in talking about it. Even then, the only criticism I threw at him was that his idea has been done to death. That said, if I'm being honest, I wish I could have found video footage of a more established composer instead. I also didn't actually set out to print his name, it just came with the video he posted on UA-cam and it was difficult to edit out. I still take the point though: at the very least, the editing was clumsy and I perhaps should have not featured him so prominently. This is why editors exist in traditional media. I would love to have one.
In fairness, right now I can't think of a good example of a major composer abusing data specifically, it really is more of an undergrad thing. I remember a lot of my uni colleagues doing things like "the amount of notes in this piece is determined by the baking temperature of toast" etc. Maybe a self-composed example could have been an appropriate stand-in? In any case for the most part I really liked the video.
This has gotten me thinking. And gotten a lot of people thinking though. In one of my current points of view about art - isn't that this the point of art? To create to provoke? To get people thinking (doesn't really matter what about, but likely the subject matter)? The video's main critique may have been about when effectively arbitrary art is mistaken as or purported to be science, which can distort the public's understanding of science. But maybe it also gets the public excited (aka emotionally invested) about science and why science matters. Maybe it gets them excited about learning, about being curious about how data can get mapped into different mediums, about the maths behind it all. And curiousity is what drives science to begin with. So I guess for me currently, I feel that even if "the amount of notes in this piece is determined by the baking temperature of toast" is derivative, or uninspired, or misrepresentative - that's ok - because it started with someone being curious about "what does the baking temperature of toast sound like?" And for me, currently, I wouldn't laugh at someone's music for being curious and making that creative choice (however many times it's been done before), because look, it got people talking about climate change, which as cheap of a gimmick it might be to create a song from, it got us talking about what I consider to be a world tragedy. I suppose there's an argument for "well that's not the conversation we need to be having about climate change." and I'd likely agree with that, but in terms of getting all hands on the proverbial deck to be aware of such an issue (to then make informed choices at home, when purchasing, and when engaging in democracy) I'll take it. It shows that we value it.
But this was my first video of yours! Thanks for the thoughts, and love the engaged comments, and how you acknowledge fair points, and push back on others to ensure we're all pursuing truth!
@@businessbusiness9407 I think maybe John Cage's experiments with the I Ching might qualify as abusing data (even though the data is random, like the space stuff and LHC stuff in TC's video).
He's back!
You know, that actually sounds like a really cool way for musicians to make music. Like, for entertainement and artistic purposes, not just data. The best thing is that to make really unique art with it, you still will need and use your musical knowledge on top of that. Sounds fun.
i absolutely love Your content and the insight given in this video, but i hate You so much for the arbitrary sonification example at 7:25. I cried during that scene, but now i was laughing maniacally while hating myself for the monster ive become
whyyyyy
This video gave me more motivation to continue writing my music and improving my writing skills. Definitely gave me some more insight into sonication too.
What if instead of using this process to generate the "sound of Jupiter" or the "sound of climate change/sensitive topic data" or whatever. Someone used a bunch of random data (never mind how accurate the sounds actually are) and interpreted it (not encrypting it and calling it a day, but rather trying to create a chord progression by analyzing random noise) What would be the results? "The sound of sorting" is a project in which sorting algorithm activity is converted into sound. The result is a random set of data that no one in news is certainly going to care about (unless it were some kind of niche computer science news) I think this would be fun to speculate on. What is the best way to derive harmony from random noise? What kind of compositions can be written as a result. What kind of chord progressions could a composer create if they did not just stop at the encryption step, but rather tried to write a chord progression or a melody from all of that sound?
Music derived from a sorting algorithm is definitely a useful aid, but I'm doubt that counts as sonification.
According to Wikipedia: "Several different techniques for auditory rendering of data can be categorized:
Acoustic Sonification [33]
Audification
Model-Based Sonification
Parameter Mapping
Stream-Based Sonification"
I would take each on their own theoretical merits. I'm going to study the shit out of this now.
my man out here pretending radix lsd sort base 10 isn't real music
@@rg5580 radix is a banger
Well, except, if you had asked, we could have explained the actual similarities between, say, plasma waves in the thin charged gas of space and sound waves in the dense collisional gas of our atmosphere. Yes, there is a good deal of sonification done with various different mappings of data to audio characteristics, but there is also the sound we produce with is really exactly like amplifying very very quiet audio. Plasma waves are not free-space propagating radio waves, but waves propagating via the electromagnetic interactions between charged particles (and fields) in space. Yes, humans could not hear these plasma waves in space with their unaided ears because the pressure of the oscillating particles is far far too small to influence an ear drum (or any standard microphone), however, they could hear the signals as we present them by connecting long antennas directly to speakers inside their space helmets, with no modifications. Long wire antennas would collect enough of the oscillating electric field (associated with the oscillating charged particles) to drive earbuds without any additional amplification or modulation.
Yes the "eerie sounds of Saturn" at 6:15 is something different: These are free-space propagating radio waves at frequencies far above human hearing, but sampled in the time
domain and shifted down in frequency to be played back as audio. In other cases, we have released audio synthesized from the spectral information, and, yes, this sometimes results in spooky phase effects that are not present in the original signal.
In the end, however, I mostly agree with your primary thesis. Sonification is, more often than not, just a gee-whiz public outreach toy. Additionally, the vast majority of "space audio" you find on social media is heavily overdubbed with synthesizer music. Notably, the whole "Brain/Mind Research" set of recordings from the 1990's have little or no actual plasma wave data in them and their claims of collaboration with NASA are entirely fiction (well, unless using publicly available data in your creative work should be considered a collaboration ;-) ).
I'm happy to correct any specific errors in the description. I hope it came across that I respect the discipline of sonification and I tried to distinguish it from other blatant public facing BS.
Not actual errors -- I just wanted to protest: "But, but, *we* (sort of) actually *do* present sounds from space!"
There's a discussion item "What do we mean by sound in space" on my channel that might help, and a very simplified video that was done by someone else titled "What Space Sounds Like -- ACOUSTICS" (a UA-cam search should find it) that gives a hand-waving description of plasma waves.
The real scientists here don't spend any time listening to our data and mostly the audio we release is for public outreach purposes, however, it is true that some features can more easily detected in the audio than by looking at spectrograms or waveform plots. For example, individual dust impacts on the various spacecraft can be quite easily detected by ear whereas it is very tedious to pour through screens and screens of waveform plots to spot them. Also, very subtle, especially very low frequency features can often be detected more easily by ear.
In defense of sonification, where arbitrary data values are mapped to arbitrary sound characteristics, note that it really isn't so much different than mapping those same data to various types of visual plots.
The bottom line is that public outreach or popular science presentations necessarily entail a simplification of the actual professional-level science. We always worry about slipping into the realm of BS and are not always successful at completely avoiding it. Also, of course, there are the outrageous misinterpretations or sensational presentations done by reporters and other non-scientists . . . As always, it's useful to remain skeptical.
+Space Audio Very cool that you're engaging with this post! Would love to see more of this discussion, if only youtube's comment system wasn't total junk. One that always winds me up is "the sound of a black hole" or "the sound of the earth" which turns out to be some simple oscillation or orbit pitch-shifted up an arbitrary amount of octaves and marinated in reverb. Most people would not identify common household sounds if they were shifted only an octave or two; a sound that has been shifted by 10 octaves or more is not in any meaningful way the same sound.
Yup, there's a lot of credulity stretching done. Many of the new-age types are enthralled by the "frequency" of the planets which are some arbitrary multiple of either their rotational or orbital frequency . . . and it never seems to occur to them that you can get any value you desire this way. Sometimes they get slightly more scientific and incorporate the frequency of the longest wavelength that could resonate between, for example, the surface of the earth and the nominal height of the ionosphere. This is all the audio equivalent of numerology.
what you said about "writing music is sometimes soul destroying, spending days and days pursuing a creative idea with nothing to show for it at the end."
I am going through a bit of that myself at the moment. (and am distracting myself with music related videos) it was a surprise hearing my feelings communicated to me from the outside. pattern seeking brains are funny
your content is so good, btw! i particularly love your humor and editing lol
My studies are about sonification of musical archetypes and loops creation from modular equations which involve fibonacci and other mathematical functions. That's musimathics. Also the sonification of new exponential gravity field equations with nbody creates really nice sound in simulation. I found a way on projectate to human listenable form that sounds good.
so many big terms that my head is done in lmao.
me me small brain
love this channel, brought here by cardiacs, don't think I'm smart enough for most of these videos, but I'm happy to find a source of musical knowledge that's informative as well as entertaining
I always thought about matter-ification of music would be interesting, to give the deaf a simile-idea of musical experience, with shapes and colours and movement.
Your work is of public interest. Thank you very much !
CERN scientists discovering beethoven's symphony inside the higgs boson is one of my favorite tantacrul moments ever 🙌
God damn, that was the most interesting YT I've watched to date.
After watching this I'm really conCERNed.
Noo stop!
"I've laid out my critical tools, and it's up to the listeners to use them as they see fit" you've earned a subscriber, my friend.
6:51 I believe you meant to say that the wavelengths were lengthened in order to fall in human hearing. Radio waves are many orders of magnitude shorter in wavelength than audible sound waves in air.
Absolutely. This is one of those script typos that really bugs me.
no! radio waves have quite long wavelengths. a 100MHz (typical radio wave) lightwave would correspond to a ~115Hz soundwave if you let the wavelength be the same
7:30 😂😂
HanBurritoz that was TERRIBLE LMAO
That made me do one of those wheezy laughs.
I lost it when I heard that heart monitor (or really GM square wave) go ham 7 minutes in
I am just started to do sonification on my own data and boy oh boy I am very happy to have watched your video. It definetly gave me ideas of what I don't want to do.
Ever turned an old crt television to an empty channel and listened to the static? Now that gave me goose bumps !
came from hallelujah and now I love this channel, I'm always looking for a channel like this, I came across adam neely, rick beato, julian ciancilio and now this
I love this channel so much...
I love science and I love math. I also love music. I know when both can work together well and when they don't, and that when they don't, we should never change one or the other so as to satisfy our biases to the point that what you here doesn't actually represent anything. You took the words right out my mouth on this one.
Wow, this is the greatest video I've ever seen, in some way shape or form.
oof
Brilliant and it's great to realize I'm not the only one more than a bit dubious about sonification ever since I heard a piece supposedly based on the flow and undercurrents of the river Mersey and wondered if one based on the local sewage farm would sound at all different.
Love this vid. I've been working on-and-off with sonifications of visual art, and struggling to find the happy medium between staying true to the source material and the medium, but creating something that doesn't sound hideous.
Plenty to think about from this video, (looks like some of my stuff is recommended in the side bar, not sure if that's just for me?)
just found this channel and love it
Honestly my favorite version of sonification are those sorting algorithm videos
By the thumbnail alone, I was expecting something about Gustav Holst's 'Jupiter'...
it would be coo to see you do a video like this but about turning math into music, which I think is a lot different than just random data and seems pretty interesting. I would definitely watch that
I love this so much, thank you. You put into words almost all the things I found so annoying about this.
I love your channel. Thank you for doing this.
7:11 gose bump moment with 81st patch of GS Wavetable's original sound module.
Ay,
Nice find
Fucking beautiful. You've elucidated vague feelings I've had about this practice for a few years now. The second to last segment using your viewer data was hilarious. Praise be.
An amazing scientific understanding! Data is meaningless without context and explanation of manipulation. Good job!
So good! Love the content
I know it's not sonification but the video reminds me of how engineers can often diagnose problems by sound. I can hear if a circuit has a short by the sound of my bench power supply for instance.
I think this is a bloody great video, and I lobe this channel. HOWEVER, I think that it doesn't matter how you get there, good music is good music and if you find something within that, cool. If not..... Cool. So I don't think using data and manipulating it to create music is wrong, It's just a method to create something unique.
Excellent dissertation.
i love yer content and how hard line you approach things.