Talking about yesterday's conference call explaining the failure, and adding my own high quality graphics using Microsoft paint. Text Transcript of the www.spacex.com/...
cpypcy The scientist that calculate and stay on the ground are playing it on easy, that's the problem. They had a different department that was in charge of building and making sure that the costum manifactures struts were going to hold. They can all throw the ball of "Who's fault is it" lower and lower until some random employees get fired and sued! Welcome to CAPITALISM!
Mark Zytphen-Adeler The cliche "You can't make an omelette without breaking a few eggs" is so very true in regards to engineering that it's comical. Metallurgy is all about destructive testing and that's before you even build anything more complicated than a metal bar. 80% below rating though? Who was the strut supplier, the ACME of Wile E. Coyote fame? I betcha they pay off a good chunk of that rocket to keep their name disassociated with this!
***** Two different forms of learning. You gather more data and fine-tune from successes, which certainly counts. Learning something unexpected and new, however, often comes through failure. But the very word "failure" is prejudicial in English. So people look at SpaceX's attempts to land their first stage as failures, and therefore bad things. Yet their own stated goal was never "to land our first stage on a barge". It's to "learn how to land our first stages on a barge". So I see them as successes since they learn new things each time. Or consider the infamous light bulb: Edison's team tried 2,000 different approaches before they developed a reliable light bulb. 2,000 failures and one success. They sucked, right? Or the Wright brothers: Their first powered flight came after almost a decade of building gliders and gradually, failure after failure, learning ways to control them in-flight. I think that's how Mark meant it. :)
Thanks for putting this together Scott. I got the basic ideas all the articles, but walking through it in a video with "visual aids" really helped solidify it all. Thanks again.
A brilliant video. You've conveyed an understanding of the malfunction that I've not seen anywhere. It's the perfect illustration of use of the minimal force necessary to accomplish at task. Your graphics have the perfect resolution to contribute to understanding without introducing obscuring noise into the communication process. Great job.
Headrock Let me just direct you to this image of scientists at the Higgs Boson announcement, an audience full of some of the best educated people in the field, many taking notes on a laptop they have chosen after careful consideration using the analytical skills which make them excell in science. i.dailymail.co.uk/i/pix/2012/07/04/article-2168557-13EB9DA9000005DC-708_634x423.jpg
Scott Manley Ouch... being an American, I know a burn like that is going to cost you around $248,000. Best of luck, Headrock. Hope you aren't in the U.S.
Scott Manley I want to say this is the logical fallacy of association. An intelligent person can smoke, this does not mean smoking is intelligent. It is not viable to say the machines are themselves superior to other machines in most/all senses using only this evidence. There are many factors to buying a computer, and most of them aren't for their specs. To list a few reasons why some of the best educated people in the world may have Apple products: software convenience (the file format their colleagues use is a format Apple software uses and it is faster and simpler to simply use what everyone else uses when collaborating, or submitting their work, etc...), company agreements/deals to use either the machine itself, or products/software used on the machine, and lastly ease of use/familiarity.
Hey Scott, thanks for the translation! I can't speak for anyone else, but I find this stuff fascinating despite my total illiteracy in physics. Having you explain these sorts of things in layman's terms is exactly what I need.
***** The programmers would have weighed the probability of a catastrophic failure which leaves the capsule intact vs the probabiliity of the software going wrong and deploying the chutes during launch.
Bram06 I would assume explosions are a real possibility. Also maybe I am biased because I play so much KSP. But I tend to think the top of the rocket survives more then the bottom. Scott Manley Makes sense I guess.
Scott Manley as someone who did study computer science and is studying aerospace engineering at the moment i would have made the same decision for unmanned flights. i don't trust this damn softwares :D
It's amazing that the failure was caused by something as simple as faulty struts. I'm actually surprised that they weren't already testing every piece of the structure prior to launch though; seems like a no-brainer but I guess when the supplier gives you wildly false specs you can't really see that coming.
Ideally the supplier should have been doing the testing and provided certified materials : ( selling things at 20% their listed strength seems illegal or grossly negligent to me. Just glad no one was on the craft when this fault came to light!
Chris Lundquist The issue with testing a part like this is that you need to destroy it. You never actually know what the failure limit is until it fails, you just have to take it on faith that what your supplier says is true. And once a part is tested, you can't use it in production, because even if the part passes the test and it isn't actually destroyed, the test itself is considered destructive. You can order 10-20% extras and do your own testing, but at that point you're duplicating effort, while also losing correlation effects. It's much better if this testing is being done by the manufacturer, who are able to discard entire batches, and end users just need to trust that proper procedures were followed during testing.
Michael Braedley While it's true you need to destructively test to determine a parts failure load BEYOND its specification, it is quite possible to exercise a part within its operating parameters without destroying it. The part on the rocket (and several others in the factory) apparently failed well within their rated operating window. Testing could/should/would have found this. SpaceX get criticised regularly by "old space" employees and former employees for a lack of testing. Looks like it bit them here. Personally, I hope that they put a proportional response in place. Large old companies in my experience tend to end up with so much process, testing, double checking, reviewing... that they end up completely unable to innovate. SpaceX needs to stay young and keep the innovation fast paced and relatively risky if Musk wants to get to Mars.
Craig Britton The issue is that, if I remember my materials science class (I only have the one, since I'm an EE), you can only really test a part to about 25% of the failure limit before the test becomes "destructive", even if the part under test isn't actually destroyed. (This is the limit between elastic and plastic deformation.) Sure, testing to 2500 lbs would have discovered a problem with this part, but if the part actually had a failure limit of 5000 lbs when rated at 10000 lbs, you've just "destroyed" the part, possibly without even knowing it. (In reality, you probably would know about it, since the part would probably be plastically deformed, and you should be able to detect that.) Also, every test you do, even well below the destructive limit, possibly weakens the part, so what was a part that could handle 10000 lbs might now only be able to handle 8000 lbs.
You don't know if it was wilful neglect on the suppliers side. 20 something years ago, there were a lot of cases where bolts used in aerospace construction prematurely failed because inferior steel had been used. But regardless, this is a prime example for the importance of testing the validity of claimed specs.
If you have a helium balloon in a car with closed windows, it will veer to the inside of the curves you take, whereas your body will feel a force towards the outside :D
abledbody You could also argue that quality testing of the components came down to SpaceX. Could be SpaceX could have been aware of the potential quality issues, but opted to forgo testing to save costs. Blaming the failure on a single person/entity isn't always fair, especially with something as complicated as spaceflight. Apollo 1, Columbia, and Challenger were failures that Nasa could have left the blame on a single supplier, but instead looked at solving the problems at the root and introducing redundancies into the design.
I am into high power rocketry. I really like the explanation of Space X failer. Very simple an easy to understand. I have froward you video to some friends because I could never explain as simply as you did. Great job. Rocket Nut
when you said someone asked about how the problem might have been avoided with more struts i lost it, had to pause the video for a minute to get myself together
That's simply astounding that with such a workaround, they could figure out the problem with the use of tools that were not necessarily designed for these measurements.
***** these kinds of failures used to happen in aviation all the time. Aviation has gotten a lot safer over the last 40 years. The still do happen, to some extent. I remember in a certain engine used on 737's had a failure where a fuel feedpipe was out of spec, so it ruptured due to normal engine vibration, causing fuel to flow into the engine uncontrolled until the engine caught fire, oversped, and blew up. The thing with airplane engines is, even if that happens, they can still fly without one engine, and land the plane (sometimes), whereas rockets are basically fuel tanks with engines strapped to them and no external control surfaces, so if one thing explodes, the whole thing explodes.
TheBalefire They have built rockets for over 40 years as well. Quite many more airplanes are flying than rockets, so if the probability of fubar is the same then we shouldn't ( p > 0,05) have many rocket accidents.
***** And rockets are orders of magnitude harded to build, harder to fly, and for all intents and purposes is still in an experimental stage of development. Remember that the SpaceX rockets are, literally, still under development.
Awesome explanation! That was very easy to understand yet very thorough. I think that should give everybody a pretty good idea as to what actually happened, and how they are fixing it to make future flights safer.
FYI "Popular Science" did an article on the Falcon Heavy which shows that the LOX tank is above the RP1 tank. Also, at the launch, you can see the LOX being vented from the top. Also, the struts in the picture are connecting the tank to the suspended wall of the Falcon 9.
Thanks for the video, this was super clear and informative. Maybe it would have been helpful to spend a little more time explaining why the helium tanks want to rise up faster through the LOX tanks when under acceleration (buoyancy is a function of weight, less LOX above the tanks to hold them down, etc)
Surely there will be more than just "oh we arent using that supplier", i mean surely there is gonna be some formal investigation, to find out why so many of their struts are failing. Had this cost some people their lives, some one would be going to jail i would assume ?. I mean there has to be an official investigation into this company, why their products arent up to specs. Having multiple struts fail and not just by a small amount, seems really odd. I mean only having 20% of its original strenght, just seems like an insane error.
This is a great video. I watched that launch live on NASA TV, and read the press releases about the conclusions concerning exactly what happened. I never really understood it until now.
One minor thing, the helium is added to prevent the tank from collapsing under the outside force/pressure as the fuel is removed, if not it would crumple up like a capri-sun juice bag. While this also stabilizes and increases the flow of fuel/oxidizer, I think that the structural aspects are slightly more pressing!
LOX is above the RP1 tank. You can confirm this by checking the frost on the vehicle on the pad. RP1 is currently loaded at near ambient temperature. LOX is obviously loaded at ~-300F. The top part of both S1 and S2 are where the frost is, ergo, that's where the LOX is.
Hey Scott, take a look at some of the older launch videos. I've forgotten which one it was that had a very cool view inside a second stage tank (must have been the O2 tank from the blue colour of the liquid), during and immediately after thrust. Very cool in it's own right, but as the tank empties you can clearly see a large tube at the bottom centre of the tank which likely passes the O2 through the middle of the RP1 tank, so the O2 probably is in front.
TonboIV Did some digging. SpaceX really liked that in tank camera for a while. They flew it at least on AsiaSat 8 and 6 and CRS-4 and 5. The two CRS flights give the best views and you can even see the He tanks attached to the inside.
To my understanding, LOX is in front of RP-1 because of its higher density, causing the rocket to be stable; the center of mass (essentially the point about which the rocket turns) is more in front of the aerodynamic center (the point where drag can be simplified to apply), so a disturbance causes the rocket to be moved back to the velocity vector, rather than away from it (compare this to a weather vane) Source: Aerospace Engineering student
Adrian T More math than women ;) but in all honesty it's tough but rewarding (Largely because I reaelly like the subject matter), my university (Delft, Netherlands) has a lot of international students in the Aerospace program so there's a really diverse group
I don't personally know that much about the composition of the Saturn V (Which stages did they turn it around on?), but keep in mind the upper stages are used in a lot thinner air, thus the aerodynamic torque is severely less
just leaving a comment to increase your engagement ration. No seriously beautifully explained! could you do the same with all major and minor space accidents? you could make this into a great series!
The way I understood the question and answer session is that the helium tank didn't actually burst from colliding with the top of the tank but from rupturing the line it was attached to which was to feed the helium close to the engine to heat it up and pressurize the LOX tank after. This line was yanked from the rising helium container causing the gas leak then the helium container apparently twisted closing the line which caused the pressure in the container to temporarily rise again which made for the "counterintuitve data" that Musk was talking about, but enough gas had already been released to blow the main tank.
I would consider the fact that they were able to diagnose the problem a not at all insignificant triumph for the overall program. Understanding your failures is as large a part of success as is accomplishing your initial goals.
Well, it would be interesting to find out more. This would be an excellent learning opportunity I've been an ASME plant inspector for years now, with the occasional QA/QC job. I've seen some bad construction. This usually is the result of cutting costs/corners to get the product out of the door as quickly and cheaply as possible. The rest of the time it's mostly bad design. Mechanical failures like these could have lots of contributing factors: wrong base materials, a batch of bad base materials (even though the material certificate check out), improper welding, bad welding procedure, over-/ undertightening of bolts, mechanical damage (grooves, notches,...) caused by improper handling, embrittlement of construction caused by cryogenic temperatures, dissimilar weld metal cracking, metal fatigue and last but not least greed and/or stupidity. Here's a short list of things I've come across in my career: - plastic deformation and snapping of bolts caused by overtightening which didn't allow for thermal expansion of the pipeline in question. - internal corrosion and leak of piping after only 300 running hours (less than a week) because of improper design. (wrong base material for chemicals handled. bad pipespec.) - cracking and leaking of welds on a new steam boiler for a power plant after only 100 running hours, caused by improper design and welding (base material too hard to be welded properly. welded it anyway.) - cracking of pressure vessel wall on a chemical reactor after 3 months caused by grinding marks + agressive external atmosphere. - failure of pipe supports because installer didn't take thermal expansion into account. - leaks of corrosive chemicals due to the installation of improper gaskets. - implosion of tank after hydrotest because a plastic bag was blocking the vacuum relief valve after paint job. - leaks of stainless steel piping because of inadequate chromium, nickel and molybdenum content, even though material certificates stated otherwise. - cracks in flanges of a new plastic storage tank because of overtightening of steel bolts. - 20 radiographs with exacly the same defect in the weld, caused by lazy radiographer. - misrepresentation of NDT reports, because the original inspection revealed defects the manufacturer couldn't be bothered to repair. - strange shapes on radiograph which turned out to be entire welding rods that had "fallen into" the weld preparation ( V ) and the welder "failed to notice" and subsequently welded over. Fun times.
Your order of the tanks is correct. In all the in-tank camera views we've seen of the LOX, there is no pipe running through the middle of the tank. The aft tank has plumbing in it to allow the propellant in the forward tank to get to the engine.
Informative and fun video Scott. It's nice to get a glimpse into the details of rocketry. The information blows my mind because; a) how could anyone have foreseen this during design and assembly to prevent the problem and b) if there hadn't been accelerometers on the spacecraft that had worked at such efficiency that they could detect the soundwaves then some of these details would still be a mystery. They just wouldn't know how things may have gone wrong. What I want to know is how do people perform this Rocket CSI? What do engineers do to retrace steps to find a problem? My dad's a mechanic and I know trouble shooting a car is one thing... but being able to sift through a mess of debris and data to find the cause of an "unplanned disassembly" is another. Cool stuff! Makes me want to do more research.
Scott, I've been watching your channel and playing ksp for many years. I'm not typically a contributor to the comments, but I would like to say that your channel is phenomenal. Thank you. And now, for information purposes, my comment with autocorrect uncorrected. Scott, I've been watching your channel and playing ksp for many years. I'm not typically a conductor to your discuss, cut I world allow to eat that type channel is pacifism asks. Thank got
Actually, the reason why you put the Helium tanks inside the Lox tank is that it reduces the difference in pressure between the outside and inside of the tank which means you can use a lighter tank and/or pressurize the helium more.
Thank you very much for the explanation. I look forward to future videos. I did get curious about the falcon spaceship being able to go into survival mode if the explosion doesn't deteriorate it badly? Thanks for pointing that out.
I just like the drawings. :D Every failure will bring us closer to perfection. It's better that this happened on a Supply Vessel rather than a manned mission. :D
Scott Manley There is one thing that is indisputable here... I am amazed that they did not have emergency return capability on the Dragon capsule. It's already designed for re-entry, so its only a big red button tied to some software which should already exist, albeit in a slightly different form, for re-entry. If I was flying a multi million dollar payload, I would have put some rudimentary RCS and parachute control designed for emergency abort onto the capsule just in case. This mission would have had a far bigger upside at this point, and I can only guess that the absence of that system has to do with someone, or someone's, saying "Its totally reliable, and we have done all the math.. we don't need to burn the extra man hours on that because we'll miss our deadlines" when it came up in meetings. It had to come up in meetings.. its so bleeding obvious of a question. So, I would say SpaceX getting a bloody nose here is a very good thing. They are a fantastic organization, but perhaps a bit too cocky. Risk taking is one thing, but put the contingency systems in place that you can easily predict. As an adjunct, I have worked in ISP and Telecom Infrastructure for over 20 years now, so I have witnessed this same mentality many times, and it has always.. 100% of the time.. created a disaster of some kind, at which point people would actually fix the problems I had been harping about for quite some time before. This event is identical in all respects to my experience, so it seems the Silicon Valley mentality has it's downsides.
I didn't look thru all the comments, but I believe the reason that RP1 is usually placed aft of LOX has something to do with distribution of mass - keeping the center of mass behind the center of gravity to help stability during flight.
the irony is incredibly strong, all the years we have been playing ksp and adding struts, it was not having enough struts that causes a failure in a real life mission.
Scott, you mention that the helium is there to pressure the tank so that the LOX gets pushed out of the valves. I thought that the helium pressurisation is there to give more mechanichal strength to the LOX tank. Or is it both reasons?
Swipe Basically, from my understanding, the helium tanks are very light (And helium is buoyant), so the struts are holding it in place. Liquid oxygen is also fairly dense, so the difference means the helium tanks really, REALLY want to shoot upwards. The struts were the only thing holding them in place.
This issue will cause Space X to re-evaluate every aspect of design and testing which is no bad thing given the impending requirement to transport astronauts to orbit.
Usually the LOX tanks are placed under the fuel (RP1 or LH2), this due to their higher density and thus also overall weight. In this case you get a more stable reverse pendulum model in which the CoM sits closer to the thrust point.
The Bird 9 schematic that SpaceX tweeted a while ago explains the inner structure of the Falcon 9 pretty accurately. I'm surprised that noone has seen it.
Is the oxidizer side of the fuel system on the second stage pressure fed only? I'd normally expect a turbo-pump set-up for such a large throughput. Or is it a hybrid system?
Interesting video Scott, you put a lot of work into this (maybe not the art, but atleast the research. :D ) I'm wondering tho, why would they need the helium to push the LOX out? Wont the LOX "boil" when it warms up, causing it to make it own pressure? I guess maybe they use the Helium to decrease the boil off from the LOX, so they save fuel? Or do they manage to keep the entire tank so cold LOX doesn't boil?
Was it actually said in the conference how many struts they tested during investigation and how many failed at what force? I've found info ranging from 1 failure to a few and that struts failed below specs but either force value wasn't stated at all or it was 6000lbs. I don't think I've seen any info that they managed to break any of those struts at just 2000lbs.
I have personally seen the inside of one of the falcon heavy 1 stages. the helium tanks are completely separate of the fuel tanks, outside of them. They are held on by struts. (the heavy stages are the same as the F9 stages) but since I did not see any of the second stage (except for the engine) I can't say that it has the same set-up
aggiefan88 The helium pods looked like the pod on the right (in the picture you showed) but much wider. they were situated between the 2 main fuel tanks. (When I saw the inside, the kerosene tank was missing, but the LOX tank+ tube that ran in through the kero was in place) It blows my mind to see something that sturdy and tough, literally disintegrate into bits and shreds by sheer air resistance.
![[UA] Team Vitality проти NAVI | IEM Cologne 2024](http://i.ytimg.com/vi/wttO1zVRTC0/mqdefault.jpg)
"The problem could have been solved with more struts."
It really is the answer to everything in rocket science.
Uraneum
Uhmm no. sometimes you also need more booster.
Keyboard runner , you also need more money )))
Keyboard runner , you also need more money )))
Sometimes reaction wheel spam could also help
+Uraneum exept weight
Revert to assembly, build more struts, relaunch.
cpypcy they can't revert, they play on hard mode
They play career mode
cpypcy The scientist that calculate and stay on the ground are playing it on easy, that's the problem. They had a different department that was in charge of building and making sure that the costum manifactures struts were going to hold. They can all throw the ball of "Who's fault is it" lower and lower until some random employees get fired and sued! Welcome to CAPITALISM!
Anonymously Sounds more like Politics to me.
Anonymously If anyone gets sued it would be the strut manufacturers.
all problems can be solved with more struts and boosters
zisteu logic: doesnt get to where you want add moar boosters with 80 solid booster stages... done!
And never sit Jeb next to Valentina again!
He cant think and concentrate his self with her on his side!
ashley beaumont MOAR STRUTS
HaqqAttak In real life, it's not quantity, it's quality. Not more struts -- better struts.
stcredzero sooo...MOAR (NovaPunch Even Heavier) STRUTS?
I don't think the real Falcon 9 got hit by a big sphere of this unknown white material
Yeah.
Matthew Nguyen It did. The government is just trying to cover it up.
Matthew Nguyen It was a ULA experimental anti-rocket sphere.
GeorgeMonet LMFAO
Matthew Nguyen It was aliens!
"...someone asked...Could this have been solved with more struts?...."....EPIC LOL!
stealthblack07 that was definitely my favourite thing to come out of the whole unfortunate event!
stealthblack07 What I want to know is how many of the professional rocket scientists present had a sudden coughing fit at that point....;-)
stealthblack07 I guess the Person who asked this is playing KSP.
stealthblack07 haha what was the answer?
I was laughing so hard I feared I woke my neighbors :D
I love how we learn the most from failure...
Mark Zytphen-Adeler The cliche "You can't make an omelette without breaking a few eggs" is so very true in regards to engineering that it's comical. Metallurgy is all about destructive testing and that's before you even build anything more complicated than a metal bar. 80% below rating though? Who was the strut supplier, the ACME of Wile E. Coyote fame? I betcha they pay off a good chunk of that rocket to keep their name disassociated with this!
***** Two different forms of learning. You gather more data and fine-tune from successes, which certainly counts. Learning something unexpected and new, however, often comes through failure. But the very word "failure" is prejudicial in English. So people look at SpaceX's attempts to land their first stage as failures, and therefore bad things. Yet their own stated goal was never "to land our first stage on a barge". It's to "learn how to land our first stages on a barge". So I see them as successes since they learn new things each time.
Or consider the infamous light bulb: Edison's team tried 2,000 different approaches before they developed a reliable light bulb. 2,000 failures and one success. They sucked, right? Or the Wright brothers: Their first powered flight came after almost a decade of building gliders and gradually, failure after failure, learning ways to control them in-flight.
I think that's how Mark meant it. :)
***** Well we're all different ;) And all I said was most
Watch Scott Manley explain and break down the failure of the Space X Rocket, hopefully next time we will remember to add more struts!
Joseph Caruso Its OK. That's actually cool.
Joseph Caruso then don't fucking say it
Justin Koenig Okay, no need to be rude.
Justin Koenig < this. I find the 'first' thing very annoying and a LOT of other people do to.
***** I find it annoying too... I'll delete that comment.
Thanks for putting this together Scott. I got the basic ideas all the articles, but walking through it in a video with "visual aids" really helped solidify it all. Thanks again.
Someone featured your video in a Gizmodo article. It's called "The SpaceX Launch Explosion Explained in Geeky Detail with MS Paint" by Chris Mills.
A brilliant video. You've conveyed an understanding of the malfunction that I've not seen anywhere. It's the perfect illustration of use of the minimal force necessary to accomplish at task. Your graphics have the perfect resolution to contribute to understanding without introducing obscuring noise into the communication process. Great job.
Scott, why aren't you working for Elon Musk, doing space software for the good of humanity, instead of dumbing us all down with Apple products?
***** this is disappointing to read.
Headrock Let me just direct you to this image of scientists at the Higgs Boson announcement, an audience full of some of the best educated people in the field, many taking notes on a laptop they have chosen after careful consideration using the analytical skills which make them excell in science.
i.dailymail.co.uk/i/pix/2012/07/04/article-2168557-13EB9DA9000005DC-708_634x423.jpg
Scott Manley Ouch... being an American, I know a burn like that is going to cost you around $248,000. Best of luck, Headrock. Hope you aren't in the U.S.
Scott Manley I want to say this is the logical fallacy of association. An intelligent person can smoke, this does not mean smoking is intelligent. It is not viable to say the machines are themselves superior to other machines in most/all senses using only this evidence. There are many factors to buying a computer, and most of them aren't for their specs.
To list a few reasons why some of the best educated people in the world may have Apple products: software convenience (the file format their colleagues use is a format Apple software uses and it is faster and simpler to simply use what everyone else uses when collaborating, or submitting their work, etc...), company agreements/deals to use either the machine itself, or products/software used on the machine, and lastly ease of use/familiarity.
Hey Scott, thanks for the translation! I can't speak for anyone else, but I find this stuff fascinating despite my total illiteracy in physics. Having you explain these sorts of things in layman's terms is exactly what I need.
It's kind of funny how close to life KSP is sometimes.
So.... none of the programmers of the rocket thought "what will we do if the large container of explosive propellant somehow explodes"....
***** What would you have done?
***** The programmers would have weighed the probability of a catastrophic failure which leaves the capsule intact vs the probabiliity of the software going wrong and deploying the chutes during launch.
Scott Manley They need a giant red "ABORT" button in mission control. (For the unmanned ones at least. That delay could really hurt/kill a passenger)
Bram06 I would assume explosions are a real possibility. Also maybe I am biased because I play so much KSP. But I tend to think the top of the rocket survives more then the bottom.
Scott Manley Makes sense I guess.
Scott Manley as someone who did study computer science and is studying aerospace engineering at the moment i would have made the same decision for unmanned flights. i don't trust this damn softwares :D
It's amazing that the failure was caused by something as simple as faulty struts. I'm actually surprised that they weren't already testing every piece of the structure prior to launch though; seems like a no-brainer but I guess when the supplier gives you wildly false specs you can't really see that coming.
Ideally the supplier should have been doing the testing and provided certified materials : ( selling things at 20% their listed strength seems illegal or grossly negligent to me. Just glad no one was on the craft when this fault came to light!
Chris Lundquist The issue with testing a part like this is that you need to destroy it. You never actually know what the failure limit is until it fails, you just have to take it on faith that what your supplier says is true. And once a part is tested, you can't use it in production, because even if the part passes the test and it isn't actually destroyed, the test itself is considered destructive. You can order 10-20% extras and do your own testing, but at that point you're duplicating effort, while also losing correlation effects. It's much better if this testing is being done by the manufacturer, who are able to discard entire batches, and end users just need to trust that proper procedures were followed during testing.
Michael Braedley While it's true you need to destructively test to determine a parts failure load BEYOND its specification, it is quite possible to exercise a part within its operating parameters without destroying it. The part on the rocket (and several others in the factory) apparently failed well within their rated operating window. Testing could/should/would have found this. SpaceX get criticised regularly by "old space" employees and former employees for a lack of testing. Looks like it bit them here. Personally, I hope that they put a proportional response in place. Large old companies in my experience tend to end up with so much process, testing, double checking, reviewing... that they end up completely unable to innovate. SpaceX needs to stay young and keep the innovation fast paced and relatively risky if Musk wants to get to Mars.
Craig Britton The issue is that, if I remember my materials science class (I only have the one, since I'm an EE), you can only really test a part to about 25% of the failure limit before the test becomes "destructive", even if the part under test isn't actually destroyed. (This is the limit between elastic and plastic deformation.) Sure, testing to 2500 lbs would have discovered a problem with this part, but if the part actually had a failure limit of 5000 lbs when rated at 10000 lbs, you've just "destroyed" the part, possibly without even knowing it. (In reality, you probably would know about it, since the part would probably be plastically deformed, and you should be able to detect that.) Also, every test you do, even well below the destructive limit, possibly weakens the part, so what was a part that could handle 10000 lbs might now only be able to handle 8000 lbs.
You don't know if it was wilful neglect on the suppliers side. 20 something years ago, there were a lot of cases where bolts used in aerospace construction prematurely failed because inferior steel had been used. But regardless, this is a prime example for the importance of testing the validity of claimed specs.
I never considered the rockets acceleration playing a part in bouyancy. Easy to just throw that 9.81 in there without considering it.
SoulAir yeah, that was surprising a bit (though obvious afterwards of cause)
If you have a helium balloon in a car with closed windows, it will veer to the inside of the curves you take, whereas your body will feel a force towards the outside :D
They could of learned from KSP always add MORE STRUTS!
Alex Lindsay Haha! or use the KW rocketry struts that are 1 TON
escraftTH No, they just say 1 tonne. AFAIK they still don't actually contribute to vessel weight.
AlienCollective oh! so they are massless!
escraftTH They were in 0.90 for sure, and I haven't noticed/heard anything to the contrary about 1.0.x. It's easily checked, though.
AlienCollective ksp doesn't need 2 types of strut, does it?
**Rocket explodes**
Scientist in launch tower to other scientist: I *told* you we needed more boosters!
Bram06 As much as I enjoyed the joke, wouldn't they have left the launch tower and been in mission control, probably at least a couple miles away?
I'm a geek, not a nerd Probably
Even though I'd read a few articles about the CRS7 analysis, this was extremely helpful. Thank you, Scott!
Think there'll be a lawsuit? After all, whoever sold those struts just sent millions of dollars down the drain.
I hope so
abledbody Whoever sold those struts may have some trouble gaining buyers trust again.
abledbody StrutCo you fuckersss!!
abledbody Pretty sure Elon is quite averse to litigation. He'll punish them by not buying from them, and not hiding who sold the faulty parts.
abledbody You could also argue that quality testing of the components came down to SpaceX. Could be SpaceX could have been aware of the potential quality issues, but opted to forgo testing to save costs. Blaming the failure on a single person/entity isn't always fair, especially with something as complicated as spaceflight. Apollo 1, Columbia, and Challenger were failures that Nasa could have left the blame on a single supplier, but instead looked at solving the problems at the root and introducing redundancies into the design.
Love your channel. You simplify rocket science down to a level where everyone can understand it, which is great.
I am into high power rocketry. I really like the explanation of Space X failer. Very simple an easy to understand. I have froward you video to some friends because I could never explain as simply as you did. Great job.
Rocket Nut
when you said someone asked about how the problem might have been avoided with more struts i lost it, had to pause the video for a minute to get myself together
Hey Scott!
Your videos recently started to emerge on science popularization news feeds of ex-USSR countries! Well done! Thanks for keeping it simple!
That's simply astounding that with such a workaround, they could figure out the problem with the use of tools that were not necessarily designed for these measurements.
Interesting. It seems components used in rockets are not as highly regulated as components used in aviation.
***** these kinds of failures used to happen in aviation all the time. Aviation has gotten a lot safer over the last 40 years. The still do happen, to some extent. I remember in a certain engine used on 737's had a failure where a fuel feedpipe was out of spec, so it ruptured due to normal engine vibration, causing fuel to flow into the engine uncontrolled until the engine caught fire, oversped, and blew up.
The thing with airplane engines is, even if that happens, they can still fly without one engine, and land the plane (sometimes), whereas rockets are basically fuel tanks with engines strapped to them and no external control surfaces, so if one thing explodes, the whole thing explodes.
TheBalefire Actually, to be fair, on a previous Falcon 9 launch an engine failed during ascent and it was able to continue just fine.
TheBalefire
They have built rockets for over 40 years as well. Quite many more airplanes are flying than rockets, so if the probability of fubar is the same then we shouldn't ( p > 0,05) have many rocket accidents.
***** And rockets are orders of magnitude harded to build, harder to fly, and for all intents and purposes is still in an experimental stage of development. Remember that the SpaceX rockets are, literally, still under development.
Mythricia
All that is meaningless. It is equally easy to check that components meet required specs.
Great explanation of what happened, Scott. Well done.
Thank you. It's nice to have a breakdown interpretation of how this event occurred.
I love your channel Scott. It involves so many things that all incorperate science. There's nothing quite like it.
Thank you for posting this video Scott, very interesting. I always love to watch these kind of videos, I hope you keep making them.
Awesome explanation! That was very easy to understand yet very thorough. I think that should give everybody a pretty good idea as to what actually happened, and how they are fixing it to make future flights safer.
Another fantastically informative video, Scott. Your graphics are top tier.
FYI "Popular Science" did an article on the Falcon Heavy which shows that the LOX tank is above the RP1 tank. Also, at the launch, you can see the LOX being vented from the top. Also, the struts in the picture are connecting the tank to the suspended wall of the Falcon 9.
Thanks so much Scott for simplifying this for us!
Thanks for the video, this was super clear and informative. Maybe it would have been helpful to spend a little more time explaining why the helium tanks want to rise up faster through the LOX tanks when under acceleration (buoyancy is a function of weight, less LOX above the tanks to hold them down, etc)
Surely there will be more than just "oh we arent using that supplier", i mean surely there is gonna be some formal investigation, to find out why so many of their struts are failing. Had this cost some people their lives, some one would be going to jail i would assume ?. I mean there has to be an official investigation into this company, why their products arent up to specs. Having multiple struts fail and not just by a small amount, seems really odd. I mean only having 20% of its original strenght, just seems like an insane error.
Thanks for the video! As a layman it's definitely interesting to hear what went down. I hope Space X will make a swift recovery from this.
leave it to spaceX to get single stage rockets higher in the air than multi stage rockets.
Very nice explanation, thx. Hope that was indeed the problem, which can now be fixed.
At least the oxygen was released into the atmosphere...
Tanks Scott for the description what happens. Simply but good for me to understand.
This is a great video. I watched that launch live on NASA TV, and read the press releases about the conclusions concerning exactly what happened. I never really understood it until now.
One minor thing, the helium is added to prevent the tank from collapsing under the outside force/pressure as the fuel is removed, if not it would crumple up like a capri-sun juice bag. While this also stabilizes and increases the flow of fuel/oxidizer, I think that the structural aspects are slightly more pressing!
As always very concise and to the point, thanks for the video!
LOX is above the RP1 tank. You can confirm this by checking the frost on the vehicle on the pad. RP1 is currently loaded at near ambient temperature. LOX is obviously loaded at ~-300F. The top part of both S1 and S2 are where the frost is, ergo, that's where the LOX is.
Hey Scott, take a look at some of the older launch videos. I've forgotten which one it was that had a very cool view inside a second stage tank (must have been the O2 tank from the blue colour of the liquid), during and immediately after thrust. Very cool in it's own right, but as the tank empties you can clearly see a large tube at the bottom centre of the tank which likely passes the O2 through the middle of the RP1 tank, so the O2 probably is in front.
TonboIV Did some digging. SpaceX really liked that in tank camera for a while. They flew it at least on AsiaSat 8 and 6 and CRS-4 and 5. The two CRS flights give the best views and you can even see the He tanks attached to the inside.
TonboIV Did you look at the card I added next to the Elon Musk image? It points to an addendum with the image you describe.
Scott Manley: the only guy who can say "I'm smart" like a fact and have it sound completely humble. respect xD
To my understanding, LOX is in front of RP-1 because of its higher density, causing the rocket to be stable; the center of mass (essentially the point about which the rocket turns) is more in front of the aerodynamic center (the point where drag can be simplified to apply), so a disturbance causes the rocket to be moved back to the velocity vector, rather than away from it (compare this to a weather vane)
Source: Aerospace Engineering student
Arjan Vermeulen This is correct. Source: Guy who has played a lot of Kerbal.
Arjan Vermeulen How is being a student in that field of study going for you?
Adrian T More math than women ;) but in all honesty it's tough but rewarding (Largely because I reaelly like the subject matter), my university (Delft, Netherlands) has a lot of international students in the Aerospace program so there's a really diverse group
I counter with the upper stages of the Saturn 5.
It depends on the vehicle, your logic was used on the space shuttle, but not everywhere.
I don't personally know that much about the composition of the Saturn V (Which stages did they turn it around on?), but keep in mind the upper stages are used in a lot thinner air, thus the aerodynamic torque is severely less
Thanks for a great, lay-kerbal explanation, Scott.
Awesome explanation and demo! Thanks Scott!
just leaving a comment to increase your engagement ration. No seriously beautifully explained! could you do the same with all major and minor space accidents? you could make this into a great series!
Kraken loves clipping! Helium tanks clipped into Oxidiser tank that's the matter!
I was there at kennedy space station and I remember the explosion! I will NEVER forget it.
Thanks Scott that was very interesting and easy to grasp.
So cool to see how such a small problem can lead to a catastrophic outcome. Similar to the refueling mishap that happened too.
The way I understood the question and answer session is that the helium tank didn't actually burst from colliding with the top of the tank but from rupturing the line it was attached to which was to feed the helium close to the engine to heat it up and pressurize the LOX tank after. This line was yanked from the rising helium container causing the gas leak then the helium container apparently twisted closing the line which caused the pressure in the container to temporarily rise again which made for the "counterintuitve data" that Musk was talking about, but enough gas had already been released to blow the main tank.
I would consider the fact that they were able to diagnose the problem a not at all insignificant triumph for the overall program. Understanding your failures is as large a part of success as is accomplishing your initial goals.
Well, it would be interesting to find out more. This would be an excellent learning opportunity
I've been an ASME plant inspector for years now, with the occasional QA/QC job. I've seen some bad construction. This usually is the result of cutting costs/corners to get the product out of the door as quickly and cheaply as possible. The rest of the time it's mostly bad design.
Mechanical failures like these could have lots of contributing factors: wrong base materials, a batch of bad base materials (even though the material certificate check out), improper welding, bad welding procedure, over-/ undertightening of bolts, mechanical damage (grooves, notches,...) caused by improper handling, embrittlement of construction caused by cryogenic temperatures, dissimilar weld metal cracking, metal fatigue and last but not least greed and/or stupidity.
Here's a short list of things I've come across in my career:
- plastic deformation and snapping of bolts caused by overtightening which didn't allow for thermal expansion of the pipeline in question.
- internal corrosion and leak of piping after only 300 running hours (less than a week) because of improper design. (wrong base material for chemicals handled. bad pipespec.)
- cracking and leaking of welds on a new steam boiler for a power plant after only 100 running hours, caused by improper design and welding (base material too hard to be welded properly. welded it anyway.)
- cracking of pressure vessel wall on a chemical reactor after 3 months caused by grinding marks + agressive external atmosphere.
- failure of pipe supports because installer didn't take thermal expansion into account.
- leaks of corrosive chemicals due to the installation of improper gaskets.
- implosion of tank after hydrotest because a plastic bag was blocking the vacuum relief valve after paint job.
- leaks of stainless steel piping because of inadequate chromium, nickel and molybdenum content, even though material certificates stated otherwise.
- cracks in flanges of a new plastic storage tank because of overtightening of steel bolts.
- 20 radiographs with exacly the same defect in the weld, caused by lazy radiographer.
- misrepresentation of NDT reports, because the original inspection revealed defects the manufacturer couldn't be bothered to repair.
- strange shapes on radiograph which turned out to be entire welding rods that had "fallen into" the weld preparation ( V ) and the welder "failed to notice" and subsequently welded over.
Fun times.
Your order of the tanks is correct. In all the in-tank camera views we've seen of the LOX, there is no pipe running through the middle of the tank. The aft tank has plumbing in it to allow the propellant in the forward tank to get to the engine.
Is there a video of somebody asking the "moar struts" question? I'd love to see it :)
Informative and fun video Scott. It's nice to get a glimpse into the details of rocketry. The information blows my mind because; a) how could anyone have foreseen this during design and assembly to prevent the problem and b) if there hadn't been accelerometers on the spacecraft that had worked at such efficiency that they could detect the soundwaves then some of these details would still be a mystery. They just wouldn't know how things may have gone wrong. What I want to know is how do people perform this Rocket CSI? What do engineers do to retrace steps to find a problem? My dad's a mechanic and I know trouble shooting a car is one thing... but being able to sift through a mess of debris and data to find the cause of an "unplanned disassembly" is another. Cool stuff! Makes me want to do more research.
they use hax to place the struts inside of the tanks I assume?
Alt f12 0 allow clipping in part editor (bug hazard)
They check that
Solid Banana Not hax, just the offset tool.
ohhhhhh, no wonder it has a bug hazard.
Solid Banana in KSP we call theese CHEAATTTSSSSS
Scott, I've been watching your channel and playing ksp for many years. I'm not typically a contributor to the comments, but I would like to say that your channel is phenomenal. Thank you.
And now, for information purposes, my comment with autocorrect uncorrected.
Scott, I've been watching your channel and playing ksp for many years. I'm not typically a conductor to your discuss, cut I world allow to eat that type channel is pacifism asks. Thank got
Hey Scott -
I am a rocket scientist. You did a great job explaining this!
Moar struts... that cracked me up! :D In any case, thanks for the informative video, Mr. Manley! :)
they should have stick with *'Kerlington Model Rockets and Paper Products Inc.'* as main Struts supplier ;)
Actually, the reason why you put the Helium tanks inside the Lox tank is that it reduces the difference in pressure between the outside and inside of the tank which means you can use a lighter tank and/or pressurize the helium more.
Thank you very much for the explanation. I look forward to future videos. I did get curious about the falcon spaceship being able to go into survival mode if the explosion doesn't deteriorate it badly? Thanks for pointing that out.
Great explanation of what happened in there. I guess it's never too many struts!
Thank you Scott! I really enjoyed this simple explanation, even as a computer scientist i understood what happend :D Your videos are great!
Let's be honest, KSP players have been waiting for this moment: ALL TOGETHER NOW.
NEEDS MOAR STRUTS
KSP TEACHES REAL LIFE LESSONS CONFIRMED!!!!!!!!!!!!
WE NEED MOOAAR FUUCKING STRUTS
WHAT IS IT THAT U DONT UNDERSTAND SPACEX????????????????????????
I just like the drawings. :D
Every failure will bring us closer to perfection. It's better that this happened on a Supply Vessel rather than a manned mission. :D
Thank you for this. A great explanation as always.
Scott Manley There is one thing that is indisputable here... I am amazed that they did not have emergency return capability on the Dragon capsule. It's already designed for re-entry, so its only a big red button tied to some software which should already exist, albeit in a slightly different form, for re-entry. If I was flying a multi million dollar payload, I would have put some rudimentary RCS and parachute control designed for emergency abort onto the capsule just in case. This mission would have had a far bigger upside at this point, and I can only guess that the absence of that system has to do with someone, or someone's, saying "Its totally reliable, and we have done all the math.. we don't need to burn the extra man hours on that because we'll miss our deadlines" when it came up in meetings. It had to come up in meetings.. its so bleeding obvious of a question. So, I would say SpaceX getting a bloody nose here is a very good thing. They are a fantastic organization, but perhaps a bit too cocky. Risk taking is one thing, but put the contingency systems in place that you can easily predict. As an adjunct, I have worked in ISP and Telecom Infrastructure for over 20 years now, so I have witnessed this same mentality many times, and it has always.. 100% of the time.. created a disaster of some kind, at which point people would actually fix the problems I had been harping about for quite some time before. This event is identical in all respects to my experience, so it seems the Silicon Valley mentality has it's downsides.
I didn't look thru all the comments, but I believe the reason that RP1 is usually placed aft of LOX has something to do with distribution of mass - keeping the center of mass behind the center of gravity to help stability during flight.
the irony is incredibly strong, all the years we have been playing ksp and adding struts, it was not having enough struts that causes a failure in a real life mission.
Scott, you mention that the helium is there to pressure the tank so that the LOX gets pushed out of the valves. I thought that the helium pressurisation is there to give more mechanichal strength to the LOX tank. Or is it both reasons?
Amazing video as always, thanks!
Quick question, why is the vessel acceleration upwards? wouldnt the mini tanks be forced downwards during takeoff?
Well yes, but if you balance the force of boyancy against this, it's not even close.
Oh so you mean when it's launched out with highspeed it creates an upwards force on the canister? or is it stricly related to the heat?
Swipe Basically, from my understanding, the helium tanks are very light (And helium is buoyant), so the struts are holding it in place. Liquid oxygen is also fairly dense, so the difference means the helium tanks really, REALLY want to shoot upwards. The struts were the only thing holding them in place.
Swipe both the LOX and the helium tanks would be forced down with the same relative forces, so it shouldn't have any impact on the relative boyancy
This issue will cause Space X to re-evaluate every aspect of design and testing which is no bad thing given the impending requirement to transport astronauts to orbit.
Excellent explanation. Thanks Scott!
Usually the LOX tanks are placed under the fuel (RP1 or LH2), this due to their higher density and thus also overall weight.
In this case you get a more stable reverse pendulum model in which the CoM sits closer to the thrust point.
You have it backwards, you want the COM to be as far forwards as possible to increase aerodynamic stability.
Scott Manley
I guess you're right, how terribly embarrassing on my part. Please don't tell anyone...
I'M GOING TO TELL EVERYONE
Junn Kopf Por favor, NOOOOoooooOOOO!
Is the "SIMULATION" warning an Iain Banks reference like the barge names?
Bill Kuker well spotted!
Congrats on making the front page of Gizmodo!
SpaceX should install the joint reinforcement mod.
Thanks Scott - I found that fascinating. And it all made perfect sense to me.
"…some of the struts were failing at 20% of their certified value…" Sounds like there'll be lawsuits in the near future.
The Bird 9 schematic that SpaceX tweeted a while ago explains the inner structure of the Falcon 9 pretty accurately. I'm surprised that noone has seen it.
Love your videos. Keep doing what you're doing
You put the LOX tank on the top for keeping the COM close to the tip for aerodynamic stability reasons. ^^
Thank you for not calling it an "explosion" :-)
You've got some of the greatest minds at your company, and NONE of them thought "Hey, we should test these struts before we use them!"?! FFS people!
When are you going to start your RSS series?? And what mods are you going to use? I'm excited to see you do another ksp series.
Is the oxidizer side of the fuel system on the second stage pressure fed only? I'd normally expect a turbo-pump set-up for such a large throughput. Or is it a hybrid system?
As a rocket scientist you made a great explanation.
Helium also remains a gas at the very low temperature of liquid oxygen (it's also used to pressurize the liquid hydrogen tank in rockets that use LH2)
I have a question.
Why not put the helium tanks at the top of the oxygen container? Why put them in the middle and then have to keep them in place?
pooya130 Easier to mount them on a surface that's only curved in one axis I guess
Interesting video Scott, you put a lot of work into this (maybe not the art, but atleast the research. :D )
I'm wondering tho, why would they need the helium to push the LOX out? Wont the LOX "boil" when it warms up, causing it to make it own pressure? I guess maybe they use the Helium to decrease the boil off from the LOX, so they save fuel? Or do they manage to keep the entire tank so cold LOX doesn't boil?
Slakto It doesn't boil fast enough.
Was it actually said in the conference how many struts they tested during investigation and how many failed at what force? I've found info ranging from 1 failure to a few and that struts failed below specs but either force value wasn't stated at all or it was 6000lbs. I don't think I've seen any info that they managed to break any of those struts at just 2000lbs.
I have personally seen the inside of one of the falcon heavy 1 stages. the helium tanks are completely separate of the fuel tanks, outside of them. They are held on by struts. (the heavy stages are the same as the F9 stages) but since I did not see any of the second stage (except for the engine) I can't say that it has the same set-up
aggiefan88 The helium pods looked like the pod on the right (in the picture you showed) but much wider. they were situated between the 2 main fuel tanks. (When I saw the inside, the kerosene tank was missing, but the LOX tank+ tube that ran in through the kero was in place) It blows my mind to see something that sturdy and tough, literally disintegrate into bits and shreds by sheer air resistance.