I'm gonna go out on a limb here and guess that this is the only video in the entire universe that covers boiling eggs and making Senko Hanabi in the same video.
I spent 20 years working materials and process development in the semiconductor industry and these design of experiment (DOE) methods are foundational to quickly identifying meaningful improvements to manufacturing processes.Great work articulating and demonstrating these methods.
This is a very generic reply. 1 you place yourself as an expert in the field. 2 you don't discuss any relevant point. 3 you don't present any conclusions. Typical bot comment.
As an engineer who spent many years in R&D, I just wanted to say thanks. Introducing system optimization and DOE concepts can help people understand that there are so many tools available to them. And, as you illustrated, their use is not confined just to labs and manufacturing. Your clear examples and approachable teaching style make your content so impactful to the community. Thanks again.
Another commenter has mentioned Bayesian Optimization, I just wanted to expand a bit on what the difference is, and why you might prefer a more general optimization framework over Taguchi arrays. First, it's important to mention that Taguchi arrays are only accounting for pairwise interactions: for any two variables we choose, all possible combinations of those two variables will appear in the array. In the wikipedia article on Orthogonal Arrays, they generalize this idea to choosing t variables, where t=2 is the Taguchi array, and t=n is the full factorial analysis. But we could also generate arrays for intermediate variables of t. These arrays would be helpful if we expect up to t variables to interact together to create an effect, which is often the case. But any time we are choosing t
My father (an engineer) tried to explain this to me (a chemist) a couple of years ago. Unfortunately, he didn't understand how it worked and couldn't explain the process. This video was extremely informative and makes me want to study up on Design of Experiments, since I have heard that it is an extremely useful field for scientists. Thanks for sharing this with us!
I have used these methods in industry. Your presentation and explanation of the methods are really wonderful. I can see this inspiring many young scientists to improve their methods and develop better ways of doing nearly anything.
And not only young, I can attest to. My husband who has been working with wastewater treatment processes for 20 years asked me today for the link to that video about test optimisation that I mentioned a month ago.
As a process engineer, I've completed plenty of DOEs ... we use a statistical software call JMP. Among other things, JMP will design and analyze DOEs, telling you which of your factors are significant. It even has prediction sliders, letting you virtually adjust the levels for each factor.
Haha, you work for the big I company too, huh?? Honestly, I f'ing hate JMP through - the most convoluted scripting language to actually get anything working. Once me and a co-worker spent an entire afternoon trying to figure out how to script something that took 2 seconds to do by hand, but was nearly impossible to get running in a JMP Script... and yes, we looked at the "journal" - it didn't help... apparently everything DOESN'T get logged to the journal like they claim... eventually we figured it out by googling and combining 3 different solutions from the JMP forums, but my god was it an ugly solution. Something that you could do in Python with Pandas easily without JMP'ing through so many hoops...
@@askquestionstrythingsminitab holds your hands a lot more. JMP seems a bit more powerful, but the results are harder to interpret without a solid statistical foundation.
I could use this to improve my air cannons in the future. I've always wondered how much air pressure affected velocity, and to what extent. But along with that there's also tank volume, projectile mass, barrel length, valve response time, flow rate, and how tightly the projectile sits in the barrel. Now I can test all of these variables without doing hundreds of tests!
I have an idea for you if you'd like. People who do gold sniping use a simple bottle and tube called a "Snuffer bottle". If you could reverse the mechanism of a squirt gun so that it pulled in water and material instead of pushing it, you could replace the traditional snuffer bottle. Call it a "Snuffer Gun". If you do this and get rich, please don't forget about meeeeeeee😅.
If I'm remembering fluid mech, you can find the relationships between some of those values with dimensional analysis, and then run an experiment to solve for the coefficients of your setup. That would give you performance curves that you can use to optimize the design.
This is one of the best UA-cam channels. The clear method of communication, worthwhile projects, and deep insights like this make me feel like I get more from these videos than most others. Keep it up, I love what you're doing.
I feel so guilty for never commenting about design of experiments (DOE) before. I use these daily in research, Taguchi array is the one I use to introduce persons to DOE. There are so many different methods, especially for non-linear (non-orthogonal) analysis. Another cool one is the Mixture DOE, which is mainly for recipes.
Looking into this topic I realized there is much more to learn, but Taguchi arrays at least seem like they should be common knowledge because of how much you can learn from them without going beyond very simple math.
@@Nighthawkinlight Kudos for even attempting to explain DoE in such a limited time. Not sure how far down the rabbit hole you've gone, but you can take your results and plot them in a 3d graph in excel to better visualize where the sweet spot is.
In case you are interessed I suggest to take a look at specific statistical software that would help you in gathering more info from your data. In particular Minitab is the preferred option in many industries including the one in which i work (i.e. Pharmaceutical development)
@@Nighthawkinlight "should be common knowledge" - i feel the same way about probability and statistics. it's so much more useful than trigonometry / calculus to most of the population that i think it's criminal to have public schools push students into trig / calc while sometimes not even offering prob / stat. it's the same kind of thinking that would lead one to a more intuitive understanding of the benefits of a taguchi array, IMO
I'm an industrial quality analyst and I am OBSESSED with Genichi Taguchi, especially his Loss Function. It's joked in QC that you have to have a doctorate in mathematics in order to fully understand his ideas, even though he wasn't a doctor, just an ingenious engineer. He's a founding father of Quality Control, as well as Kaoru Ishikawa (inventor of the Fishbone Diagram), Dr. W. Edwards Deming (the "father of quality control"), Dr. Walter Shewhart (the inventor of the Control Chart and a mentor to Deming), etc. Thanks for helping me understand Taguchi's experimentation method! I was never going to get it by just reading. the math is just insane
This is possibly one of the best videos I've seen on this platform. It reignited my passion to tune and refine a project (3D printer) that had me demoralized for ages because this method showed that I can combine multiple tests to reduce time and filament used for testing.
I have literally worked as a scientist and had never heard of this. In computer science we mostly just stuck with changing one thing at a time and slapping on more computing power :| Of course we also did random grid searches and then computed statistical tests on the results to still try to figure out something about the influence of each variable, but it wasn't nearly as elegant as specifically designing an experiment beforehand... Thanks a lot for this video! Looks like a super useful skill
As an egg farmer that tested egg pealing extensively, you are correct about egg age being the biggest factor. The factor you missed is cracking the eggs before cooling. My theory is flash cooling only loosens the shell if it is broken an can move as it contracts. Some say its because water can get in. But either way, cracking before cooling seems to help
@@jasonnickel384 Dont think I have ever pressure cooked eggs before. I hace both a stove top pressure cooker and an instapot. But just use them for big meals.
@@jasonnickel384 the best way is to steam your eggs, just use any vegetable steamer or a colander in a pot of water to steam them. Then refrigerate them assuming you make many at a time to eat on some other days down the week.. Once they are cooled the shell basically falls off. If you crack immediately after cooking it's still easier than any boiling method except maybe the vinegar. also it generally uses less energy to steam far less water than to boil a lot more water
Whether or not you crack before or after the post-cook ice bath is a factor I hadnt considered - I will definitely try that, thx. I do have something to add to all this, which is (1) the role of the membrane (which lies between the white and the shell), and (2) the cracking extent & technique used, and how it somewhat relates to (1). As for me, im a retired engineer who's just used to thinking analytically. My household is only 2 adults, but I love eggs, so we often use as many as a family of 4+. (1) Membrane: I believe this correlates with the degree of freshness factor, because as an egg ages the subtle permeability of the shell allows for some moisture loss over time, which in turn both increases the telltale size of the bubble at the big end, and reduces the degree of surface vaccum (i dont know the correct term) between the shell and the outside of the membrane. I believe this is the main mechanism behind why fresh eggs are noticeably harder to peel than less fresh examples. 2) Cracking Extent & Technique: First, consider how the face of a freshly cut slice of raw waxy potato tends to seize onto the smooth flat side of a chef knife (surface vaccum effects). With that in mind, i suspect the egg membrane of a hard boiled egg can have a similar effect on how easy/hard an egg is to peel to some degree. The next time you peel a bunch of eggs, pay attention in any given moment to whether you're peeling a piece of shell away from the membrane (which is harder), or if you're peeling shell and membrane together away from the white (which i find easier). My point is that ive found the shell comes off easier if you make sure you're UNDER the membrane every time ... which brings me to my larger point of cracking extent and technique. When dealing with surface vaccum adherence effects, the 2 biggest factors are the surface area (read: size of the current shell fragment being peeled) and the surface tension of the moisture between the cooked white and the interior side of the membrane ... if you think about it, cracking the shell more thoroughly not only reduces surface contact area with any given piece of shell, it also creates more points of compromise in the membrane, which allow it cling less to yhe white and also to tear apart more easily. A last subfactor to consider might be adding something (food grade obviously) to either the cooking water or ice bath that reduces surface tension of the moisture between cooked white and membrane. Soap, surfactants, emulsifiers, ph, temperature and even electric current can all affect that - not all seem appropriate for use with eggs, but clearly is an angle worth considering. Regardless, I think that may explain how a little vinegar helps. So, bottom line: For easier peeling, I believe it helpful to fragment the shell more thoroughly, but without compromising the egg white (if pushed too hard). It might be fun to play around with ultrasonics, but since i dont have that handy, i content myself with just cracking the shell gently but as thoroughly as possible. I start by laying down a single sheet of paper towel, for traction and absorption. Then I usually crack the big end, tilt about 20 degrees, then rotate it several times with gentle pressure to extend the cracking in a ring around the pole. Then I quickly repeat at the other end. Last, i turn the egg on its side, crack the equator, then gently rest my palm on it and roll the egg back and forth at least a half dozen times, gently rocking my hand to one side then the other until the shell is thoroughly cracked EVERYWHERE, from pole to pole. Then I roll a few more times until I feel the shell and membrane noticeably loosen. Then I peel. With practice, and eggs of optimal degree of freshness, im usually able to peel several eggs per minute. One caveat: if the eggs are a bit older than optimum, the white begin to weaken and are prone to tearing away with the membrane, so there is a definite bell curve to optimal freshness for peeling ... you dont want too fresh or too old, but rather something in the middle. I suspect the sweet spot is fresh enough for the boiled white to still be firm (and less prone to tearing), and old enough for moisture loss to create some space between the membrane and shell, but not so old that the white weakens and becomes prone to tearing. Sorry to ramble, but I think about stuff like this all the time.
According to J. Kenji Lopez-Alt who ran a ton of experiments on egg peeling, the number 1 variable that makes eggs easier to peel is the temperature of the water in the pot when you start boiling your eggs. If you place the eggs in a cold pot of water and slowly bring it up to temperature, they will tend to be much more difficult to peel. Always boil the water first, then place the eggs into the boiling water.
Good to know. Also, if you don't peel the eggs after boiling them and leave them for a later time (next day), from my experience they will peel easier if you keep them in water. My speculation is that the water keeps the membrane right under the shell hydrated so it doesn't stick to the egg white.
Glad to see another Kenji follower. His experiments and results have been the basis for many of my own recipes and tests, especially when it comes to sous vide! I immediately thought about his egg testing too!
This explains to me why I've struggled with it so much, and I'm excited to make deviled eggs with an hour of agonizingly removing shell superglued to the whites
@@busimagen thx I will give it a whirl. I do have one of those small machines to make hard-boiled eggs that does the same but it also starts from cold.
This is a lovely and uncomplicated way to do DOE, especially when your variables are discrete (or, as you did for this video, discretized). If you're interested in optimizing under continuous changes, which is probably especially relevant for chemical recipes, you may look into Bayesian Optimization. The method can be tuned depending on your variables but would already mostly work straight out of the box using a sensible default configuration. Let me know if you want to discuss this!
I am interested in learning about this, do you have any resources you would recommend? Everything I'm finding online revolves around data science but I am interested in learning it in the context of chemistry.
This was fantastic! It's inspired me to have a play with some genetic algorithms using this type of approach to programmatically evaluate the successive generations of code changes
I work with evolutionary algorithms for a living, and tuning the parameters is definitely one of the most difficult parts about using them. There are so many interacting parameters that can dramatically affect their behavior. I've tried things like adaptive parameter control, "self-adaptive" parameter control (which is a fascinating technique of letting solutions in the EA evolve their own parameters as it runs: if e.g. a high mutation rate was helpful in producing children with a higher fitness, then that high mutation rate gets passed down to the children), and even using a second EA to tune the parameters of the first EA in some kind of meta-evolution. But for some reason, despite studying this type of experiment design in school, I've never actually tried it out in practice for parameter tuning. I think I never realized how much potential time savings there was when you have so many variables and levels, since the examples in class were always tiny for demonstration purposes. I guess I should give it a try too.
I was thinking about using this as a mutations strategy as he was explaining it, though I think you're in luck! If you've ever heard of Covariance-Matrix-Adapatation, it's doing something closely related, after you generate a set a candidates, you select the top n candidates and create a covariance matrix of all their features, and then generate the next generation to be a spread of values with the same covariance distribution. It doesn't have the same kind of rigour but it has excellent coverage and convergence. If you wanted to adapt an evolutionary strategy from this, you could generate a Taguchi Array for low, medium, and high values of your n dimensions, and then use the covariance matrix trick to define your values for low medium and high. Is there anywhere I can follow your progress in case you pursue this? Because I'm extremely interested in novel genetic algorithms, and CMA is already super interesting for being state of the art while having very few hyper-parameters to deal with.
As a MSc in applied physics (20 yrs ago) with strong experimental part of my education I'm speechless that this method was never taught in uni. My mind is seriously blown. I will use this method in next week experiments to get maximum color separation in epoxy pour (it's heavy temperature dependent, at least 3 variables to play with). Thanks!
I have never heard of this method for experimental design. Thank you for sharing this will significantly speed up data gathering and even reduce the amount of experiments needed to be done saving resources
I don't know whose gonna read but I'll leave it out here. Systems design engineer here. I do DoEs day in and day out, Taguchi method is an awesome tool to narrow down the design search space. When there are a handful of inputs Xs for a measured output Y you can control and intuitively understand, this is all you need to optimize . When there are a large number of Xs, there's something engineers do which is called parameter sensitivity/pareto analysis. There are many fancy ways to do this pareto chart but I try to keep the intuitive part alive. I min-max normalize the inputs and output between 0 to 1 and do a multiple linear regression. The coefficients of Xs gives its impact on Y and its sign gives the direction. This helps eliminate less influential inputs to further reduce the dimensionality of search space. Also, a cool way to visualize them is using a parallel-coordinates chart 😁
I had a special class on DOE which covered the statistics and 2k factorial design for one of my technical electives. It's definitely something more engineers should study.
2k factorial and design of experiments is amazing. Sadly people (especially in industry) are not willing to make the effort to utilize these methods especially when the tests are time consuming or expensive. You did a great job explaining the benefits.
As soon as I saw the title I chuckled, "Taguchi Method". You did a good job of introducing the topic to lay-people, with a VERY simple and relatable experiment set. (I will certainly use some vinegar from here on). On any more sensitive ($$$) projects there are SO many more support elements behind defining the DoE and then interpreting / validating the results. PS: in the 80's we didn't have software for this... charts / calcs were done by hand.
That's an extremely useful tool that really needs to be more well-known. Hopefully you use your new-found knowledge to optimise some of your older projects too. I've always steamed my eggs and had no issues peeling them, but adding vinegar to the water when boiling them does make a lot of sense to do.
As a hobby, I play to do fermentation sodas. I stuck to the same recipe since each batch takes days or weeks to be ready. With this method I might be able to improve my results. Same when you grow plants from seeds!
This is incredible. It's like seeing a huge chunk of the world come into focus. A major part of how I think about problem solving has been rewritten in watching this. Thank you so much
I could see this working really well for recipe testing. A lot of cooking UA-cam channels talk about how important it is to only change one variable at a time while recipe testing, but I imagine that this could speed things up by a lot.
Awesome video! I am an engineer by trade and work in quality. Have used this method with great success in understanding process inputs on quality outcomes!
This type of experimentation is very common in industry and has been around over 80 years. I am always amazed this isn't used more in scientific research.
I know absolutely nothing about the method you used to improve your sparklers..however, the second I saw your score sheet at @20:00 alarm bells start going off inside my head. You are scoring your sparklers on non uniform parameters which which in itself is not a mistake but because your values are so different you are effectively washing out all contribution from effects which have a numerically smaller measurement. This is a very important factor to consider and correct for when using something like PCA on a larger dataset. Image for example if we want to guess a persons likely hood to get a heart disease based on their eye color, height, weight and age. If we measure height in millimeters rather than centimers we will have HUGE values which will overshadow the weight in Kg and the age in years! Even if we only measure in centimeters or meters we still have vastly different scales. Furthermore we cant assign numbers to eye color as 1=green and 2=blue because then blue eyes are more important than green just because we chose so! This means in order to have a well rounded sparkler you must standardize your scoreboard no matter how you measure success. You do this by subtracting the mean value of a score column from itself and then divide by the standard deviation. Do this for each column of results and now every column will have the same equal weight in your scoreboard. This would allow "climbing ember" scores to have an equal impact on your evaluation compared to the "bursting sparks" score no matter how big or small the numbers you assign are.
Yes you're exactly right. I weighted one sparking stage far heavier than the others. I did this on purpose but did not find a place in the script to talk about it. In this case I wanted to record the data from all stages so I could discover what contributed to each, but overall my priority for considering success was based on the first and most difficult to achieve stage.
I was introduced to the design of multivariate experiments in the early 1990's as an engineer producing optical metrology instruments used for semiconductor manufacturing. It changed my life as an engineer. Instead of endless meetings to prioritize which treatment conditions to vary, we could test them all in a Taguchi design and let analysis of variance figure out what really mattered. Higher resolution designs could be used to de-confound interactions between variables.
Thanks for the useful info. I had paused the video just before he mentioned Taguchi and read this comment, then went back to see him literally using those exact words. (Good reinforcement).
I slowly discovered that multivariable table over the course of several jobs in different industries... once I got that figured out, I also discovered another side effect of working in so many different industries via my automation and robotics skills... I was able to cross platform experiences and bring fascinating new insights into seemingly unrelated fields. While focusing into a niche position can help you gain a mastery of a given craft, there is also a strong value in the "Jack of All trades, Master of none" approach as well. By not getting stuck in a proverbial rut in areas that have "always been done that way", I was able to leapfrog things into new areas that provided substantial gains rather than the incremental ones at larger time and expense. Baseline idea is to definitely pursue whatever specialization interests you most, but also do not sacrifice reasonably passing knowledge in other fields... who knows what could be that "Eureka" moment gleaned from something seemingly wildly different then your current course or field.
Aha! Then you are the person I should ask. At 12:08 he shows us an example of an array of 4 variables, and 3 settings. It dropped the experiment count from 81 to 9. I noticed the result is the square root of the original count. Are there simple equations that will tell the reduction in experiments needed by using Taguchi arrays? Note: if the explains it in the last of the vid, nevermind the question. I have to go before the end.
@@jtadevich So this was a while ago, but from memory there's actually an empirical look-up table that you use to find the right array size for your experiment. There accordingly isn't a strict mathematical relationship between number of variables and number of experiments. Taguchi methodology is supposed to be used in industry to improve processes, so it's kind of rough-and-ready rather than being mathematically or statistically rigorous. It's biggest advantage is simplicity - allowing you to rapidly identify important variables and zoom in on the optimal region for that system.
I just realized that your channel is the one I've been continuously watching on and off basically since shortly after youtube started...and FAR longer than any other channel by like probably 8 years.😅 Really speaks to how your approach to topics even as the videos and topics shifted from me going like "ohhh shiny fireworks" to "hm this isn't something I ever thought or cared about but I'm thoroughly enjoying this!" over the years just captured my imagination for like 15 or so years at this point if I had to guess. Even though I have been subscribed for a fairly short time that's because I liked the idea of most creators and topics to just percolate back into the subconscious until they organically reemerged after months or years at which point I could go on a watch spree hehe. But as youtube, upload schedules and monetization changed so did my viewing behavior...also I'm getting older and there are so many quality channels now I just can't keep track anymore 😁 Anyways it was just a fun realization to have that this is the only channel for me that I've been watching for this long and I wanted to say thank you and show this tiny bit of appreciation, for whatever it's worth, and hope you have a great day!
Experimental design should be part of every technical education, along with statistical analysis. After I took a job doing R&D in polymer manufacturing, I soon realized that my education had not prepared me for the job. I taught myself with books and offsite seminars, and this soon became one of my most-used skills. I now believe that college professors should do a turn in industrial settings before teaching engineers and scientists what they will need. UA-cam videos like this provide a valyable service.
Can confirm. I taught factories to use multivariate DOE studies (Design Of Experiment). It does require good software like JMP or Minitab statistical analysis software, but quite often, it's the ONLY way to develop better, cheaper, and faster processes. Classic science where you only vary one parameter can only solve about 5% of real world problems!
This blows my mind tbh. I knew there must be a better method but I was never smart enough to figure it out. Might help 3D printing nerds with perfecting their quality without wasting 100s of hours and a ton of filament among other things. Also, I might finally get into cooking due to this haha. I don't mind eyeballing things 'to taste' but having a recipe on paper that is modified to your liking would be cool. Always love your videos because they are super easy to follow, I still use the button trick to hold pieces of fabric together.
It's delightful to see design of experiments get such a clear and concise treatment. There are a LOT of holes to fall down, and you managed to miss all of them. Well done!
This does look extremely useful. Especially when testing things like ratios where it's literally impossible to change just one thing at a time. When one goes up another goes down.
Congrat: you have now step up to the next Dunning-Kruger level. Even you, with your very open mind, clever, know it all, curious, etc There is always room to get surprised by our lack of knowledge!
Uh-oh... Brilliant better watch out... I think Eggland's Best is angling for a sponsor role! 😂 Seriously though, thanks for the shout-out and introducing me to Taguchi... It has absolutely changed the way I look at experimental design.
I learned this on accident through baking and applied it to engineering projects on motors and engines. Most helpful combos of batteries, charge times, engines, start times, motors, and max rpm are a lot of variables. This process works amazing for testing anything
I'm a laborer in a steel foundry, and I applied to a quality control position cause I am capable of so much more than just labor. This orthogonal array method will be awesome for me to bring up during the interview!
Taguchi arrays are new to me. I will try it out next time I need to debug code and there are multiple possible causes. Thanks for sharing this concept. It's sure to help a lot of people.
What breaks me often is thinking I've settled a test that I use to compare to others and when nothing works I go back, re-do that test and realize something went wrong and had been using a broken yard stick. Especially with chemistry or cooking and other "flimsy" subjects where it's very hard (for me) to make 2 "exact" runs of a particular set of variables.
Outstanding video! 🎇 I started a hydroponics experiment, to see if the Kratky method can replace the current way I've been growing vegetables. So, I started out with one tomato plant and one lettuce plant. I figure it's going to work, but the next step is going to need fine tuning. I only guessed at the amount of water and nutrients. How can I grow better vegetables? Room temperature, light level, starting water level, nutrient levels - nitrogen (N), phosphorus (P) and potassium (K), different nutrient types (AeroGarden vs mixing my own), jar size, should I add Hydrogen Peroxide (if so, how much?) It's so many variables that I started getting discouraged. That's a lot of tests and it could be different for tomatoes, strawberries, and lettuce… or even different types… Iceberg vs Romaine… Cherry Tomatoes or Bigger tomatoes. I was thinking, to properly measure the changes, I could only change one variable at once. So, when I played this video, I thought I was going to see a nice experiment… perhaps something with fire… but almost instantly I realized that Orthogonal / Taguchi arrays could dramatically simplify things. It's not only quicker, but your video shows that this method could reveal more information - possibilities previously unconsidered. Previously, I was just planning to eyeball it and get a rough estimate. But now, I might actually do legit science. 😄
I'm a mechanical engineering student and this seems incredibly useful for some of the projects I have done. In the past I have sometimes not done changes one at a time when speed was necessary, but this seems like such a much better way to gain insight. Thanks for the amazing video!
I've been doing this in the kitchen for years. Because whenever I run an experiment in my kitchen, I'll have to eat the results, so I'd much rather make the same thing 4 times than 8. I always considered it a bit of a dirty trick, though.
I'm a software engineer. I don't work in manufacturing. I don't have a PhD. I've really only heard of the term "multivariate testing" being used by colleagues in marketing. Never heard of DoE. Your video made this concept accessible enough that it pushed me to give it a try. It came in at a time where I was working on debugging ridiculously complex software that I didn't write, which can take ages and involves a lot of small changes and observation. So I wanted to thank you for taking the time to produce this video. It is truly helping out and elevating some of us.
Okay dude, are you reading my mind? I was experimenting with boiled eggs about the time you released this video. Exponential search on boiling times, peeling techniques, etc. >.>
This has to be on my top 5 list of most useful videos I've seen, thank you for making it. I have long strongly suspected there are valid ways to make several changes at once because even my disorganized methods worked but not as well as this more organized and thought out tool appears to.
1) Experimental design is the backbone of scientific inquiry and I would eagerly watch more from you on this topic. 2) Now I think I need to dive into the math behind making these tables, because that's a personal interest of mine.
Wonderful! Thank you for this information. My kids are getting to the age where science projects are beginning a thing, and I always turn to your channel for inspiration. We thank you for your thorough, organized and entertaining videos.
I took statistics in high school, laboratory chemistry classes in college, and I never heard about using orthogonal experimental arrays to test multiple variables at the same time. So thanks for sharing! And thanks to everyone in the comments for pointing me in the right direction if I ever want to learn more
What a wonderful, educational video! I’ve practiced medicine for almost 40 years and over that time, I’ve read a huge number of articles comparing different diagnostic and therapeutic methods. What I never came across is a single paper that used the Taguchi array to compare multiple variables in the test population. Actually, I never heard of the Taguchi array until watching this video. Thank you so much for all of your awesome videos!
Wow, thanks so much Ben! You were the reason that I started making these fireworks and I brute forced my way through hundreds (?) of hours of making them to get to this point, a fraction of the time you spent I'm sure. This method is absolutely a game changer. It's hard to describe that feeling of trying to solve a problem and being completely lost with no direction on where to go next. So happy you've improved the recipe and I can't wait to try it, and to try to improve on it with this method!!
Wow, that is amazing! I love looking at data and seeing significant information pop out of it. I’d been completely unaware of this method before; this is just *so* useful and important! Thanks!!!
Finally, thanks! This is such valuable information! Please continue your excellent work with all the due diligence you continue to invest in your world class instruction.
This motivates the method well enough to make it seem like the necessity it is. Rather than the menial curiosity it feels like in most presentations. As per the MLA Handbook you don't need detractive statements in your scripts. Your work speaks for itself.
If you had made this video 6 months earlier, my master thesis would have been so much easier ! Your explanations are very clear and would have saved me a lot of headaches!
Thanks for figuring it out. Years past, someone came up with this concept but never disclosed the formula other than it was simple in home chemicals. He did his demonstration to the media even showing cooking an egg without damaging the interior. If he did patent the formula it should be available. Looking forward to experimenting with new tiles too produce tile better high efficiency wood stoves. Best
I remember how i was in my early teens watching Nighthawkinlight and the king of random diy projects after school, amazing how im now in my mid 20 and still enjoy your videos!
1:00 What's funny is back in HS Bio (which I wound up with the top score in the school that year), we had an experiment on dissolving gummy bears. The variables were something like: Salinity of the water, Level of sunlight, Volume of water, % Vinegar etc... I misunderstood the instructions at first, thinking we needed to test them all, and so came up with a "3D Test Grid" wherein we could test 3 different variables at a time AND test their compounded effects. So, X = Salinity increases; Y = Sunlight levels/distance from light/heat source; and Z = Vinegar %. Or something like that. And drew out a cubic grid to allow for about 5 models for each arm of the experiment, for a total of about 125 mixed models. 0,0,0 being the control batch with no salinity, no light/heat, and no vinegar. Recording how much was dissolved every so many units of time. The teacher pretty quickly shut that down though.
Nighthawk, I have done this and so many other experiments on the boiled egg. The best I have ever found hands down is to tap on the boiled egg with a spoon until it pops. you will actually hear the sound. It works every time.
Always a treat to watch one of your videos. Eloquently narrated, and always some idea or take on a subject that i didn't know or hadn't considered. And I especially admire that you shout out other intelligent creators
i loved this! thank you for articulation, this was exactly the right moment to stumble upon in my life with my biochemie experiments which leaning towards to be more complex gradually. hero!
I really enjoyed this video! I'm amused by the notion that I got excited about experimental methods and slow methodical work. My inner kid is saying it's boring compared to doing more active things. But my older self has dealt with so many questions that I did not have the knowledge or tools to find answers to. So seeing someone find answers to questions they've been working towards is like breaking through a wall. And now my inner kid is back on board because breaking walls seems fun!
I'm impressed! Your explanation of this process is MANY times more concise and MUCH clearer than the dreary explanations I learned in school. THANK YOU!
Thank you! I’ve always been stuck in the changing one variable realm and was highly skeptical before watching the video. You have opened my eyes and I thank you for it!
Loved this as I had never heard of this technique. I have to say after the first few mins I thought it looked needlessly complicated and would lose value against the normal factorial approach. So was blown away by how it still captured the normal outputs but also gave even more information (but with less tests). Very well demonstrated and glad you showed such different examples. Thanks for sharing your learning as always.
As a programmer i also want to highlight the method of trying the same thing again, without changing anything, expecting different results.
The definition of insanity...
That's all fun and games until it works o.0
Gpupdate and reboot a 3rd time, maybe that'll fix it
@@TallMilk Add to that all the times, when looking over that "perfect" code, you find yourself saying "wtf was I thinking when I wrote this part"
ALWAYS account for 'human error'. We are not precision machines, and that fact should be added as a factor.
I'm gonna go out on a limb here and guess that this is the only video in the entire universe that covers boiling eggs and making Senko Hanabi in the same video.
What about boiling eggs using Senko Hanabi? Mega Senko Hanabi? Mini Eggs?
On earth, maybe. But there's a few planets that do this regularly.
what if we make the lamp black with burning eggs hmm
Is there a tiny hole drilled in the shell before boiling...?
IKR.. no hole no worky worky. Eggz explode when no hole.👍👍
I spent 20 years working materials and process development in the semiconductor industry and these design of experiment (DOE) methods are foundational to quickly identifying meaningful improvements to manufacturing processes.Great work articulating and demonstrating these methods.
Please recommend course so we can learn more
I learned from this book. Practical Experiment Designs for Engineers and Scientists by William J. Diamond
This is a very generic reply. 1 you place yourself as an expert in the field. 2 you don't discuss any relevant point. 3 you don't present any conclusions. Typical bot comment.
@garrysekelli6776 are you a fun guy? yknow, at parties?
@@dimm__ ask urmoomson.
As an engineer who spent many years in R&D, I just wanted to say thanks. Introducing system optimization and DOE concepts can help people understand that there are so many tools available to them. And, as you illustrated, their use is not confined just to labs and manufacturing. Your clear examples and approachable teaching style make your content so impactful to the community. Thanks again.
Another commenter has mentioned Bayesian Optimization, I just wanted to expand a bit on what the difference is, and why you might prefer a more general optimization framework over Taguchi arrays. First, it's important to mention that Taguchi arrays are only accounting for pairwise interactions: for any two variables we choose, all possible combinations of those two variables will appear in the array. In the wikipedia article on Orthogonal Arrays, they generalize this idea to choosing t variables, where t=2 is the Taguchi array, and t=n is the full factorial analysis. But we could also generate arrays for intermediate variables of t. These arrays would be helpful if we expect up to t variables to interact together to create an effect, which is often the case. But any time we are choosing t
I'm just commenting because I've made similarly long comments before that took a lot of hard work and I hate it when nobody bothers to read it xD
Yeah, gonna look at this.
@@Metal_Master_YT I can definitely relate, lol. It's fascinating to learn how deep the field of Design of Experiment (DoE) goes.
@@porkeyminch8044 I just want my experiments to work the first dang time...
Incredible comment. Thanks so much for sharing I had no idea this existed, nor the tradeoffs of the Taguchi Arrays. Thanks!!
My father (an engineer) tried to explain this to me (a chemist) a couple of years ago. Unfortunately, he didn't understand how it worked and couldn't explain the process. This video was extremely informative and makes me want to study up on Design of Experiments, since I have heard that it is an extremely useful field for scientists. Thanks for sharing this with us!
Chemistry sucks
I have used these methods in industry. Your presentation and explanation of the methods are really wonderful. I can see this inspiring many young scientists to improve their methods and develop better ways of doing nearly anything.
Thanks for the encouragement!
Exactly the feeling I'm getting watching it. Drive to improve the way I measure effectiveness in both work and personal life.
And not only young, I can attest to. My husband who has been working with wastewater treatment processes for 20 years asked me today for the link to that video about test optimisation that I mentioned a month ago.
As a process engineer, I've completed plenty of DOEs ... we use a statistical software call JMP. Among other things, JMP will design and analyze DOEs, telling you which of your factors are significant. It even has prediction sliders, letting you virtually adjust the levels for each factor.
Haha, you work for the big I company too, huh?? Honestly, I f'ing hate JMP through - the most convoluted scripting language to actually get anything working. Once me and a co-worker spent an entire afternoon trying to figure out how to script something that took 2 seconds to do by hand, but was nearly impossible to get running in a JMP Script... and yes, we looked at the "journal" - it didn't help... apparently everything DOESN'T get logged to the journal like they claim... eventually we figured it out by googling and combining 3 different solutions from the JMP forums, but my god was it an ugly solution. Something that you could do in Python with Pandas easily without JMP'ing through so many hoops...
Same here great fun loved this part of the job.
We are saddled with using minitab for our DOE analysis but in school it was all done in R. I wonder how JMP compares.
@@askquestionstrythingsminitab holds your hands a lot more. JMP seems a bit more powerful, but the results are harder to interpret without a solid statistical foundation.
@@gorak9000haha, sounds familiar. But it's not just Intel that use JMP; it's used fairly widely in the industry. And yes, JSL sucks.
I could use this to improve my air cannons in the future. I've always wondered how much air pressure affected velocity, and to what extent. But along with that there's also tank volume, projectile mass, barrel length, valve response time, flow rate, and how tightly the projectile sits in the barrel. Now I can test all of these variables without doing hundreds of tests!
I have an idea for you if you'd like. People who do gold sniping use a simple bottle and tube called a "Snuffer bottle". If you could reverse the mechanism of a squirt gun so that it pulled in water and material instead of pushing it, you could replace the traditional snuffer bottle. Call it a "Snuffer Gun".
If you do this and get rich, please don't forget about meeeeeeee😅.
@@GimmeOo-mox It already exists, it's called a hand dredge :)
@@whynotdean8966 I didn't know... But now I know. Lol.
@@whynotdean8966 Actually, I knew about the hand dredge but the concept I'm thinking of is much smaller and would replace snuffer bottles.
If I'm remembering fluid mech, you can find the relationships between some of those values with dimensional analysis, and then run an experiment to solve for the coefficients of your setup. That would give you performance curves that you can use to optimize the design.
Senko, microspheres, and process improvement all in one video is such a heavy hitter, this is peak UA-cam content
YES
This is one of the best UA-cam channels. The clear method of communication, worthwhile projects, and deep insights like this make me feel like I get more from these videos than most others. Keep it up, I love what you're doing.
I feel so guilty for never commenting about design of experiments (DOE) before. I use these daily in research, Taguchi array is the one I use to introduce persons to DOE. There are so many different methods, especially for non-linear (non-orthogonal) analysis. Another cool one is the Mixture DOE, which is mainly for recipes.
Looking into this topic I realized there is much more to learn, but Taguchi arrays at least seem like they should be common knowledge because of how much you can learn from them without going beyond very simple math.
@@Nighthawkinlight Kudos for even attempting to explain DoE in such a limited time. Not sure how far down the rabbit hole you've gone, but you can take your results and plot them in a 3d graph in excel to better visualize where the sweet spot is.
In case you are interessed I suggest to take a look at specific statistical software that would help you in gathering more info from your data. In particular Minitab is the preferred option in many industries including the one in which i work (i.e. Pharmaceutical development)
@@Nighthawkinlight "should be common knowledge" - i feel the same way about probability and statistics. it's so much more useful than trigonometry / calculus to most of the population that i think it's criminal to have public schools push students into trig / calc while sometimes not even offering prob / stat. it's the same kind of thinking that would lead one to a more intuitive understanding of the benefits of a taguchi array, IMO
@@marcolivio3128 do you know of any free alternatives?
I'm an industrial quality analyst and I am OBSESSED with Genichi Taguchi, especially his Loss Function. It's joked in QC that you have to have a doctorate in mathematics in order to fully understand his ideas, even though he wasn't a doctor, just an ingenious engineer. He's a founding father of Quality Control, as well as Kaoru Ishikawa (inventor of the Fishbone Diagram), Dr. W. Edwards Deming (the "father of quality control"), Dr. Walter Shewhart (the inventor of the Control Chart and a mentor to Deming), etc.
Thanks for helping me understand Taguchi's experimentation method! I was never going to get it by just reading. the math is just insane
This is possibly one of the best videos I've seen on this platform. It reignited my passion to tune and refine a project (3D printer) that had me demoralized for ages because this method showed that I can combine multiple tests to reduce time and filament used for testing.
How did I not know about that. It's so useful. Thanks for enlightening me.
That's what I said when first hearing about this!
I have literally worked as a scientist and had never heard of this. In computer science we mostly just stuck with changing one thing at a time and slapping on more computing power :|
Of course we also did random grid searches and then computed statistical tests on the results to still try to figure out something about the influence of each variable, but it wasn't nearly as elegant as specifically designing an experiment beforehand...
Thanks a lot for this video! Looks like a super useful skill
As an egg farmer that tested egg pealing extensively, you are correct about egg age being the biggest factor. The factor you missed is cracking the eggs before cooling. My theory is flash cooling only loosens the shell if it is broken an can move as it contracts. Some say its because water can get in. But either way, cracking before cooling seems to help
Eggs peel perfectly EVERY time if you cook them in an InstantPot (or any other pressure cooker).
@@jasonnickel384 Dont think I have ever pressure cooked eggs before. I hace both a stove top pressure cooker and an instapot. But just use them for big meals.
@@jasonnickel384 the best way is to steam your eggs, just use any vegetable steamer or a colander in a pot of water to steam them. Then refrigerate them assuming you make many at a time to eat on some other days down the week.. Once they are cooled the shell basically falls off.
If you crack immediately after cooking it's still easier than any boiling method except maybe the vinegar.
also it generally uses less energy to steam far less water than to boil a lot more water
Unfortunate that the biggest factor is the age of the egg, and that wasn't included in the experimental design...
Whether or not you crack before or after the post-cook ice bath is a factor I hadnt considered - I will definitely try that, thx.
I do have something to add to all this, which is (1) the role of the membrane (which lies between the white and the shell), and (2) the cracking extent & technique used, and how it somewhat relates to (1).
As for me, im a retired engineer who's just used to thinking analytically. My household is only 2 adults, but I love eggs, so we often use as many as a family of 4+.
(1) Membrane: I believe this correlates with the degree of freshness factor, because as an egg ages the subtle permeability of the shell allows for some moisture loss over time, which in turn both increases the telltale size of the bubble at the big end, and reduces the degree of surface vaccum (i dont know the correct term) between the shell and the outside of the membrane. I believe this is the main mechanism behind why fresh eggs are noticeably harder to peel than less fresh examples.
2) Cracking Extent & Technique: First, consider how the face of a freshly cut slice of raw waxy potato tends to seize onto the smooth flat side of a chef knife (surface vaccum effects). With that in mind, i suspect the egg membrane of a hard boiled egg can have a similar effect on how easy/hard an egg is to peel to some degree. The next time you peel a bunch of eggs, pay attention in any given moment to whether you're peeling a piece of shell away from the membrane (which is harder), or if you're peeling shell and membrane together away from the white (which i find easier). My point is that ive found the shell comes off easier if you make sure you're UNDER the membrane every time ... which brings me to my larger point of cracking extent and technique. When dealing with surface vaccum adherence effects, the 2 biggest factors are the surface area (read: size of the current shell fragment being peeled) and the surface tension of the moisture between the cooked white and the interior side of the membrane ... if you think about it, cracking the shell more thoroughly not only reduces surface contact area with any given piece of shell, it also creates more points of compromise in the membrane, which allow it cling less to yhe white and also to tear apart more easily. A last subfactor to consider might be adding something (food grade obviously) to either the cooking water or ice bath that reduces surface tension of the moisture between cooked white and membrane. Soap, surfactants, emulsifiers, ph, temperature and even electric current can all affect that - not all seem appropriate for use with eggs, but clearly is an angle worth considering. Regardless, I think that may explain how a little vinegar helps.
So, bottom line: For easier peeling, I believe it helpful to fragment the shell more thoroughly, but without compromising the egg white (if pushed too hard). It might be fun to play around with ultrasonics, but since i dont have that handy, i content myself with just cracking the shell gently but as thoroughly as possible. I start by laying down a single sheet of paper towel, for traction and absorption. Then I usually crack the big end, tilt about 20 degrees, then rotate it several times with gentle pressure to extend the cracking in a ring around the pole. Then I quickly repeat at the other end. Last, i turn the egg on its side, crack the equator, then gently rest my palm on it and roll the egg back and forth at least a half dozen times, gently rocking my hand to one side then the other until the shell is thoroughly cracked EVERYWHERE, from pole to pole. Then I roll a few more times until I feel the shell and membrane noticeably loosen. Then I peel. With practice, and eggs of optimal degree of freshness, im usually able to peel several eggs per minute.
One caveat: if the eggs are a bit older than optimum, the white begin to weaken and are prone to tearing away with the membrane, so there is a definite bell curve to optimal freshness for peeling ... you dont want too fresh or too old, but rather something in the middle. I suspect the sweet spot is fresh enough for the boiled white to still be firm (and less prone to tearing), and old enough for moisture loss to create some space between the membrane and shell, but not so old that the white weakens and becomes prone to tearing.
Sorry to ramble, but I think about stuff like this all the time.
According to J. Kenji Lopez-Alt who ran a ton of experiments on egg peeling, the number 1 variable that makes eggs easier to peel is the temperature of the water in the pot when you start boiling your eggs. If you place the eggs in a cold pot of water and slowly bring it up to temperature, they will tend to be much more difficult to peel. Always boil the water first, then place the eggs into the boiling water.
Good to know. Also, if you don't peel the eggs after boiling them and leave them for a later time (next day), from my experience they will peel easier if you keep them in water. My speculation is that the water keeps the membrane right under the shell hydrated so it doesn't stick to the egg white.
Glad to see another Kenji follower. His experiments and results have been the basis for many of my own recipes and tests, especially when it comes to sous vide! I immediately thought about his egg testing too!
This explains to me why I've struggled with it so much, and I'm excited to make deviled eggs with an hour of agonizingly removing shell superglued to the whites
One time when I put them in boiling water they cracked and some of the white oozed out. Do not recommend.
@@busimagen thx I will give it a whirl. I do have one of those small machines to make hard-boiled eggs that does the same but it also starts from cold.
This is a lovely and uncomplicated way to do DOE, especially when your variables are discrete (or, as you did for this video, discretized). If you're interested in optimizing under continuous changes, which is probably especially relevant for chemical recipes, you may look into Bayesian Optimization. The method can be tuned depending on your variables but would already mostly work straight out of the box using a sensible default configuration. Let me know if you want to discuss this!
I am interested in learning about this, do you have any resources you would recommend? Everything I'm finding online revolves around data science but I am interested in learning it in the context of chemistry.
This was fantastic! It's inspired me to have a play with some genetic algorithms using this type of approach to programmatically evaluate the successive generations of code changes
Holy crap that’s genius.
Talk about a method for optimization and bug catching!!!
I work with evolutionary algorithms for a living, and tuning the parameters is definitely one of the most difficult parts about using them. There are so many interacting parameters that can dramatically affect their behavior. I've tried things like adaptive parameter control, "self-adaptive" parameter control (which is a fascinating technique of letting solutions in the EA evolve their own parameters as it runs: if e.g. a high mutation rate was helpful in producing children with a higher fitness, then that high mutation rate gets passed down to the children), and even using a second EA to tune the parameters of the first EA in some kind of meta-evolution. But for some reason, despite studying this type of experiment design in school, I've never actually tried it out in practice for parameter tuning. I think I never realized how much potential time savings there was when you have so many variables and levels, since the examples in class were always tiny for demonstration purposes. I guess I should give it a try too.
Good
I was thinking about using this as a mutations strategy as he was explaining it, though I think you're in luck!
If you've ever heard of Covariance-Matrix-Adapatation, it's doing something closely related, after you generate a set a candidates, you select the top n candidates and create a covariance matrix of all their features, and then generate the next generation to be a spread of values with the same covariance distribution.
It doesn't have the same kind of rigour but it has excellent coverage and convergence.
If you wanted to adapt an evolutionary strategy from this, you could generate a Taguchi Array for low, medium, and high values of your n dimensions, and then use the covariance matrix trick to define your values for low medium and high.
Is there anywhere I can follow your progress in case you pursue this? Because I'm extremely interested in novel genetic algorithms, and CMA is already super interesting for being state of the art while having very few hyper-parameters to deal with.
As a MSc in applied physics (20 yrs ago) with strong experimental part of my education I'm speechless that this method was never taught in uni. My mind is seriously blown. I will use this method in next week experiments to get maximum color separation in epoxy pour (it's heavy temperature dependent, at least 3 variables to play with). Thanks!
Let me know how it works for you!
I have never heard of this method for experimental design. Thank you for sharing this will significantly speed up data gathering and even reduce the amount of experiments needed to be done saving resources
I don't know whose gonna read but I'll leave it out here. Systems design engineer here. I do DoEs day in and day out, Taguchi method is an awesome tool to narrow down the design search space. When there are a handful of inputs Xs for a measured output Y you can control and intuitively understand, this is all you need to optimize . When there are a large number of Xs, there's something engineers do which is called parameter sensitivity/pareto analysis. There are many fancy ways to do this pareto chart but I try to keep the intuitive part alive. I min-max normalize the inputs and output between 0 to 1 and do a multiple linear regression. The coefficients of Xs gives its impact on Y and its sign gives the direction. This helps eliminate less influential inputs to further reduce the dimensionality of search space. Also, a cool way to visualize them is using a parallel-coordinates chart 😁
I'm an engineer and i have never used this method! Now I'm excited to use this for my experiments!
I had a special class on DOE which covered the statistics and 2k factorial design for one of my technical electives. It's definitely something more engineers should study.
Same here! It's hard to believe it will work as well as the video shows. I will find out for myself.
2k factorial and design of experiments is amazing. Sadly people (especially in industry) are not willing to make the effort to utilize these methods especially when the tests are time consuming or expensive. You did a great job explaining the benefits.
As soon as I saw the title I chuckled, "Taguchi Method".
You did a good job of introducing the topic to lay-people, with a VERY simple and relatable experiment set. (I will certainly use some vinegar from here on).
On any more sensitive ($$$) projects there are SO many more support elements behind defining the DoE and then interpreting / validating the results.
PS: in the 80's we didn't have software for this... charts / calcs were done by hand.
Multivariate analysis was one of my favorite things about my STEM classes. It's a shame it's been missing from hobby science!
That's an extremely useful tool that really needs to be more well-known. Hopefully you use your new-found knowledge to optimise some of your older projects too.
I've always steamed my eggs and had no issues peeling them, but adding vinegar to the water when boiling them does make a lot of sense to do.
Steamed... hmm, ok I will give that a try as well... Thanks for the tip.
As a hobby, I play to do fermentation sodas. I stuck to the same recipe since each batch takes days or weeks to be ready. With this method I might be able to improve my results. Same when you grow plants from seeds!
I thought the same about beer brewing. There are so many possible variables and long cycle times. Or bread making, for that matter. This is exciting!
it always surprises me when each video you post is better than the last
This is why we love NightHawkinLight 😁
You should put your surprise into an orthogonal table and see what you get.
This is incredible. It's like seeing a huge chunk of the world come into focus. A major part of how I think about problem solving has been rewritten in watching this. Thank you so much
I could see this working really well for recipe testing. A lot of cooking UA-cam channels talk about how important it is to only change one variable at a time while recipe testing, but I imagine that this could speed things up by a lot.
Awesome video! I am an engineer by trade and work in quality. Have used this method with great success in understanding process inputs on quality outcomes!
This type of experimentation is very common in industry and has been around over 80 years. I am always amazed this isn't used more in scientific research.
I know absolutely nothing about the method you used to improve your sparklers..however, the second I saw your score sheet at @20:00 alarm bells start going off inside my head. You are scoring your sparklers on non uniform parameters which which in itself is not a mistake but because your values are so different you are effectively washing out all contribution from effects which have a numerically smaller measurement. This is a very important factor to consider and correct for when using something like PCA on a larger dataset. Image for example if we want to guess a persons likely hood to get a heart disease based on their eye color, height, weight and age. If we measure height in millimeters rather than centimers we will have HUGE values which will overshadow the weight in Kg and the age in years! Even if we only measure in centimeters or meters we still have vastly different scales. Furthermore we cant assign numbers to eye color as 1=green and 2=blue because then blue eyes are more important than green just because we chose so!
This means in order to have a well rounded sparkler you must standardize your scoreboard no matter how you measure success. You do this by subtracting the mean value of a score column from itself and then divide by the standard deviation. Do this for each column of results and now every column will have the same equal weight in your scoreboard.
This would allow "climbing ember" scores to have an equal impact on your evaluation compared to the "bursting sparks" score no matter how big or small the numbers you assign are.
Yes you're exactly right. I weighted one sparking stage far heavier than the others. I did this on purpose but did not find a place in the script to talk about it. In this case I wanted to record the data from all stages so I could discover what contributed to each, but overall my priority for considering success was based on the first and most difficult to achieve stage.
@@Nighthawkinlight Fair enough :D
This gave me a whole new perspective on how to run software A/B split tests to optimize customer conversion rate. Thank you!
Oh no! the last thing we need is advertisers getting better at brainwashing! LOL
In my company this technique known as Design of Experiment (DoE). And AFAIK this method also being taught at Engineering Master class.
I was introduced to the design of multivariate experiments in the early 1990's as an engineer producing optical metrology instruments used for semiconductor manufacturing. It changed my life as an engineer. Instead of endless meetings to prioritize which treatment conditions to vary, we could test them all in a Taguchi design and let analysis of variance figure out what really mattered. Higher resolution designs could be used to de-confound interactions between variables.
Thanks for the useful info. I had paused the video just before he mentioned Taguchi and read this comment, then went back to see him literally using those exact words. (Good reinforcement).
I slowly discovered that multivariable table over the course of several jobs in different industries... once I got that figured out, I also discovered another side effect of working in so many different industries via my automation and robotics skills... I was able to cross platform experiences and bring fascinating new insights into seemingly unrelated fields. While focusing into a niche position can help you gain a mastery of a given craft, there is also a strong value in the "Jack of All trades, Master of none" approach as well. By not getting stuck in a proverbial rut in areas that have "always been done that way", I was able to leapfrog things into new areas that provided substantial gains rather than the incremental ones at larger time and expense.
Baseline idea is to definitely pursue whatever specialization interests you most, but also do not sacrifice reasonably passing knowledge in other fields... who knows what could be that "Eureka" moment gleaned from something seemingly wildly different then your current course or field.
100% true.
Sweet, I did my masters on using Taguchi arrays to improve a biotech system.
I've long thought that this process doesn't get enough love...
Aha! Then you are the person I should ask. At 12:08 he shows us an example of an array of 4 variables, and 3 settings. It dropped the experiment count from 81 to 9. I noticed the result is the square root of the original count. Are there simple equations that will tell the reduction in experiments needed by using Taguchi arrays?
Note: if the explains it in the last of the vid, nevermind the question. I have to go before the end.
@@jtadevich So this was a while ago, but from memory there's actually an empirical look-up table that you use to find the right array size for your experiment. There accordingly isn't a strict mathematical relationship between number of variables and number of experiments. Taguchi methodology is supposed to be used in industry to improve processes, so it's kind of rough-and-ready rather than being mathematically or statistically rigorous. It's biggest advantage is simplicity - allowing you to rapidly identify important variables and zoom in on the optimal region for that system.
@@sangomasmith thanks. That is awesome. I have studied science for a long time, and realizing how much time this method can save is astounding to me.
Yup factororal experimentation is a big time time and resource savingings. Software really help to design and analyze the exp.
this was a WAY more engaged presentation of DOEs than my manufacturing classes, six sigma classes, and general online reading. well done!!
This is amazing! I'm gping to start using this in my own experiments!
I just realized that your channel is the one I've been continuously watching on and off basically since shortly after youtube started...and FAR longer than any other channel by like probably 8 years.😅
Really speaks to how your approach to topics even as the videos and topics shifted from me going like "ohhh shiny fireworks" to "hm this isn't something I ever thought or cared about but I'm thoroughly enjoying this!" over the years just captured my imagination for like 15 or so years at this point if I had to guess.
Even though I have been subscribed for a fairly short time that's because I liked the idea of most creators and topics to just percolate back into the subconscious until they organically reemerged after months or years at which point I could go on a watch spree hehe.
But as youtube, upload schedules and monetization changed so did my viewing behavior...also I'm getting older and there are so many quality channels now I just can't keep track anymore 😁
Anyways it was just a fun realization to have that this is the only channel for me that I've been watching for this long and I wanted to say thank you and show this tiny bit of appreciation, for whatever it's worth, and hope you have a great day!
That's nice to hear, thanks for sticking around!
UA-cam's been around a LOT longer than 8 years...
@@gorak9000 I said I had been watching his channel for 8 years LONGER than any other channel I'm currently watching, around 15 or 16 years in total. 😉
Experimental design should be part of every technical education, along with statistical analysis. After I took a job doing R&D in polymer manufacturing, I soon realized that my education had not prepared me for the job. I taught myself with books and offsite seminars, and this soon became one of my most-used skills.
I now believe that college professors should do a turn in industrial settings before teaching engineers and scientists what they will need. UA-cam videos like this provide a valyable service.
Can confirm. I taught factories to use multivariate DOE studies (Design Of Experiment). It does require good software like JMP or Minitab statistical analysis software, but quite often, it's the ONLY way to develop better, cheaper, and faster processes. Classic science where you only vary one parameter can only solve about 5% of real world problems!
This was really insightful and helpful thank you 😊
This blows my mind tbh. I knew there must be a better method but I was never smart enough to figure it out. Might help 3D printing nerds with perfecting their quality without wasting 100s of hours and a ton of filament among other things. Also, I might finally get into cooking due to this haha. I don't mind eyeballing things 'to taste' but having a recipe on paper that is modified to your liking would be cool. Always love your videos because they are super easy to follow, I still use the button trick to hold pieces of fabric together.
It's delightful to see design of experiments get such a clear and concise treatment. There are a LOT of holes to fall down, and you managed to miss all of them. Well done!
I've only seen this in a text book, your examples were top notch and explained vary well!
You're doing a great job with these videos!
Probably your most valuable video to date. Thanks as always :)
This does look extremely useful. Especially when testing things like ratios where it's literally impossible to change just one thing at a time. When one goes up another goes down.
Congrat: you have now step up to the next Dunning-Kruger level.
Even you, with your very open mind, clever, know it all, curious, etc There is always room to get surprised by our lack of knowledge!
Uh-oh... Brilliant better watch out... I think Eggland's Best is angling for a sponsor role! 😂 Seriously though, thanks for the shout-out and introducing me to Taguchi... It has absolutely changed the way I look at experimental design.
I learned this on accident through baking and applied it to engineering projects on motors and engines. Most helpful combos of batteries, charge times, engines, start times, motors, and max rpm are a lot of variables. This process works amazing for testing anything
I'm a laborer in a steel foundry, and I applied to a quality control position cause I am capable of so much more than just labor. This orthogonal array method will be awesome for me to bring up during the interview!
did you get the job?
Taguchi arrays are new to me. I will try it out next time I need to debug code and there are multiple possible causes. Thanks for sharing this concept. It's sure to help a lot of people.
What breaks me often is thinking I've settled a test that I use to compare to others and when nothing works I go back, re-do that test and realize something went wrong and had been using a broken yard stick. Especially with chemistry or cooking and other "flimsy" subjects where it's very hard (for me) to make 2 "exact" runs of a particular set of variables.
Outstanding video! 🎇
I started a hydroponics experiment, to see if the Kratky method can replace the current way I've been growing vegetables. So, I started out with one tomato plant and one lettuce plant. I figure it's going to work, but the next step is going to need fine tuning. I only guessed at the amount of water and nutrients. How can I grow better vegetables?
Room temperature, light level, starting water level, nutrient levels - nitrogen (N), phosphorus (P) and potassium (K), different nutrient types (AeroGarden vs mixing my own), jar size, should I add Hydrogen Peroxide (if so, how much?)
It's so many variables that I started getting discouraged. That's a lot of tests and it could be different for tomatoes, strawberries, and lettuce… or even different types… Iceberg vs Romaine… Cherry Tomatoes or Bigger tomatoes.
I was thinking, to properly measure the changes, I could only change one variable at once. So, when I played this video, I thought I was going to see a nice experiment… perhaps something with fire… but almost instantly I realized that Orthogonal / Taguchi arrays could dramatically simplify things.
It's not only quicker, but your video shows that this method could reveal more information - possibilities previously unconsidered. Previously, I was just planning to eyeball it and get a rough estimate. But now, I might actually do legit science. 😄
Very cool how the method of testing can make such a difference. Btw, I hope you didn't have to eat all those eggs alone 😂
I did eat a LOT of eggs.
@@Nighthawkinlight Deviled eggs, egg salad, egg sandwiches, scotch eggs...
@@whifflingtove Egg gumbo, egg tar tar, fish eggs ... Sorry, couldn't resist the Forest Gump reference xD
@@3nertia Scrambled eggs, fried eggs, poached eggs, eggs benedict..
His parrot take care of eating all of those eggs, don't worry about that.
I'm a mechanical engineering student and this seems incredibly useful for some of the projects I have done. In the past I have sometimes not done changes one at a time when speed was necessary, but this seems like such a much better way to gain insight. Thanks for the amazing video!
Only a few minutes in but this couldn't have come at a better time as I try to perfect my conductive painting process.
This is potentially my favorite video of yours so far.
I've been doing this in the kitchen for years. Because whenever I run an experiment in my kitchen, I'll have to eat the results, so I'd much rather make the same thing 4 times than 8. I always considered it a bit of a dirty trick, though.
This is absolutely amazing. Never heard of orthogonal/Taguchi arrays before, and your demo blew my mind.
I'm a software engineer. I don't work in manufacturing. I don't have a PhD. I've really only heard of the term "multivariate testing" being used by colleagues in marketing. Never heard of DoE. Your video made this concept accessible enough that it pushed me to give it a try. It came in at a time where I was working on debugging ridiculously complex software that I didn't write, which can take ages and involves a lot of small changes and observation. So I wanted to thank you for taking the time to produce this video. It is truly helping out and elevating some of us.
Okay dude, are you reading my mind? I was experimenting with boiled eggs about the time you released this video. Exponential search on boiling times, peeling techniques, etc. >.>
Truly incredible. I’m a PHD student in EE and this will significantly improve my research speeds.
Also you’ve earned an immediate sub from me.
I think this will make a huge difference in a bunch of my hobbies. Thank-you so much. Challenges that were very daunting seem doable now.
This has to be on my top 5 list of most useful videos I've seen, thank you for making it. I have long strongly suspected there are valid ways to make several changes at once because even my disorganized methods worked but not as well as this more organized and thought out tool appears to.
1) Experimental design is the backbone of scientific inquiry and I would eagerly watch more from you on this topic.
2) Now I think I need to dive into the math behind making these tables, because that's a personal interest of mine.
Fascinating and instructive. Those final bright light shots are stunning.
Wonderful! Thank you for this information. My kids are getting to the age where science projects are beginning a thing, and I always turn to your channel for inspiration. We thank you for your thorough, organized and entertaining videos.
I've been subscribed to your channel for about as long as you've made these sparklers, I didn't even speak English back then
Add this alongside fermi estimating to the list of skills I should learn. Can't wait to start practicing this!
I took statistics in high school, laboratory chemistry classes in college, and I never heard about using orthogonal experimental arrays to test multiple variables at the same time. So thanks for sharing!
And thanks to everyone in the comments for pointing me in the right direction if I ever want to learn more
What a wonderful, educational video! I’ve practiced medicine for almost 40 years and over that time, I’ve read a huge number of articles comparing different diagnostic and therapeutic methods. What I never came across is a single paper that used the Taguchi array to compare multiple variables in the test population. Actually, I never heard of the Taguchi array until watching this video. Thank you so much for all of your awesome videos!
Wow, thanks so much Ben! You were the reason that I started making these fireworks and I brute forced my way through hundreds (?) of hours of making them to get to this point, a fraction of the time you spent I'm sure. This method is absolutely a game changer. It's hard to describe that feeling of trying to solve a problem and being completely lost with no direction on where to go next.
So happy you've improved the recipe and I can't wait to try it, and to try to improve on it with this method!!
This is an incredibly interesting method that has applications across so many domains. I had no idea this existed, so THANKS!
Wow, that is amazing! I love looking at data and seeing significant information pop out of it. I’d been completely unaware of this method before; this is just *so* useful and important! Thanks!!!
This is great. I used to teach this at MIT using paper airplanes with multivariable analysis. Loved the video.
Finally, thanks! This is such valuable information! Please continue your excellent work with all the due diligence you continue to invest in your world class instruction.
This motivates the method well enough to make it seem like the necessity it is. Rather than the menial curiosity it feels like in most presentations. As per the MLA Handbook you don't need detractive statements in your scripts. Your work speaks for itself.
If you had made this video 6 months earlier, my master thesis would have been so much easier ! Your explanations are very clear and would have saved me a lot of headaches!
Thanks for figuring it out. Years past, someone came up with this concept but never disclosed the formula other than it was simple in home chemicals. He did his demonstration to the media even showing cooking an egg without damaging the interior. If he did patent the formula it should be available. Looking forward to experimenting with new tiles too produce tile better high efficiency wood stoves. Best
I remember how i was in my early teens watching Nighthawkinlight and the king of random diy projects after school, amazing how im now in my mid 20 and still enjoy your videos!
That is wicked efficient. It may be old news to some but I think it will really change the way I consider certain problems going forward. Thank you!
1:00 What's funny is back in HS Bio (which I wound up with the top score in the school that year), we had an experiment on dissolving gummy bears. The variables were something like: Salinity of the water, Level of sunlight, Volume of water, % Vinegar etc...
I misunderstood the instructions at first, thinking we needed to test them all, and so came up with a "3D Test Grid" wherein we could test 3 different variables at a time AND test their compounded effects. So, X = Salinity increases; Y = Sunlight levels/distance from light/heat source; and Z = Vinegar %. Or something like that. And drew out a cubic grid to allow for about 5 models for each arm of the experiment, for a total of about 125 mixed models. 0,0,0 being the control batch with no salinity, no light/heat, and no vinegar. Recording how much was dissolved every so many units of time.
The teacher pretty quickly shut that down though.
Nighthawk, I have done this and so many other experiments on the boiled egg. The best I have ever found hands down is to tap on the boiled egg with a spoon until it pops. you will actually hear the sound. It works every time.
Your presentation and delivery has really improved over the years. Thumbs up!!
Always a treat to watch one of your videos. Eloquently narrated, and always some idea or take on a subject that i didn't know or hadn't considered. And I especially admire that you shout out other intelligent creators
i loved this! thank you for articulation, this was exactly the right moment to stumble upon in my life with my biochemie experiments which leaning towards to be more complex gradually. hero!
When you repeat your orthogonal array experiments, you can also use simplex optimization techniques to choose how much to vary each input.
I really enjoyed this video!
I'm amused by the notion that I got excited about experimental methods and slow methodical work. My inner kid is saying it's boring compared to doing more active things.
But my older self has dealt with so many questions that I did not have the knowledge or tools to find answers to. So seeing someone find answers to questions they've been working towards is like breaking through a wall. And now my inner kid is back on board because breaking walls seems fun!
I'm so glad you put this out I've been doing this for years since my previous job
This really is terrific. I've been experimenting in one way or another for many decades. First time i've ever heard of Orthogonal/Tomaguchi Arrays.
A great way to explain the use of Taguchi arrays in process optimization! Bravo!
Bravo! I discovered MED from an essay by a chemist extolling it's virtues a decade ago and have looked for ways to use it ever since. Great video!
I'm impressed! Your explanation of this process is MANY times more concise and MUCH clearer than the dreary explanations I learned in school. THANK YOU!
Thank you! I’ve always been stuck in the changing one variable realm and was highly skeptical before watching the video. You have opened my eyes and I thank you for it!
Loved this as I had never heard of this technique. I have to say after the first few mins I thought it looked needlessly complicated and would lose value against the normal factorial approach. So was blown away by how it still captured the normal outputs but also gave even more information (but with less tests). Very well demonstrated and glad you showed such different examples. Thanks for sharing your learning as always.