As an engineer myself I appreciate your scientific method of measuring performance. When I first got into reloading I did the same 3-5 shot dogma groups that everybody does but I’m definitely seeing that trying to find an optimal system with that method is basically chasing your tail. Loading and shooting is a science just like any other, you just need the right methodology. When you said “the smaller the change the larger sample size you need to characterize it” that really clicked for me. Also, that funnel chart is an amazing and elegant way to visualize how important a statistically significant data set is. Great stuff!
Great video. The "OnTarget TDS" program will give you X,Y coordinates for each shot and you can export them. You just have to make each shot its own group. I use this data to build excel sheets which give me complete details based on the data as discussed here. One of my benchrest for score guns gave me an average POI distance from center of 0.168"(0.160MOA) with an SD of 0.090"(0.087MOA) at 100 yards over 90 shots(3 matches). The 95% CEP(the size of the circle that if drawn would encompass 95% of the shots) was 0.290" Thing is a laser beam.
Wow! So many takeaways from this video, thank you. I think maybe my head hurts a bit, not only taking it all in but how does one apply it in a way that takes you where you want to go when developing a load. Love to see a follow up video titled; Bringing it all home. Thanks again to the both of you for taking the time to share your knowledge.
See... it is rocket science! What an incredible discussion. That is the great thing about video media, the cumulative knowledge of hundreds or thousands of people can be disseminated to multitudes more who will in turn build upon it and thus increase the knowledge base exponentially. The science of ballistics is rapidly advancing because of exactly this kind of video. Thanks for sharing.
One of the things that was mentioned in passing but is very important is that if you set an upper acceptable boundary on performance you only need to take as many shots as it takes to break that boundary before rejecting the combo that created it. For example if you need your match barrel to shoot 0.500" or better and you fire two shots that are 0.750" apart(assuming shooter error wasn't the cause) you've broken your upper boundary and can then reject that load without further testing. This obviously can save a lot of time and components.
If you're working in mean radius instead of group size, the metric can get better with more shots. This is where the topic starts to turn more into how do you compare two different populations, ie when you change a variable and gather data on the new set, how do you assess if it's significantly different or not. I use the t test to do this, which takes into account the number of samples and the variance in the samples. Working from your example, say you are comparing two loads with mean radii 0.25 and 0.375, a 50% difference. The number of samples you need to tell these apart will depend on how large the variance within each set of samples is. If the standard deviation in radius is large, say 25%, you might have trouble saying confidently that one is really worse than the other without more samples because the tails of those populations overlap quite a bit. If the standard deviation is small, say 10%, you might have better luck with fewer samples because the populations are more isolated.
@@solarshooter Yeah, When I want to compare sets of groups with different fixed variables I use a two tailed T-TEST that I setup in google sheets. I have mine set at 90% confidence level. However, I was just giving an example on setting upper boundaries. You can't use distance from POI when you're testing powder/seating depth because they are all going to naturally land different distances from your POA(mean radius obviously still works). Distance from POI is great once you want to verify an established load where the variables aren't changing. Or your skill level in varying conditions(you'd expect POI distance to increase with the severity of wind/mirage etc).
@@MMBRM If you first calculate the relative center of each group, you can use mean radius even as your group poi shifts. Plus this gives you a precise measurement of group to group poi shift to assess the harmonic trend.
Just reloaded my first set of rounds last night. Found this video and I'm fairly new to the firearms community. This information is alot to take in, but very enlightening. Thank you.
I was excited to see that you were in my neighborhood in the intro. Even more excited to see that you're interviewing a guy from my range. We've talked about reloading often. I saw Michael's Dads rifle a couple weeks ago at the range. Cool interview.
This video is worth its weight in gold. I watched the video three times. I dissected the video every possible way and its like a light bulb turned on. Took me a while to fully understand everything that was covered in the video. Thank you. 👍
Think I'll have to watch this several times to take it in, I've been thinking along the same lines for some time now, but struggled to think of a way to put it into practice, so thanks to you both for doing this. An example of how you took the 195 rounds shot and created a mass of data points from these to be analyzed would be great. Also thanks for the tips below on On Target TDS, I've been using an App on my phone, but it gives limited data.
Once you have x,y coordinates of each shot, you can calculate the radius from center for each point. Then you can compute mean and standard deviation of that set of data. This is probably as far as you need to go in order to characterize your rifle. Just keep adding to the list of shots that you log and add to the calculation of mean and sd radius. And keep in mind that as you get more and more shots those numbers will become more and more confident. If you wanted to go farther, to generate the same sort of "funnel chart" presented in this video, you would need to take that list of shot radii, calculate the running average, randomize the list, repeat, and so on until you build the chart that shows your "funnel of error". Then you could determine for your particular rifle/ammo how much error/confidence your mean and sd radius has.
This video was eye opening & the clincher. I've joined your patreon channel. I love geeking out on data. I'll be working up some loads and going to the range. Keep making great content!👍
I concur. I have taken data on every shot for years when testing, and compare loads statistically and have found all the same results. Now I did not present them as well, and I commend you. I would suggest one thing, instead of just measuring radius, I measure X and Y and then calculate radius to the adjusted center. Why? it lets me compare on the basis of radius, horizintal and vertical, and it is not big deal to calculate with a spreadsheet. I have found I can tell when the wind is bringing excessive noise to the test by comparing results of vertical and horizontal. t any rate, well done, I think you are absolutely on the right track, and are theoretically sound in your approach. Besides that your video made me feel a lot better about what I have been doing all along, that somone with a similar background in engineering and statistics has arrived at the same place.
Thank you for presenting this. I confess that by the 40 minute mark my brain hurts. I am not a competitive shooter. However, I do want an accurate system (me/rifle/optic/ammunition). The only conclusion I can draw here is that I can identify BAD loads, but zeroing in on the ideal load is far beyond what I am willing or even capable of achieving.
8:00 I applaud your dedication in measuring and calculating by hand, but just an FYI there is an app for that. The one I use is called Range Buddy, but there are several others that are perfectly identical. They're also free. Just shoot the group, take a picture of it, mark a reference size in the picture, mark the holes, and it spits out all kinds of data, including Mean Radius(MR) AKA Figure of Merit(for you 19th century types) along with MOA, X and Y offset of the center of the group to point of aim(makes sight adjustment or testing a sight super easy and fast), and a few other data points. The one I use allows you to export the info, not sure if you're looking for a specific format though. It's great for analyzing rifles, pistols, buckshot, and slugs. Anyway, I appreciate your contributions to the reloading community, just thought I'd share something that might make analysis faster and easier🙂
@@solarshooter lol well it doesn't do that. Not sure if you could mess with some CAD/CAM software to make that part easier or not. CAD/CAM is the only thing I know of that spits out X and Y. Either way, keep up the good work and keep sharing with the community.
@@solarshooterHey, I just found a target analysis app that does X and Y of each individual shot. Hornady's 4DOF app. The target analysis part is $5 one time fee. I'd show you if I had a way to share screen shots, but I thought you'd be excited to have an app to do the X and Y of each individual shot for you! Keep up the good work Sir.
What a great interview! and Eye opener! Thank you guys! If Michael can do that with a hunting rifle; Imagine him with a heavy barrel, a 25, 30 pound rifle shooting a match? Thanks again 👍
Wow, wow wow. This is just what I have been looking for in doing load development. As an engineer as well and a new handloader for hunting rifles, I have been looking for the details that describe those variables that have the major effect on precision. WITW I have been following you for awhile and it was great you see dispel the "one shot ladder" approach to finding a flat spot (only after I tried it multiple times!). Velocity is proportional to powder charge! After watching Erik Cortina interview Chris Long, OBT became my goal; unfortunately my reloading skills and lack of statistical significant sampling has gone only so far. With QuickLoad I have been able to find out how much more powder charge has in determining barrel time than seating depth and therefore with the info presented here I can now move "more confidently" to put this information to use. I do have a question though, have you or anyone else ever used Design Of Experiments (DOE) to find the optimal charge/seating depth and maybe throw in primers?
Using the term 'DoE' around reloading will get your mouth washed out with soap! The "scientific method" begins and ends with the classic one-factor-at-a-time (aka OFAT) everyone learned in grade-school science class. Anything else is heresy 🤣 Just in case it didn't come through, yes, I am joking. Kind of. I would love it if *anyone* in the shooting community would even dabble in DoE... but it's hard enough to get people to use mean radius rather than extreme spread 🤷♂
The trouble With all these theories. Is your barrel has only so many competitive shots, So if you do such extensive testing. As a competitive shooter, you would probably come way. Short utilizing your barrel for competition. All the top competitive shooters in f class. Pretty much do the same thing all the time not much changes. It's great information.
I’m not sure how it might apply to centre fire stuff, but in the rimfire world a Harrell tuner is common. I think they weigh in the 9 oz. neighbourhood. They have clicks that move it in and out by 1 thou. And 1 or 2 clicks is enough to see a meaningful difference on paper with those. How heavy a tuner needs to be and how fine the adjustment needs to be depends on many things, including barrel length and contour, distance you’re shooting at, etc.
I can really relate to Michael's situation. I don't compete at 1,000 yards but I do want to build a load for my .270 Winchester that can ethically take a whitetail or other game out to 300 yards without having to dial in elevation or hold over. My current goal and load development is to build a load velocity and zero distance combination that will give me a consistent rise of no more than 3" at 150 yards and no more than a 3" drop at 300 yards. Basically I want my rifle to shoot anywhere inside 300 yards and never be more than 3" above or below my point of aim. If I can tune my load and rifle to those specifications with 1 MOA or less groups throughout the range then all that is left is any extreme windage which is on me to resolve as needed in the field.
OnTarget software will measure things for you. You just scan the target. I use the #7 target, which is a practice ARA target, which has 25 bulls. One shot at each bull and it locates all the holes for you, usually, and gives you a 25-shot group with some numbers to go along with it. It also has a lot of other targets, and you can use your own. It is only fully automated with its own targets, though.
I have tried the "one shot at one bull" before, and found greater dispersion in mean radius than taking 5 shots per bull. My theory on why is that I'm constantly shifting my rest to hit all the different aim points, and that it's harder to pace myself and manage the heat since I don't have the obvious 5 shot breaks. Both could be mitigated easily however so I see the merit here.
Very good info, well done! Using mean radius is very practical as you can use the data from 4 shots in a ragged hole and the one "flyer" that opens the group to over an inch. One idea came to mind was can this approach be used to demonstrate the effects of barrel fouling and accuracy? It would be nice to know how many shots with out cleaning makes an actual difference.
interesting point about shooting and hiding the results. make a target that has a point of aim, and a point of impact that is, say, 5MIL below the POA. dial your scope 5MIL and shoot at the POA without seeing the POI.
It's less about shooting out the aim point on the target, and more about losing the actual x,y coordinates of the shots inside the big hole. Yes the total group size will be a more accurate number, but you won't be able to characterize the distribution of shots inside that group, which is really what tells you about the precision of the system.
Im going through load development on a 300 blackout at the moment, and have been using a ladder test to get through the charge weights quickly, with 1 shot per .2 grains. I am looking for 2 subsequent charges that do not change vertical impact, and or, velocity very much, then i run the string one more time to take out some human error, and to see how each charge reacts to a more fouled rifle. Im not sure how statistically significant these tests are, but i am going back and verifying areas that look promising, loading inbetween the .2 grain increments to give me some leeway with small scale errors.
2 shots is a very small sample to try and determine true average point of impact, especially when you are making such small changes in powder. You are trying to assess a small delta with a small number of samples, which will likely not result in a meaningful answer. It might be better to do slightly more shots per charge, say 3-5, and use larger steps in charge. Your total number of shots won't change, but you'll be able to characterize the same harmonic pattern with higher confidence. Also make sure to accurately calculate average POI for each charge. After you get this initial sweep, you can focus on a particular range with finer increments just as you are currently doing, but you should also increase the number of shots per charge, say 10, when you do this.
I learn some" from u'r videos. Enjoy a lot. After a while i get this creepy feeling that somhow( a little) shadows of these video's verify things i wasn't sure about did i really get that? 🐎
Yes, for two reasons. First, it demonstrates the concepts at the available ranges (100 and less). Second, 22lr is used for various competitions. Accuracy and precision might not what we expect from a CF rifle, but it is still important to those that compete with it.
22lr is relatively economical and time efficient to shoot hundreds of rounds with, plus with little to no recoil it doesn't fatigue the shooter nearly as much. It would be interesting to repeat the test with a centerfire rifle and see if the trends or conclusions are different.
Not a statistician, just and engineer and those stats classes were a long time ago for me. You mentioned, that the plot of group size did not appear normally distributed. I'm not certain that I would expect a bar chart of group size to be normally distributed. Seems to me more likely to have a positive skew, since group size is bounded on one side and can never be less than zero. I will need to revisit the info from Brian and Hornady on this topic, but from what I recall they both presented that the shot data from a large enough group size (50 per Hornady) gave a normal distribution and pretty much stayed there beyond 50 shots. I'm guessing that it must be the mean radius (from a large enough group) that will have a normal distribution. Which raises another question, does it makes sense to take horizonal and vertical spread and combine them into mean radius? I've often wondered if it wouldn't be better to look at X and Y independently. Meaning if you took your data from the 39 groups and plotted raw X and Y values as calculated from the overall group centers, and then looked at the X and Y data independently would both be normally distributed and would they have a similar SD? Seems to me that by using distance we could be missing some important information in our group analysis.
The plot at 6:16 shows my results for the distribution of group size, and yes I was also surprised by this result. I think Litz presented real group data, but I think the Hornady folks generated their data via simulation (might be wrong about that). This looks more like a "log normal" distribution to me, but again I'm out of my depth here. I doubt I would start to see a normal at 50 groups that I couldn't see at 38, especially since they would all have to be extremely small to "fill in" the missing part of a normal distribution. If you look at the scatter plot of all 190 shots (9:31), it does look like it's centered about 0 with a "normal-ish" distribution but I did not try to assess that here. Perhaps this is yet another emergent trait of sampling a larger true group with small 5 shot sets - it skews the answer from reality. I didn't get a chance to show the mean radius distribution plot but it has a Rayleigh shape to it (MLE = 0.159), which would be expected when combining a 2d problem (x,y coordinate) where each variable is normally distributed and uncorrelated into a 1d metric (radius). This indicates that what I said above is probably true, and that vertical vs horizontal dispersion may be uncorrelated. If I had tested at different range, this might change, since different types of dispersion affect each axis differently.
@@solarshooter I’m pretty sure that hornady said that the data for a group, not groups, is normally distributed and that 50 shots was enough to clearly establish the normal distribution. What they did not explain is how are you putting 2D data into a normal distribution! I asked after watching their small sample size podcast, and they said that the data being presented was from actual groups fired in their test rig.
Guys.. You need a MACHINE REST. This will eliminate the greatest variable. The person. THEN you know its the load of the rifle. This has been know for 150 years or more. I shoot service rifle competition. The channel has helped me with my 600 yard loads I enjoy all of it so don’t take this the wrong way. But you need a machine rest and a GOOD one.
What’s the point of knowing what your gun does on a machine rest? That’s not how you’re going to use it…. Therefore that’s not going to perform the same when it counts. I need it to shoot well on MY SHOULDER. When the shots count. You’re introducing another variable is all I’m saying
Very interesting info, thanks. So what do you take the mean radius from? Each group will have a different center, so how do you find the center which is common to 5 or 10 different groups? Are you using an app for this?
I am a believer in barrel exit time. My question to the both of you is Chromoly or Stainless steel. Is there a factor that could be calculated to adjust for using a stainless barrel?
With all the vids out there no information on your reloading practice. Did you use a pressure ind. on the bullet seating? Is all the necks turned to a consistence size. Primers seated the same. What crush is on them. And last but not least, Did you filter your powder to use all the same kernel size?
Thanks so much for the first 13 minutes. I will comment as I go. First off do you know your scope is holding well. See Alex Wheeler on a scope holding center. Next, are you holding the rifle the same? You mentioned this already. Only a rail gun will react the same every time.
The stiffening of the barrel really makes me think closely about carbon fiber. I should say "correctly done" carbon fiber with all the different manufacturers these days. This also makes me think on the inherent problems of digital scales. The VC (virtual center) seems to be what is most important here. Making me think of GD&T. 😂
Where does heat make a change in the barrel to change? Must mean if the barrel moves less the group should be tighter. May mean one chases load development over tuning the rifle.
It lowers the stiffness of the barrel material, which will change the bending mode frequencies. It also causes expansion of the material, which could cause warping and/or expansion/contraction of the bore.
@@charlesking8542 Part of the issue with forums, is a lot of good info is hidden in a thread with a title or preamble that wouldn't lead a person to look at it for given reason. Or it is so far down in the thread that people have quit reading before they get near it. Found a few like that over the years, such as one on the Campfire that in the back part of a big thread, had people like Ken Oehler participating on internal ballistics, and a few others who worked in test labs. Or others that someone had put up a link to something by someone who really could speak on the subject, or a study on it.
What are your thoughts on crimping a 7mm bullet in a bolt gun with a cannelure ? My 139 gr btsp which is what I have to load has a cannelure and I see mixed answers to this!
There’s an issue with taking data shot at different distances and treating them as if they’re part of the same data pool. It has to do with harmonics and what you refer to as bullet time. What you’re really talking about is positive compensation. Changing the angle of departure based off bullet velocity in order to make shots land as closely as possible to the same point of impact. And the issue is that this concept of positive compensation involves a different amount of compensation at a different shooting distance. You do not get the same magnitude of dispersion at two different distances. It is physically impossible to do so. Ignore wind. If you shoot out to 1000 yards but have electronic targets at your intended 1000-yard distance and also at 100 yards you will not see the same group sizes at both targets. Will not. It is not possible to see 1 MOA at 100 yards and also see 1 MOA at 1000 yards from the same shots. The group sizes at both distances will differ. They must. The only way they will not differ is if all shots have precisely the same velocity. That never happens. If you play with “bullet time” to get your groups at 100 yards to be as small as you possibly can those same shots will print a larger group at 1000 yards, and vice versa. If you play with “bullet time” to get the smallest possible groups at 1000 yards the same shots will print larger groups at 100 yards. “Optimal bullet time” is distance specific. Another way to say this is you need to adjust the angle of departure for a load with a 20 fps spread by a different amount to hit the same point of impact at different distances. The angle of departure required for a shot at 2800 fps to hit zero at 100 yards will differ from the angle of departure required for a 2820 fps shot by x amount. Change the distance to target to 1000 yards and now the angle delta between the 2800 fps shot and the 2820 fps shot is no longer x amount. It is going to be much larger than x amount. And this is why if you’re getting the best possible groups that you can at 100 yards you are hurting your groups at 1000. Or 600. Or 300. Or whatever. The angle of departure delta requires changes for all distances. If you’re shouting the same load at 600 and 1000 you’re not getting the best results you could at both individually. And the same goes for using the same tuner setting at 600 and 1000. Whatever method you’re using to try to approach the optimal “bullet time” needs to be tailored to specific distances, no matter if it is a tuner, or charge weight, or bullet seating depth, all that stuff. It is distance specific. Simple projectile flight behaviour dictates this.
Positive compensation is an interesting topic, and one Bryan Litz also explored in his book. He did find some kernel of truth to it, though it was not conclusive. I look forward to seeing further study on the topic. The way to test it is to set up multiple targets on the same flightpath, so you can log the dispersion of the same shots at multiple ranges. I would expect that wind, mv, and bc dispersions, which are a nonlinear effect with range, will cause your group to grow with range. This is what I have observed in my tests at different ranges.
@@solarshooter It's what barrel harmonics are. You're making shots leave at different angles according to their speed, even if you don't know it. Play with a ballistics calculator. Punch in a 1000-yard zero and a 2800 fps shot vs a 2820 fps shot. Look at the drop at 100 yards for each one. When using a 1000-yard zero the difference in drop at 1000 yards will naturally be zero, as that's your intended target distance. But since the two bullets have different speedds they will be at different heights at every other distance. They must be in order to hit the same spot at 1000 yards. The difference in drop at 100 yards for a 1000-yard zero with a delta of just 20 fps might be a little surprising. Then do the same for a 100-yard zero and compare the drop at 1000 yards. The figures you see for drop at the other distance is how much vertical you would see on target at that distance when using a zero at the other distance. This is what I'm talking about when I say you will not see the same results at a distance ABC when you are testing/tuning at distance XYZ. When you do that you are trying to optimize the compensated departure angles to reduce group size, but when you go to shoot for score at the other distance you've unintentionally ensured you'll have larger groups at that for-score distance. You'll always get better results testing/tuning at your for-score distance. For some reason quite a large group of people are skeptical of positive compensation, when I can see no other reason to explain shot behaviour. Positive compensation is barrel harmonics, it's just that you're trying to coax a specific desirable amount of harmonic to help you get better results. You can calculate how quickly you need the barrel to be swinging upwards in order to provide the necessary amount of compensation in the angle of departure for a given distance. You can then calculate whether or not your barrel can be swinging upwards near the speed required in order to approach that amount of compensation with regard to a given load. Then if you have a barrel contour that swings upwards a little too fast you can add mass (a barrel tuner) to it in order to slow it down to it lives in the sweet spot and provides just enough compensation to shrink your groups at your desired target distance. I had a rather unusual barrel built for a 22 rimfire benchrest gun, following Varmint Al's FEA design. Go to Varmint Al's website and to his page titled "22 LR rifle & tuner" and go about halfway down the page to the section titled "Two-Flats Barrel Contour with Vertical Flexibility and Greater Horizontal Stiffness" and you'll see what he came up with in the FEA software for a 50-yard barrel. He never did go to the trouble of actually building one. So I did. Hehe. Yes, it looks crazy. Yes, it works. It is just slightly too fast in its upwards swing for 50 yards. And only needed a very, very light tuner to bring it into the sweet spot. So far I've determined that it needs to be in the 31-32 gram region, but I'm still playing to find out exactly how much. Small changes can make quite a difference.
I've tried load development.. duty the whole seating depth thing I've found to be the biggest load of crap... beautiful groups won't repeat and I have no idea why.. it's seem to be a load of marlarky
This was a practical demonstration of the theoretical concepts presented in that video. It does agree with some of their conclusions, just as their video agreed with Bryan Litz's conclusions. However the takeaways regarding focusing on shot radius, the efficacy of testing with small groups, and the importance of long term characterization with data vs. constant tuning are valuable.
Seriously Michael, thank you very much for your contributions to the reloading community. This is exactly the kind of stuff we need in the community.
Good job!
This is essentially what Hornady concluded in The Hornady Podcast, episode 50.
You need 30-shot groups to be statistically significant.
Excellent video! I’ve been reloading for 50 years and learning every time I hop on your channel.
As an engineer myself I appreciate your scientific method of measuring performance. When I first got into reloading I did the same 3-5 shot dogma groups that everybody does but I’m definitely seeing that trying to find an optimal system with that method is basically chasing your tail. Loading and shooting is a science just like any other, you just need the right methodology. When you said “the smaller the change the larger sample size you need to characterize it” that really clicked for me. Also, that funnel chart is an amazing and elegant way to visualize how important a statistically significant data set is. Great stuff!
Great video. The "OnTarget TDS" program will give you X,Y coordinates for each shot and you can export them. You just have to make each shot its own group. I use this data to build excel sheets which give me complete details based on the data as discussed here. One of my benchrest for score guns gave me an average POI distance from center of 0.168"(0.160MOA) with an SD of 0.090"(0.087MOA) at 100 yards over 90 shots(3 matches). The 95% CEP(the size of the circle that if drawn would encompass 95% of the shots) was 0.290" Thing is a laser beam.
@ShootingInTheWind another vote for “OnTarget TDS” for an excellent program that gives you precise (X,Y) measurements.
Great tip. This is exactly the type of characterization described in the video. Thanks!
Wow! So many takeaways from this video, thank you. I think maybe my head hurts a bit, not only taking it all in but how does one apply it in a way that takes you where you want to go when developing a load. Love to see a follow up video titled; Bringing it all home. Thanks again to the both of you for taking the time to share your knowledge.
It makes load development much easier to understand thanks for the education!
On one hand alot of info on the other hand the best info ever shared. My hat is off to both of you and to your channel. Well done. 👍👍👍
Like to see Michael in an interview with Erik Cortina...
See... it is rocket science! What an incredible discussion. That is the great thing about video media, the cumulative knowledge of hundreds or thousands of people can be disseminated to multitudes more who will in turn build upon it and thus increase the knowledge base exponentially. The science of ballistics is rapidly advancing because of exactly this kind of video. Thanks for sharing.
As an Aerospace Engineer, I really enjoyed this discussion. Thank you both.
One of the things that was mentioned in passing but is very important is that if you set an upper acceptable boundary on performance you only need to take as many shots as it takes to break that boundary before rejecting the combo that created it. For example if you need your match barrel to shoot 0.500" or better and you fire two shots that are 0.750" apart(assuming shooter error wasn't the cause) you've broken your upper boundary and can then reject that load without further testing. This obviously can save a lot of time and components.
If you're working in mean radius instead of group size, the metric can get better with more shots. This is where the topic starts to turn more into how do you compare two different populations, ie when you change a variable and gather data on the new set, how do you assess if it's significantly different or not. I use the t test to do this, which takes into account the number of samples and the variance in the samples.
Working from your example, say you are comparing two loads with mean radii 0.25 and 0.375, a 50% difference. The number of samples you need to tell these apart will depend on how large the variance within each set of samples is. If the standard deviation in radius is large, say 25%, you might have trouble saying confidently that one is really worse than the other without more samples because the tails of those populations overlap quite a bit. If the standard deviation is small, say 10%, you might have better luck with fewer samples because the populations are more isolated.
@@solarshooter Yeah, When I want to compare sets of groups with different fixed variables I use a two tailed T-TEST that I setup in google sheets. I have mine set at 90% confidence level. However, I was just giving an example on setting upper boundaries. You can't use distance from POI when you're testing powder/seating depth because they are all going to naturally land different distances from your POA(mean radius obviously still works). Distance from POI is great once you want to verify an established load where the variables aren't changing. Or your skill level in varying conditions(you'd expect POI distance to increase with the severity of wind/mirage etc).
@@MMBRM If you first calculate the relative center of each group, you can use mean radius even as your group poi shifts. Plus this gives you a precise measurement of group to group poi shift to assess the harmonic trend.
Wow, super interview. What a great amount of info.
Excellent!! Well articulated. Michael you have excellent communication skills and presentation.
Love this tech stuff,even though it's over my head ,I listen and hopefully improve my hand loading technique. Thanks again,always watching!!!😊
What an informative and meaningful discussion. Hope to hear more from Keith and Michael going forward.
Just reloaded my first set of rounds last night. Found this video and I'm fairly new to the firearms community. This information is alot to take in, but very enlightening. Thank you.
I was excited to see that you were in my neighborhood in the intro.
Even more excited to see that you're interviewing a guy from my range. We've talked about reloading often. I saw Michael's Dads rifle a couple weeks ago at the range.
Cool interview.
Great job explaining a complex subject.
Another very aducational vid explained in mortal terms, love your channel, 👍
This video is worth its weight in gold. I watched the video three times. I dissected the video every possible way and its like a light bulb turned on. Took me a while to fully understand everything that was covered in the video. Thank you. 👍
Very interesting, thank you both for sharing.
Spectacular. Thank you both.
Thank you for the discussion. A huge amount if information for any / every level of understanding.
This data and information is amazing and feeds my engineering mind and reasoning. I love this discussion thank you so much for sharing.
This is great. I love all the numbers and the statistics.
I have changed my load development process based on this kind of information. 👍
Very good video
Excellent content. I will be reviewing this again in order to make sure I get as much learning as possible from it!
Think I'll have to watch this several times to take it in, I've been thinking along the same lines for some time now, but struggled to think of a way to put it into practice, so thanks to you both for doing this.
An example of how you took the 195 rounds shot and created a mass of data points from these to be analyzed would be great.
Also thanks for the tips below on On Target TDS, I've been using an App on my phone, but it gives limited data.
Once you have x,y coordinates of each shot, you can calculate the radius from center for each point. Then you can compute mean and standard deviation of that set of data. This is probably as far as you need to go in order to characterize your rifle. Just keep adding to the list of shots that you log and add to the calculation of mean and sd radius. And keep in mind that as you get more and more shots those numbers will become more and more confident.
If you wanted to go farther, to generate the same sort of "funnel chart" presented in this video, you would need to take that list of shot radii, calculate the running average, randomize the list, repeat, and so on until you build the chart that shows your "funnel of error". Then you could determine for your particular rifle/ammo how much error/confidence your mean and sd radius has.
Gentlemen THANK YOU for the pearls of knowledge! 🍺🍺
This video was eye opening & the clincher. I've joined your patreon channel. I love geeking out on data. I'll be working up some loads and going to the range. Keep making great content!👍
😂 yeah Keith you have a problem! Really enjoyed the interview, thank you both.
I concur. I have taken data on every shot for years when testing, and compare loads statistically and have found all the same results. Now I did not present them as well, and I commend you. I would suggest one thing, instead of just measuring radius, I measure X and Y and then calculate radius to the adjusted center. Why? it lets me compare on the basis of radius, horizintal and vertical, and it is not big deal to calculate with a spreadsheet. I have found I can tell when the wind is bringing excessive noise to the test by comparing results of vertical and horizontal. t any rate, well done, I think you are absolutely on the right track, and are theoretically sound in your approach. Besides that your video made me feel a lot better about what I have been doing all along, that somone with a similar background in engineering and statistics has arrived at the same place.
Thank you for presenting this. I confess that by the 40 minute mark my brain hurts. I am not a competitive shooter. However, I do want an accurate system (me/rifle/optic/ammunition). The only conclusion I can draw here is that I can identify BAD loads, but zeroing in on the ideal load is far beyond what I am willing or even capable of achieving.
I enjoyed the educated conversation.
8:00 I applaud your dedication in measuring and calculating by hand, but just an FYI there is an app for that. The one I use is called Range Buddy, but there are several others that are perfectly identical. They're also free. Just shoot the group, take a picture of it, mark a reference size in the picture, mark the holes, and it spits out all kinds of data, including Mean Radius(MR) AKA Figure of Merit(for you 19th century types) along with MOA, X and Y offset of the center of the group to point of aim(makes sight adjustment or testing a sight super easy and fast), and a few other data points. The one I use allows you to export the info, not sure if you're looking for a specific format though. It's great for analyzing rifles, pistols, buckshot, and slugs. Anyway, I appreciate your contributions to the reloading community, just thought I'd share something that might make analysis faster and easier🙂
X,Y coordinates for each shot is all that's needed, everything else is handled downstream. Thanks for the tip!
@@solarshooter lol well it doesn't do that. Not sure if you could mess with some CAD/CAM software to make that part easier or not. CAD/CAM is the only thing I know of that spits out X and Y. Either way, keep up the good work and keep sharing with the community.
@@solarshooterHey, I just found a target analysis app that does X and Y of each individual shot. Hornady's 4DOF app. The target analysis part is $5 one time fee. I'd show you if I had a way to share screen shots, but I thought you'd be excited to have an app to do the X and Y of each individual shot for you! Keep up the good work Sir.
What a great interview! and Eye opener! Thank you guys! If Michael can do that with a hunting rifle; Imagine him with a heavy barrel, a 25, 30 pound rifle shooting a match? Thanks again 👍
Wow, wow wow. This is just what I have been looking for in doing load development. As an engineer as well and a new handloader for hunting rifles, I have been looking for the details that describe those variables that have the major effect on precision. WITW I have been following you for awhile and it was great you see dispel the "one shot ladder" approach to finding a flat spot (only after I tried it multiple times!). Velocity is proportional to powder charge! After watching Erik Cortina interview Chris Long, OBT became my goal; unfortunately my reloading skills and lack of statistical significant sampling has gone only so far. With QuickLoad I have been able to find out how much more powder charge has in determining barrel time than seating depth and therefore with the info presented here I can now move "more confidently" to put this information to use.
I do have a question though, have you or anyone else ever used Design Of Experiments (DOE) to find the optimal charge/seating depth and maybe throw in primers?
Using the term 'DoE' around reloading will get your mouth washed out with soap! The "scientific method" begins and ends with the classic one-factor-at-a-time (aka OFAT) everyone learned in grade-school science class. Anything else is heresy 🤣
Just in case it didn't come through, yes, I am joking. Kind of. I would love it if *anyone* in the shooting community would even dabble in DoE... but it's hard enough to get people to use mean radius rather than extreme spread 🤷♂
The trouble With all these theories. Is your barrel has only so many competitive shots, So if you do such extensive testing. As a competitive shooter, you would probably come way. Short utilizing your barrel for competition. All the top competitive shooters in f class. Pretty much do the same thing all the time not much changes. It's great information.
I feel like I am watching 60mins, but the interview is actually good.
I’m not sure how it might apply to centre fire stuff, but in the rimfire world a Harrell tuner is common. I think they weigh in the 9 oz. neighbourhood. They have clicks that move it in and out by 1 thou. And 1 or 2 clicks is enough to see a meaningful difference on paper with those. How heavy a tuner needs to be and how fine the adjustment needs to be depends on many things, including barrel length and contour, distance you’re shooting at, etc.
Thanks!
Thank you!
I can really relate to Michael's situation. I don't compete at 1,000 yards but I do want to build a load for my .270 Winchester that can ethically take a whitetail or other game out to 300 yards without having to dial in elevation or hold over. My current goal and load development is to build a load velocity and zero distance combination that will give me a consistent rise of no more than 3" at 150 yards and no more than a 3" drop at 300 yards. Basically I want my rifle to shoot anywhere inside 300 yards and never be more than 3" above or below my point of aim. If I can tune my load and rifle to those specifications with 1 MOA or less groups throughout the range then all that is left is any extreme windage which is on me to resolve as needed in the field.
OnTarget software will measure things for you. You just scan the target. I use the #7 target, which is a practice ARA target, which has 25 bulls. One shot at each bull and it locates all the holes for you, usually, and gives you a 25-shot group with some numbers to go along with it. It also has a lot of other targets, and you can use your own. It is only fully automated with its own targets, though.
I have tried the "one shot at one bull" before, and found greater dispersion in mean radius than taking 5 shots per bull. My theory on why is that I'm constantly shifting my rest to hit all the different aim points, and that it's harder to pace myself and manage the heat since I don't have the obvious 5 shot breaks. Both could be mitigated easily however so I see the merit here.
@@solarshooter A rest that has windage and elevation adjustments is golden.
Dejavu from the Hornady dispersion podcast
Very good info, well done! Using mean radius is very practical as you can use the data from 4 shots in a ragged hole and the one "flyer" that opens the group to over an inch.
One idea came to mind was can this approach be used to demonstrate the effects of barrel fouling and accuracy? It would be nice to know how many shots with out cleaning makes an actual difference.
interesting point about shooting and hiding the results. make a target that has a point of aim, and a point of impact that is, say, 5MIL below the POA. dial your scope 5MIL and shoot at the POA without seeing the POI.
It's less about shooting out the aim point on the target, and more about losing the actual x,y coordinates of the shots inside the big hole. Yes the total group size will be a more accurate number, but you won't be able to characterize the distribution of shots inside that group, which is really what tells you about the precision of the system.
Im going through load development on a 300 blackout at the moment, and have been using a ladder test to get through the charge weights quickly, with 1 shot per .2 grains. I am looking for 2 subsequent charges that do not change vertical impact, and or, velocity very much, then i run the string one more time to take out some human error, and to see how each charge reacts to a more fouled rifle.
Im not sure how statistically significant these tests are, but i am going back and verifying areas that look promising, loading inbetween the .2 grain increments to give me some leeway with small scale errors.
2 shots is a very small sample to try and determine true average point of impact, especially when you are making such small changes in powder. You are trying to assess a small delta with a small number of samples, which will likely not result in a meaningful answer. It might be better to do slightly more shots per charge, say 3-5, and use larger steps in charge. Your total number of shots won't change, but you'll be able to characterize the same harmonic pattern with higher confidence. Also make sure to accurately calculate average POI for each charge. After you get this initial sweep, you can focus on a particular range with finer increments just as you are currently doing, but you should also increase the number of shots per charge, say 10, when you do this.
I learn some" from u'r videos. Enjoy a lot. After a while i get this creepy feeling that somhow( a little) shadows of these video's verify things i wasn't sure about did i really get that? 🐎
Is there any value in testing notoriously inconsistent ammo like 22lr? Even the best 22lr ammo is sketchy.
Yes, for two reasons. First, it demonstrates the concepts at the available ranges (100 and less). Second, 22lr is used for various competitions. Accuracy and precision might not what we expect from a CF rifle, but it is still important to those that compete with it.
22lr is relatively economical and time efficient to shoot hundreds of rounds with, plus with little to no recoil it doesn't fatigue the shooter nearly as much. It would be interesting to repeat the test with a centerfire rifle and see if the trends or conclusions are different.
Good Stuff
1:09:00 we're gonna need a video on the rubber band trick lol
Is there an app that helps with this?...
Not a statistician, just and engineer and those stats classes were a long time ago for me. You mentioned, that the plot of group size did not appear normally distributed. I'm not certain that I would expect a bar chart of group size to be normally distributed. Seems to me more likely to have a positive skew, since group size is bounded on one side and can never be less than zero. I will need to revisit the info from Brian and Hornady on this topic, but from what I recall they both presented that the shot data from a large enough group size (50 per Hornady) gave a normal distribution and pretty much stayed there beyond 50 shots.
I'm guessing that it must be the mean radius (from a large enough group) that will have a normal distribution. Which raises another question, does it makes sense to take horizonal and vertical spread and combine them into mean radius? I've often wondered if it wouldn't be better to look at X and Y independently. Meaning if you took your data from the 39 groups and plotted raw X and Y values as calculated from the overall group centers, and then looked at the X and Y data independently would both be normally distributed and would they have a similar SD? Seems to me that by using distance we could be missing some important information in our group analysis.
The plot at 6:16 shows my results for the distribution of group size, and yes I was also surprised by this result. I think Litz presented real group data, but I think the Hornady folks generated their data via simulation (might be wrong about that). This looks more like a "log normal" distribution to me, but again I'm out of my depth here. I doubt I would start to see a normal at 50 groups that I couldn't see at 38, especially since they would all have to be extremely small to "fill in" the missing part of a normal distribution. If you look at the scatter plot of all 190 shots (9:31), it does look like it's centered about 0 with a "normal-ish" distribution but I did not try to assess that here. Perhaps this is yet another emergent trait of sampling a larger true group with small 5 shot sets - it skews the answer from reality.
I didn't get a chance to show the mean radius distribution plot but it has a Rayleigh shape to it (MLE = 0.159), which would be expected when combining a 2d problem (x,y coordinate) where each variable is normally distributed and uncorrelated into a 1d metric (radius). This indicates that what I said above is probably true, and that vertical vs horizontal dispersion may be uncorrelated. If I had tested at different range, this might change, since different types of dispersion affect each axis differently.
@@solarshooter I’m pretty sure that hornady said that the data for a group, not groups, is normally distributed and that 50 shots was enough to clearly establish the normal distribution. What they did not explain is how are you putting 2D data into a normal distribution! I asked after watching their small sample size podcast, and they said that the data being presented was from actual groups fired in their test rig.
Guys.. You need a MACHINE REST. This will eliminate the greatest variable. The person. THEN you know its the load of the rifle. This has been know for 150 years or more. I shoot service rifle competition. The channel has helped me with my 600 yard loads I enjoy all of it so don’t take this the wrong way. But you need a machine rest and a GOOD one.
Does someone have a source for machine rests and related information? Thanks.
True! I didn’t think of that either. Kiss method
What’s the point of knowing what your gun does on a machine rest? That’s not how you’re going to use it…. Therefore that’s not going to perform the same when it counts. I need it to shoot well on MY SHOULDER. When the shots count. You’re introducing another variable is all I’m saying
Very interesting info, thanks. So what do you take the mean radius from? Each group will have a different center, so how do you find the center which is common to 5 or 10 different groups? Are you using an app for this?
I am a believer in barrel exit time. My question to the both of you is Chromoly or Stainless steel. Is there a factor that could be calculated to adjust for using a stainless barrel?
Stiffness and density are the key values. They are pretty similar for those two materials, probably not hugely significant.
With all the vids out there no information on your reloading practice. Did you use a pressure ind. on the bullet seating? Is all the necks turned to a consistence size. Primers seated the same. What crush is on them. And last but not least, Did you filter your powder to use all the same kernel size?
So good I needed to watch it twice...
What is the title for Harold Long's book that Michael is referring to?
Or did he intend to say Chris Long?
Harold Vaughn, Rifle Accuracy Facts. There is a copy online for free.
@@solarshooter Thank You
Does your gun track well on your bags?
Thanks so much for the first 13 minutes. I will comment as I go. First off do you know your scope is holding well. See Alex Wheeler on a scope holding center. Next, are you holding the rifle the same? You mentioned this already. Only a rail gun will react the same every time.
The stiffening of the barrel really makes me think closely about carbon fiber. I should say "correctly done" carbon fiber with all the different manufacturers these days.
This also makes me think on the inherent problems of digital scales.
The VC (virtual center) seems to be what is most important here. Making me think of GD&T. 😂
Where does heat make a change in the barrel to change? Must mean if the barrel moves less the group should be tighter. May mean one chases load development over tuning the rifle.
It lowers the stiffness of the barrel material, which will change the bending mode frequencies. It also causes expansion of the material, which could cause warping and/or expansion/contraction of the bore.
Where have you been Keith? Radius measurments (mean and SD) have been discussed for years.
In some circles that's probably true, but it is rarely discussed in the mainstream of recreational shooting.
@@solarshooter it has been mentioned a bit over the years at Accurate Shooter, but clearly has not made an impression on many folks.
@@charlesking8542 Part of the issue with forums, is a lot of good info is hidden in a thread with a title or preamble that wouldn't lead a person to look at it for given reason. Or it is so far down in the thread that people have quit reading before they get near it. Found a few like that over the years, such as one on the Campfire that in the back part of a big thread, had people like Ken Oehler participating on internal ballistics, and a few others who worked in test labs. Or others that someone had put up a link to something by someone who really could speak on the subject, or a study on it.
What are your thoughts on crimping a 7mm bullet in a bolt gun with a cannelure ? My 139 gr btsp which is what I have to load has a cannelure and I see mixed answers to this!
You would have to try it to see if it works for you.
@@winninginthewind thank you!
My groups always look like Michel's overlay of multiple groups, lol.
There’s an issue with taking data shot at different distances and treating them as if they’re part of the same data pool. It has to do with harmonics and what you refer to as bullet time. What you’re really talking about is positive compensation. Changing the angle of departure based off bullet velocity in order to make shots land as closely as possible to the same point of impact. And the issue is that this concept of positive compensation involves a different amount of compensation at a different shooting distance. You do not get the same magnitude of dispersion at two different distances. It is physically impossible to do so. Ignore wind. If you shoot out to 1000 yards but have electronic targets at your intended 1000-yard distance and also at 100 yards you will not see the same group sizes at both targets. Will not. It is not possible to see 1 MOA at 100 yards and also see 1 MOA at 1000 yards from the same shots. The group sizes at both distances will differ. They must. The only way they will not differ is if all shots have precisely the same velocity. That never happens. If you play with “bullet time” to get your groups at 100 yards to be as small as you possibly can those same shots will print a larger group at 1000 yards, and vice versa. If you play with “bullet time” to get the smallest possible groups at 1000 yards the same shots will print larger groups at 100 yards. “Optimal bullet time” is distance specific. Another way to say this is you need to adjust the angle of departure for a load with a 20 fps spread by a different amount to hit the same point of impact at different distances. The angle of departure required for a shot at 2800 fps to hit zero at 100 yards will differ from the angle of departure required for a 2820 fps shot by x amount. Change the distance to target to 1000 yards and now the angle delta between the 2800 fps shot and the 2820 fps shot is no longer x amount. It is going to be much larger than x amount. And this is why if you’re getting the best possible groups that you can at 100 yards you are hurting your groups at 1000. Or 600. Or 300. Or whatever. The angle of departure delta requires changes for all distances. If you’re shouting the same load at 600 and 1000 you’re not getting the best results you could at both individually. And the same goes for using the same tuner setting at 600 and 1000. Whatever method you’re using to try to approach the optimal “bullet time” needs to be tailored to specific distances, no matter if it is a tuner, or charge weight, or bullet seating depth, all that stuff. It is distance specific. Simple projectile flight behaviour dictates this.
Positive compensation is an interesting topic, and one Bryan Litz also explored in his book. He did find some kernel of truth to it, though it was not conclusive. I look forward to seeing further study on the topic. The way to test it is to set up multiple targets on the same flightpath, so you can log the dispersion of the same shots at multiple ranges. I would expect that wind, mv, and bc dispersions, which are a nonlinear effect with range, will cause your group to grow with range. This is what I have observed in my tests at different ranges.
@@solarshooter It's what barrel harmonics are. You're making shots leave at different angles according to their speed, even if you don't know it. Play with a ballistics calculator. Punch in a 1000-yard zero and a 2800 fps shot vs a 2820 fps shot. Look at the drop at 100 yards for each one. When using a 1000-yard zero the difference in drop at 1000 yards will naturally be zero, as that's your intended target distance. But since the two bullets have different speedds they will be at different heights at every other distance. They must be in order to hit the same spot at 1000 yards. The difference in drop at 100 yards for a 1000-yard zero with a delta of just 20 fps might be a little surprising. Then do the same for a 100-yard zero and compare the drop at 1000 yards. The figures you see for drop at the other distance is how much vertical you would see on target at that distance when using a zero at the other distance. This is what I'm talking about when I say you will not see the same results at a distance ABC when you are testing/tuning at distance XYZ. When you do that you are trying to optimize the compensated departure angles to reduce group size, but when you go to shoot for score at the other distance you've unintentionally ensured you'll have larger groups at that for-score distance. You'll always get better results testing/tuning at your for-score distance. For some reason quite a large group of people are skeptical of positive compensation, when I can see no other reason to explain shot behaviour.
Positive compensation is barrel harmonics, it's just that you're trying to coax a specific desirable amount of harmonic to help you get better results. You can calculate how quickly you need the barrel to be swinging upwards in order to provide the necessary amount of compensation in the angle of departure for a given distance. You can then calculate whether or not your barrel can be swinging upwards near the speed required in order to approach that amount of compensation with regard to a given load. Then if you have a barrel contour that swings upwards a little too fast you can add mass (a barrel tuner) to it in order to slow it down to it lives in the sweet spot and provides just enough compensation to shrink your groups at your desired target distance. I had a rather unusual barrel built for a 22 rimfire benchrest gun, following Varmint Al's FEA design. Go to Varmint Al's website and to his page titled "22 LR rifle & tuner" and go about halfway down the page to the section titled "Two-Flats Barrel Contour with Vertical Flexibility and Greater Horizontal Stiffness" and you'll see what he came up with in the FEA software for a 50-yard barrel. He never did go to the trouble of actually building one. So I did. Hehe. Yes, it looks crazy. Yes, it works. It is just slightly too fast in its upwards swing for 50 yards. And only needed a very, very light tuner to bring it into the sweet spot. So far I've determined that it needs to be in the 31-32 gram region, but I'm still playing to find out exactly how much. Small changes can make quite a difference.
Are all your primer holes the same?
Did you check and point all your bullets?
Maybe a stiff rifle is like a go kart. Tune the rifle like a race car. Align it to the shooter and the barrel?
Add a tuner they said - it will give you another variable to adjust.......LMAO!!!!!!
Is this not what the hornady podcast 'groups too small' were saying?
I haven't watched it yet. Was it good?
Yes its very similar to what you have been saying.
@@winninginthewind We've got a bunch of podcasts that may interest you. Thanks for looking!
ua-cam.com/video/QwumAGRmz2I/v-deo.html
ua-cam.com/video/6yZyXwy40JM/v-deo.html
Lengthy, my preference is for a more condensed discussion, conclusions, or edit. Do like the overall message.
Circumference and depth?
So the normal data test said you didn’t have a normal distribution, but we based the rest of the conversations that are based on that…
I think he's understating his statistical acumen-- any engineer has had at least some stats training.
I've tried load development.. duty the whole seating depth thing I've found to be the biggest load of crap... beautiful groups won't repeat and I have no idea why.. it's seem to be a load of marlarky
Every rifle has an average
So he watched the hornady podcast "your groups are too small" and the follow up videos and then takes it as his own info....
This was a practical demonstration of the theoretical concepts presented in that video. It does agree with some of their conclusions, just as their video agreed with Bryan Litz's conclusions. However the takeaways regarding focusing on shot radius, the efficacy of testing with small groups, and the importance of long term characterization with data vs. constant tuning are valuable.
All of this dat doesn’t mean much if you have been shooting on an outside range. The wind is just an unknown factor that messes everything up.
Every engineer I’ve met creates problems for a living. Millwrights solve them.😉
How about waxing the fore stock of the rifle like skies.
Been there, done that. It works.
Too much to understand…just load and shoot.
.....infinite ways to measure anything,....&...infinite ways to interpret,......now to see which works.......interesting to say the least.
🌹 Promo sm
Nice discussion guys, tons of great info….., maybe an executive summary/BLUF for the newer shooters
Thanks!