Garmin Sleep Tracking: A Scientist’s Perspective
Вставка
- Опубліковано 28 чер 2024
- My perspective as a scientist on a study of the Garmin sleep tracking algorithm. Subscribe to my weekly Newsletter(!) on www.robterhorst.com
Instagram: / quantified_scientist
Twitter: @QuantifiedRob
👇Amazon Affiliate* (paid) links. Supports the channel, doesn't cost you any more!👇
Garmin Vivosmart 4: geni.us/OspT8b
Garmin Venu Sq: geni.us/xKANg
Garmin Fenix 6X Pro: geni.us/PWAAdzK
Fitbit Charge 4: geni.us/wjAfjU
Fitbit Sense: geni.us/CSfYQ
Fitbit Inspire 2: geni.us/iZHlb
Huawei GT2 Pro Classic: geni.us/iCgX
Huawei GT2 Pro Nebula Grey: geni.us/yD7ew9V
Huawei GT2: geni.us/xpAWsr
Huawei GT2e: geni.us/aYVruh
*My opinions are completely my own. However, I do earn a small commission if you purchase any of these devices via my affiliate links. It does not cost you any more money if you use these links.
The EEG tracker I used is called the Hypnodyne ZMax: hypnodynecorp.com/ from @HypnoDyne Corp
📹Fitbit Videos📹
Fitbit Charge 3/4 Sleep: • FitBit Charge 4 Ultima...
Fitbit Charge 4 Heart Rate: • Fitbit Charge 4 Review...
Fitbit Sense First look review and test: • Fitbit Sense Accuracy ...
Fitbit Sense Heart rate: • Fitbit Sense Heart Rat...
📹Apple watch videos📹
Sleep Cycle App: • Sleep Cycle App Review...
Steps: • Apple Watch 6 Step Cou...
Test: • Apple Watch 6 Accuracy...
📹First Garmin Venu Sq Video📹
• Garmin Venu Sq Heart R...
📹Oura Ring videos📹
Sleep Test: • Oura Ring Scientific S...
HRV Test: • Oura Ring Review: Hear...
Temperature Test: • Oura Ring Ultimate Tem...
📹Withings Scanwatch videos📹
Unboxing and first review: • Withings Scanwatch Rev...
Sleep test: • Withings Scanwatch Sle...
Heart rate test: • Withings Scanwatch Hea...
Q&A with Withings: • Scanwatch Q&A: Answers...
⏱️This comparison is part of a larger project where I spend over 11 hours a week tracking everything in my life. If you'd like to know more:
• This mad scientist sca... ⏱️
Check out the paper: www.garmin.com/en-US/blog/hea...
Timestamps:
00:00 General intro
00:57 Study Background
02:25 Results: Sleep Test Comparison
06:34 Results: Sleep Test Overview
08:25 Results: Compared to Garmin Venu Sq
09:30 Results: Compared to Other Wearables
10:21 Conclusions
11:13 Recommendation: Good or Bad?
11:48 Limitations of the Study
🎵Music🎵
(1) Not Without the Rest by Twin Musicom is licensed under a Creative Commons Attribution license (creativecommons.org/licenses/...)
Artist: www.twinmusicom.org/
(2) Wonders by JVNA
(3) Can’t Sleep (UA-cam Audio Library)
(4) Classique (UA-cam Audio Library)
(5) Blurry Vision (UA-cam Audio Library) - Наука та технологія
Subscribe to my weekly Newsletter(!) on www.robterhorst.com; Instagram: instagram.com/quantified_scientist
Twitter: @QuantifiedRob
👇Amazon Affiliate links, support the channel!👇
Garmin Vivosmart 4: geni.us/OspT8b
Garmin Venu Sq: geni.us/xKANg
Garmin Fenix 6X Pro: geni.us/PWAAdzK
Fitbit Charge 4: geni.us/wjAfjU
Fitbit Sense: geni.us/CSfYQ
Fitbit Inspire 2: geni.us/iZHlb
Huawei GT2 Pro Classic: geni.us/iCgX
Huawei GT2 Pro Nebula Grey: geni.us/yD7ew9V
Huawei GT2: geni.us/xpAWsr
Huawei GT2e: geni.us/aYVruh
Are you Dutch Rob?
I've subscribed to your newsletter a few weeks ago and I received the news about the Amazfit GTR2. To bad I've ordered the GTR2 on the day before I received your weeklynewsletter :'). Are there any smartwatches/trackers with a round design that can compete with Fitbit's heart/sleep monitor capability's? I would love to have a nice round design and some 'accurate' data on a consumer level
Can you to test Samsung Galaxy watch 3 for sleep tracking? It will interesting how it's work in this device.
This is very helpful. I know going from a Fenix 5 to a Fenix 6 that the "advanced" sleep tracking is much improved over their old algorithm, but there is still a lot of room for improvement. Anecdotally, it is better able to detect when I fall asleep and wake up. The old algorithm was much too keen to say I was asleep when I was just merely in bed.
I tested a sense for 60 days vs a fenix 6 and they match so I would worry
Great video once again! I really appreciate that you are going through the study and explaining it to the average person! A fun video idea / project you can try is to attempt to create your own sleeping algorithm and deploy it in the form of an app onto one of the watches 😀
Hi Rayden, thanks so much for the compliment and the suggestion! Have a great day 😊
Could you please tell which fitness tracker is the most accurate among all brands and models?
+1 on the ques
@@juliens3505 According to Rob, Fitbit's accuracy isn't great, so it's a NO for me!
@@samygold100 So they're all a no then? What you end up getting an EEG?
@@NickRoeder I got a dreem 2 headband, not something I could wear on my wrist but it's more accurate than all wrist bands.
@@NickRoederYou can track amount of sleep accuracy, no wristband is very accurate at tracking actual sleepstages. However fitbit does a fairly good job
I think Samsungs sleeptracking is great, it easily picks up naps on the middle of the day and it knows when you go to sleep and when you wake up right on the minut. However you can't trust sleep stages.
Garmin is trash
Please test Garmin Fenix 6 with the new sleep update that calculates sleep locally on the watch. A HUGE improvement in my experience!
Please do!
Agreed - really want to see this done
Did he re-test? I'll looking to buy the fenix 6 pro as there is a massive price drop but this was putting me off.
Nothing beats science. Now THIS is a review. Thank you.
Science is a spectrum not a monolith. The fact that Rob doubts the peer-reviewed study with 55 individuals over his single experience is telling.
Great video again. Thanks! When I saw the EEG graph (at 2:51) I thought that this is a good way start comparing things, but it doesn't really answer some of the tough questions. But then again, it's very easy picture for most people, so I understand the reason why this chart has to be here.
Just like in some of your older videos, there was a better comparison table coming up. I wasn't disappointed. Later (at 6:58) you showed the agreement table I was waiting for and it answered pretty much all the questions I had in mind. Not only that, you showed several agreement tables from your previous experiments. That's the sort of depth I want to see.
Hey great video!
You should totally do the garmin vivosmart 4!
Keep up the great work!
Excellent review, love the details and honest opinions..maybe could you consider a review for Garmin Venu
Loved the video, I also liked the other huawei video you made, it intrigued me how it compared to other watches and saw that it did really good . Would love to see it with some of their newer watches like the band 4pro!
This is one of the best things I've ever seen on the internet. Thanks
Keep up the great work! 👍
AFAIK the Fenix 6 got a new algorithm with an update, that has not found its way to the other devices, yet. I wonder if that is better than the current algorithm.
Great video as usual !
Perfect analysis which is backed by science. I really like and appreciate your work.
Do you think the shape of the device can effect the accuracy
For over a year, I’ve been researching different brands of what would be best for sleep monitoring and best correlated with my own feelings about sleep. It would also be really interesting if you tested the Polar Vantage M2 or Vantage V2 series. Polar is the best sleep monitoring based on my own experiences. I have actually tried the following products: Garmin Venu, Huawei Watch GT2 Pro and Polar Vantage M.
Thanks for the video, Rob! Interesting that you didn’t mention probably the biggest driver of the differences between the peer-reviewed study and your anecdotal experiences: 55 individuals vs 1 individual. Furthermore, how many nights did you test the venu sq and any of the orher wearables in your channel?
Hey can you try the bio strap next ? Great work btw it’s tremendous help !
Very interesting, useful, and well presented. Thank you so much.
You're very welcome!
I'm very happy to find you. I hope you have something on polar. I'm gonna look for it now. Cheers
Have you performed a longitudinal analysis of your own sleep patterns per your EEG standard device?
Fantastic! Love the comparison between wearables
Thanks Hailey!
For any of these fitness/smart watches, I would take the data they provide as guidelines or rough estimates for the user. The trick it seems is filtering out the products that don't do it very well.
I apologise for being late to your subscription list! You are awesome!
Predicted or detected? At some point in the video I thought you were testing a crystal ball LOL! Thank you for the video!
Thanks for the interesting review! I love your videos and hope to see more tests soon. I've recently bought Vivoactive 4 myself and somehow it fails to detect my sleep falling hours badly, it keep thinking that I'm sleeping when I'm not, I wonder if you plan to review Vivoactive 4 too?
Thank you!
I have the vivosmart 4 and find it is really bad for sleep tracking.
Looking forward to your OpenBCI sleep tracker. We will built it also, sooner or later, but eventually with the Python Sleep software.
Would be good to see you review a Garmin with the new sleep tracking that is first beat (introduced to some models in mid 2020)
I swap my Fenix 6 for a Fitbit that costs a 1/10 of the price to do sleep tracking.
Great review, do you have an update for the Whoop 3.0 sleep and heart rate accuracy, its been gaining tremendous popularity and ive been wearing it religiously. I watched the Whoop video but maybe a longer term review would help. Thanks!
Do you know if all newer Garmin smartwatches have the same algorithm for sleep tracking. For example, with the more expensive Garmin Fenix 6 series. If the algorithm is different, then it would be very interesting to see how sleep tracking works in the Fenix 6 series.
Two questions for you Sir....1) are you doing a sleep study on a Garmin Fenix 6 pro 2) which model and brand health/fitness tracker smart watch do you think is most accurate? I am on the hunt and SO much information is certainly confusing. THANKS!!!
Please do HR and Sleep accuracy for the new Garmin Fenix 7 / Epix 2 devices. I am really curious if they have improve for their top end devices.
Any chance that you'll do a deep dive video on the Vivosmart 4? Specifically interested in comparing to the Inspire 2, which I think is pretty great, but I don't wish to support a company that puts some of the consumers' sleep data behind the premium paywall.
Would I’ve to see an update on this.
Subscribed hoping to see reviews for Garmin Venu 2 and Vivoactive 4.
I do see very similar results with my garmin Venus sq. Great device but when it comes to sleep tracking not useful. Yesterday evening it detected deep sleep while I was still reading...
Been sleeping with a sense and a fenix 6 for 6o days and they pretty much match, on 2 days the sense showed terrible sleep which didn't reflect the garmin nor my personal perception of that nights sleep
It's true.. Garmin's sleep tracking is very, very bad. It is so sad, because overall their products are great! I bought Lily and it keeps showing me unrealistic things! For example, I clearly remember that I could not fall asleep and was still awake by 00:00, but Lily classified it as deep sleep (I wasn't moving around, but I remember that I checked the time on my phone). Also, many times in the morning it does not differentiate between sleep and wake. I literally have to jump out of bed straight away, because it will say that I am still sleeping, even if I am sitting in bed. I am still pleased with my purchase, because I bought it a) because it is small enough for my wrist (I was looking at Fitbit sense, but it's just too huge..) and b) because I want to count my steps and keep an eye on my heart rate, stress, etc. And also I do not think that I suffer from bad sleep, so the lack of accuracy isn't so sad (it could still be very helpful if it was accurate!).
Thanks for the feedback L Z, have a great day!😊
Yeah i´ve had the same problem with bad tracking of sleep. So it was pretty much useless for me and i stopped wearing the Fenix 5 Plus over night.
@@jayrock1727 I am wondering how accurate is the Body battery metric then? IF it is based on the quality of sleep, but the sleep metric is useless, then the validity of this metric is questionable too.. It *feels* kind of accurate and does not tell me that I am ready to climb Mt. Everest when I feel like I have to take a break.
@@lvzane they state in the app that all the data acquired can be seen as a guess-o-meter. I wouldn't rely on this for mount everest. If you need data you can rely on then go see a proper doc with proper equipment to get proper numbers.
My Garmin says my sleep is always bad. I am testing a Fidbit based on advice from the Qualified Scientist, and on my first night wearing both devices, I got a Fair score (71) on the Fidbit and Poor score (53) on my Garmin. One thing I noticed is that Garmin uses Stress Levels while sleeping into account while the Fidbit doesn't. I wonder if this has anything to do with it.
I would also add the differences between your results and Garmin could be sample size (55 vs 1)
Sample size of 1 is meaningless. You could see differences between the best, worse and average percentages.
His sample study would make it #56 and slight below the average Garmin result. All that means until we get more data.
Is there a top 5 from all tested watches?
Very helpful 👍
Glad to hear that Sam! 😃
Is there a fitness tracker that doesn't look like those cyberpunk 2077 wrist things but gets a decent sleep analysis?
Please do on honor band 6 also
Hi Rob,
Great video!
Would you still recommend the Fitbit charge 4 for sleep tracking in 2021?
Thanks
You have to pay $10 for premium membership to access sleep tracking. Wanted to buy fitbit but found out about it and then gave up on it
What is the best watch to check O2 while sleeping? I really don't care what stages I'm in while sleeping, but if I have any apneas.
Its cool. Thanks
I love your videos, as someone who struggles with sleep since forever I am in search of the best wearable.
I have a garmin 6s pro solar, an oura ring and an Apple Watch, all with very different results on deep sleep. Have you done a comparison between devices? Thx
Which one of those seems to work best for you?
Can you do the Garmin Vivoactive 4 please? All sensors :)
Can you share the link to the scientific paper?
Great video. PLS do the fitbit versa 2, i really wanna know if it is worth the hype.
I have a Garmin Vivoactive 3 and my wife has the Versa 2. No comparison. The Garmin is a good activity tracker but the Versa 2 is in a different league when it comes to sleep tracking.
@@cusebike5391 thank you so much fir the answer. What is it that the versa 2 provides in terms of sleep tracking that you enjoy so much? And do you know if sleep tracking on fitbit is generally good like that?
Please do the garmin venu 2 next!
Very nice review. As someone who just purchased a Garmin device for running and triathlons in the future, I am still a bit interested in sleep tracking.
I was wondering if another reason for the discrepancy between the paper and your personal findings is that Garmin is showing the results across many individuals vs you measuring on yourself?
Inherently I'd expect a measurement on a single person to have more variability than multiple measurements on various people. Additionally, in your case you are essentially optimizing for what tracks sleep the best for you while Garmin is trying to track that for everyone including people of different ages, genders, sleep disorders, and so on so its not at all that surprising to me that the results would differ because its a trade-off inaccuracy for generalizability.
Perhaps they track sleep slightly worse for someone like you but they are slightly better at tracking in other individuals.
This is just my thought though so I can definitely be wrong XD
Your thoughts are 100% spot on.
Is FirstBeat trickled down to Vivoactive 4 yet or do you have to get a Fenix 6?
Great great great video... Objectivity is a rare thing, while here is abundant.
I second all the comments asking for an evaluation of the Fenix 6 new algorithm, since... Well... It's the newest and it's the one present in the watch I wear :) would actually love a full scientific review for the Fenix 6, though I do understand it's an old device and we should be near its sunsetting for the possibly upcoming Fenix 7
Would be interesting to compare to the Sleep as android app.
Huawei band 5 I had before getting garmin was so much better at sleep tracking and insights. Garmin quite often registers me sleeping an hour before I truly go and attempt sleeping :'D
Hi Simona! Yes, I think that kind of mistake is the biggest mistake a sleep tracker can make. Accurate sleep onset and wake-up time is the most important!
Very good waiting for honor band 6😍
I love my Honor Band 6! Been so happy with it.
could you test the samsung galaxy watch 3?
Garmin's sleep tracking is bad... what a pity, their devices generally allow to turn off bluetooth, and it's a good, good point.
Can you please do Garmin Vivomove Style (Luxe), specifically sleep tracking?
Whats the best finger spo2 device you would recommend ?
Bedankt weer Rob! Ik heb de Fenix 5 en die werkt idd ook niet al te best
could you test the garmin enduro 2?
Does anyone know of a sleep EEG that can be used for a retail user.
Why do these wearables that have SpO2 pulse ox sensors not include this data in the sleep analysis? Detection of sleep apnea through low blood oxygen measurement would be one of the most important indicators of sleep problems and a very common one. Have the devices in 2022 improved on the analysis and included SpO2 data as part of the sleep program?
Amazing video. How would you compare polar here ?
He's done a video on the Polar Vantage M if that helps!
HI @@Inzanti. Yes, and it a great video. I was hoping to see it as one of his recommendations in this video. Again this is a great video.
Which if the products you have reviewed, would you say is the overall best??? and which is best for sleep???. Ty for making these videos btw ! :D
Check out his Fitbit videos!
@@Inzanti i have, but he might have had later thoughts on some og Them 😹
Can you test the Garmin Venu 2?
Please do a video on forerunner 255
Can you suggest best smartwatch under 20k in india ?
Is this watch able to wake you up at the best moment based on your sleep analysis? I cant find any details on that
how about stress monitoring? how does that work?
You rock!!
I was hoping you could test the Garmin Lily? Sometimes I'm suspicious of products marketed to women only. I hope it's as good as products marketed to men.
Very Informative channel, NEW Subscriber and Garmin Fenix 6X Pro Solar owner.. 🇺🇸
My girlfriend is a nurse and has power naps throughout the day as her job is very demanding - its not a 9-5 job....
But Garmin only tracks the last nap... not all the naps through out the day - is there a way to get Garmin to add them together ?
Really a great content. Really like your channel. What I'm missing from these videos about sleep tracking is what does these cycle means regarding our sleep quality. Maybe you can create a video about that and explain the science behind it?
Please can you review the new garmin venu 2 plus? I would like to know if they’ve made any improvements to sleep tracking, I am considering this watch or the Fitbit sense however I prefer the smartwatch and design features on the garmin.
Hi! I would love to, but Garmin did not get back to me yet. Hopefully they can lend me one!
I have so many questions I hope you'll address someday.
1. Are there differences within a brand line (Garmin or FitBit) due to hardware or can we say all FitBits and Garmins will produce the same results because its a software algorithm.
2. Since the differences between devices seem to be based on an algorithm, can all wearable devices improve their sleep measuring accuracy with software updates or will it require new hardware too? If so will you keep us in the loop when devices get better?
Hi Joey, thanks for your questions!
1. I think that the differences in the algorithms between devices are likely the main thing that differentiates sleep tracking between companies. Most devices have a similar array of sensors (though the quality will vary), and the resulting measurements need to be converted into human interpretable data. It is at this stage where the main differences likely happen. So, yes, software updates can make huge differences. That being said, especially for things like heart-rate measurements, the quality of the signal needs to be sufficient. As they say, "garbage in, garbage out". No matter how good the algorithm, if the actual measurements are really bad, the results will be bad.
2. Yes, sleep tracking can improve after updates. However, creating a new algorithm for scoring sleep stages is a big undertaking, and I suspect that many companies will not frequently completely overhaul their system. Best would be if they collect a lot of data from many people with both their watches and polysomnography at the same time. Using this data, some data scientists could create a (hopefully robust) algorithm (e.g. neural network) to estimate sleep stages using data from the watches. All in all, this is a huge investment, and I suspect many companies do not feel much incentive to update their algorithm if consumers "feel" it is good (even if it is not).
Hope this makes sense!
@@TheQuantifiedScientist I'm hearing an opportunity for a guy like you to create a better algorithm.....thank you for the reply!
We want a video about the amazfit GTS 2
Although I think you are doing your best analyzing these devices, a limitation and a difference between your study and Garmin's is the sample size. Yours is n=1, theirs is n=55.
n=55 is a joke when compared to the millions of units sold. There are other problems, some mentioned, that n=1 a much better overall analysis.
Do smart Bands or smart watches, record when you sleep at the morning? In my experience with mi band 6 it does not record any data.. I usually sleep 8am to 10am in the morning. It was not recording any sleep data. Do you have any idea, on a smart band/watch that can record even it is not night time or dawn? Any recommendations. Thank you!
Very good video. But from where did you become these EEG trackers like Zmax or sleep profiler? And how much are they? Thanks 😊
I purchased Venu 2 a few months ago and returned it because both the sleep tracking and altimeter performed poorly. I think I am done with Garmin watches!
I wish Garmin would send you Fenix 6 watches. Because those should have improved sleep detection compared to cheap Garmin models. :)
Do Venu 2 please
Hi rob
I love those videos and your completely scientific approach.
I got pretty much every device
Our whoop dreem 1 and 2 muse s
Fit bit etc
In your opinion, which of the non-egec is the most accurate for sleep
What would be great a top 10 list?
Hi Reto! Thanks for the compliment. Cool that you have so many devices! What is your impression of these different ones? I am planning to make a video that has an overview, so stay tuned!
@@TheQuantifiedScientist
I actually got about 10 times the amount listed
Than I just mentioned I fairly deep into bio hacking.
For me my favorites are
Dreem is absolutely the best but a bit of a pain to wear
Oura ring for sleep is fairly good
Whoop great for working out love the strain feature but sleep I find not to accurate but hate that it Dosent interface with Apple health
Test the amazfit t rex pro please bro
vienna is a beautiful city!!
Hello, I need to ask you something
Which Gramin watch you can recommend to me. I'm running often, and I'm planning to run half marathon again and marathon in future. I want to track my energy level, stress level and my quality of sleep too. Small note, I'm using regularly Nike Run Club app too track my runnings, and I'm planning to use Gramin watch for that too, and sync to NRC app.
And my budget is up to 200 euros max
Thanks you for your time.
All best!
Garmin is one of the worst brands in sleep measurement, when I had a Forerunner 245 it was the worst in conjunction with the body battery, it marked me that I fell asleep at 22 and woke up at 9 am, when in reality I fell asleep at 1am and woke up 10am, a significant delay. Right now I am using the Xiaomi Mi Band 5 GLOBAL and it has an impressive accuracy in terms of identifying the hours when I sleep and wake up, sometimes it errs but for a maximum of 20 minutes.
Exactly what I have been looking for. I like the garmin watches but the sleep is always off. Also, I think you live in Vienna but you're Dutch. If so, Dankjewel!
Amazing how such a renowned brand as Garmin performs so badly compared to the much cheaper models like Fitbit.
I swap my Fenix 6 out for my Honor (Huawei) Band 6 for sleep tracking. A common theme in these comments 😂. It's just annoying to have to set your sleep time every night if you have an irregular sleep schedule or take naps. I'll often over or undersleep and the Garmin has no idea. Fitbit, Honor/Huawei, or Mi Band for sleep tracking imo (and Fitbit has some features locked behind a subscription so).
I love you. You are awesome. Just had to leave that here lol.
You look way too happy for me not to subscribe, thanks for the information brother.
waiting for new video! :)
Hi Dawid, I just released a new video (not on Garmin, but on Huawei)!
It’s like a c+.
I had a Fitbit Versa 2 before I switch to Garmin Instinct. I can say that Fitbit's algorithms are much more accurate than Garmin's.