Apple Watch Ultra Live Heart Rate : Scientific Review
Вставка
- Опубліковано 22 лип 2024
- 🔴 Apple Watch Ultra: geni.us/AppleWatch_Ultra
🔴 General Amazon link*: geni.us/thequantifiedscientist
🟢 Instagram: / quantified_scientist
🟠 Shorts: / @smartwatchshorts
🔵 Newsletter: www.robterhorst.com
🟡 Twitter: @QuantifiedRob
* Affiliate (paid) links support the channel by earning me a commission on any sale, and it doesn't cost you any more! My opinions are completely my own, and this content is not sponsored.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇 $30 off any new WHOOP membership, affiliate* (paid) link, supports the channel!👇
join.whoop.com/QuantifiedScientist ($30 off any new WHOOP membership)
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇Amazon Affiliate* (paid) links. Supports the channel, doesn't cost you any more!👇
▬▬▬▬▬▬▬▬▬▬BEST WATCHES▬▬▬▬▬▬▬▬▬▬
Whoop Strap 4.0: join.whoop.com/QuantifiedScientist
($30 off any new WHOOP membership)
General Amazon link: geni.us/thequantifiedscientist
Samsung Galaxy Watch 5: geni.us/SamsungWatch5
Huawei Watch Fit 2: geni.us/HuaweiWatchFit2
Huawei Watch GT3 Pro: geni.us/thequantifiedscientist
Huawei Watch GT Runner: geni.us/HuaweiWatchGtRunner
Huawei Watch GT 3: geni.us/Huawei_Watch_GT3
Apple Watch Ultra: geni.us/AppleWatch_Ultra
Apple Watch 8: geni.us/AppleWatchSeries8
Apple Watch 7 41 mm: geni.us/AppleWatch7_41
Apple Watch 7 45 mm: geni.us/AppleWatch7
Apple Watch 6: geni.us/0gNGeO
Apple Watch SE: geni.us/Apple_Watch_SE_44
Garmin Fenix 7: geni.us/Garmin_Fenix_7
Garmin Venu 2: geni.us/Garmin_Venu_2
Garmin Instinct 2: geni.us/Garmin_Instinct_2
Polar H10: geni.us/hnXc
Polar OH1+: geni.us/D7bs5
Polar Verity Sense: geni.us/polarVeritySense
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
📸My Gear* (paid links, supports the channel and doesn’t cost you any more)📸
Camera body (main): geni.us/SonyA6600_body
Camera lense (main): geni.us/SigmaContemporary
Camera (B-roll): geni.us/CanonEosM50MarkII and geni.us/Sony_ZVE10
Light: geni.us/Godox_VL150
Softbox: geni.us/NiceFotoSoftbox
Light stand: geni.us/WalimexLightstand
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Timestamps:
00:00 Intro
01:31 The Issue
02:24 The Test Setup
04:19 Data Processing
05:05 Test 1: Indoor Cycling
07:00 Test 2: Compared to Reference
08:33 Test 3: Quantifying the Lag
09:47 Test 4: Replicating Results
10:23 Troubleshooting
11:31 Test 5: Overview of Results
13:10 Test 6: Cycling Outdoors
14:27 Test 7: Weight Lifting
15:04 All Results Summarized
15:20 Conclusions and Consequences
I recently noticed that there seems to be a delay in the heart rate detection of the Apple Watch Ultra, and in this video I am going to systematically investigate if this is indeed the case. In previous videos I have shown that the Apple Watch Ultra is one of the most accurate heart rate trackers out there. However, this is based on the heart rate you see in the Health App after you complete your workout. The problem I noticed is the live heart rate you can see on the screen of the Apple Watch Ultra. Just looking at it while I was training, I felt it did not match exactly with the heart rate recorded by an ECG chest strap, and it even seemed different from the heart rate I saw in the Apple Health app afterwards. The heart rate in the Health App might have been processed further, and be different from the live heart rate shown on screen. So, in this video I am going to use a bunch of computer code, data science, image recognition and two GoPro’s to figure out if there are any issues with the live heart rate tracking of the Apple Watch Ultra. What I found is likely not going to be an issue for most people, but it will be for some.
I have had the Apple Watch Ultra for about 4 months now, and it is an amazing piece of technology. In many of the tests I shared with you we saw that the Apple Watch Ultra is one of the best heart rate trackers that is currently on the market. As far as I know, everyone that has tested the heart rate tracking accuracy of the Apple Watch Ultra relies on the data that is recorded in the Apple Health app, or another connected app like Strava, which is connected to the Health app. I also use this data in all of my testing, and this is the data that the extremely positive reviews I shared with you are based on. However, I recently noticed while I was on my spinning bike that the heart rate my ECG chest strap reported did not agree as well with the heart rate reported on the screen of the Apple Watch Ultra as I expected. So, I started thinking; It could be that the raw heart rate signals detected by the Apple Watch Ultra are re-processed in the Health App, giving the better final accuracy, or it could also be that there is some delay in the live heart rate shown by the Apple Watch. So, I decided to dig in deeper, and investigate what is going on here. - Наука та технологія
🔴 Apple Watch Ultra: geni.us/AppleWatch_Ultra
🔴 General Amazon link*: geni.us/thequantifiedscientist
🟢 Instagram: instagram.com/quantified_scientist/
🟠 Shorts: studio.ua-cam.com/channels/cpARGzXNaC62UQWns2dh2Q.html
🔵 Newsletter: www.robterhorst.com
🟡 Twitter: @QuantifiedRob
* Affiliate (paid) links support the channel by earning me a commission on any sale, and it doesn't cost you any more! My opinions are completely my own, and this content is not sponsored.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇 $30 off any new WHOOP membership, affiliate* (paid) link, supports the channel!👇
join.whoop.com/QuantifiedScientist ($30 off any new WHOOP membership)
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇Amazon Affiliate* (paid) links. Supports the channel, doesn't cost you any more!👇
▬▬▬▬▬▬▬▬▬▬BEST WATCHES▬▬▬▬▬▬▬▬▬▬
Whoop Strap 4.0: join.whoop.com/QuantifiedScientist
($30 off any new WHOOP membership)
General Amazon link: geni.us/thequantifiedscientist
I hope the 4-5 sec. lag is not because of a limitation of the processing power of the cpu in the Apple watch! 🤞
A las, in order to get realtime heart rate displayed without any delay, you would need to disable data recording of the heartrate...meaning, the Apple watch hardware might not be able to do both, without adding a 4-5 sec. processing delay??
Are you in contact with Apple about this, Rob? 😉
Ps: Thanks for all that you do with this channel! 👍
love your chopstick rig but you could've just mirrored apple watch screen onto your iphone and record it. It's supposedly in the iphone's accessibility settings, dunno cuz I got watch SE and the required is series 6 or better.
Amazing work as always!
I have one suggestion for you, @TheQuantifiedScientist Could you consider doing a study on body composition scales? They provide information much more useful for health tracking that simple weight (% of fat, % of muscle, bone mass, etc.), however the data quality can be rubbish in my experience and that of many opinions on the internet.
I own a Xiaomi one (quite useless), there are HUAWEI ones, and many others. Working principles change too, with only feet sensors VS feet and hand sensors, for example. So there's enough variability in price and working modes, but no one out there checking the quality of the data. It seems we need you! :)
This would also be a different kind of test for you, where you could easily compare all scales in one video but would need to spend some time gathering volunteers with different body compositions. I'm actually a Data Scientist and would be glad to help/support with the DoE if you want.
I would love this video! Please, consider it! :)
Great job !
I have 2 questions about this experimentation :
- Do you think the other Apple Watch models (Series 8, 7, 6, …) have the same display latency ?
- When you plot both the "live HR" compared to the "HealthKit HR", we can see that there seems to be more resolution on the data from HK : can we say that it is even more precise ? or does the difference between "live HR" and "HK HR" only resides in latency ?
- 3rd question (ok, I said 2, but this one is important too :p) : I know that HR from Polar H10 is sampled every second, whereas HR from HK is sampled every 5s at most : how do you perform the correlation when you have a signal 5x times "denser" than the other one ?
Thanks Rob !
Congratulations for this effort. Your community really appreciates! Thank you so much.
These scientific reviews it’s what the web is missing (you just find specs and sensations)
Seriously the time he takes to make this videos and the effort he puts on them to give us accurate information to maybe take a decision if we should buy or not any device is amazing 👏🏼
Thanks so much Don!
A joy and education to watch author's videos. I looked up author's website indicating he just started a Postdoctoral position at the CeMM Research Center for Molecular Medicine in Vienna Austria. This explains how he combined science, engineering, and biology which seems required to provide his high quality quantitative results. I completed 4 year physics degree giving perspective on his methods and results!
You could use AirPlay mirroring to your iPhone in iPhone settings -> accessbility -> apple watch mirroring, and record the iPhone screen.
But that wouldn’t involve chopsticks and imagination.
Imagine him reading this exact comment 🤣
Would have made the image recognition much more reliable and remove the GoPro variable.
Assuming it would work during a workout reliably. Causing the Apple Watch to do more work might cause a variable as well, potentially causing more of a delay than might be normally. So maybe the GoPro use is justified.
Wow, did not know this was possible, thanks for the suggestion😊! Would be curious to see if that also has some delay like someone else commented, but the text recognition would for sure be more reliable! Great suggestion and thanks for watching!
Stunning work, the effort you put into the research really shows! Kudos!
I really enjoy your videos and I really liked seeing a little bit of the process behind the scenes. It was very informative!
Awesome, thank you! 🙏🏽
Nicely done, very useful material, thank you!
Create work as always! Thank you Rob.
As always love your dedication! Another great video!
Thank you! Cheers!
Loving what you do Rob, keep it up (and humour too!)
Thank you for another great video.
Did you notice this also on other Apple Watch models?
Awesome work Rob! Love seeing you get your teeth into this, you nailed it.
Amazing work, honestly this is how the heart rates should be analyzed for all watches. That way you simulate the real life experience of actually using the screen for your workout. But this delay analysis is interesting because it shows the rate limiting step.
Glad you liked it Chris!😄
I'm about to buy an Apple Watch Ultra, will use your link to support the channel. Keep the good work!
Your work is amazing! 👏👏👏
Amazing comparison and excellent research done! Congratulations! 👏👏👏
Awesome job mate!
Your content in wearable testing is by far the best on the whole of Internet. Kudos from a fellow scientist!
Great video! I have noticed the delay when I am doing zone 5 spin workouts. Subjectively, it feels longer than 5 seconds and my heart rate goes up even during the recovery period.
My daughter is going to do one year research at Vienna University from September. Any tips for her?😊 keep up the good work!
wooow! thank you for this exceptional work!
Fantastic effort 😮
I congratulate you Rob for this amazing work
Oh, man... I love you! Image recognition to test HR measuring capabilities of the watch 😀 Great video!
awesome research! thank you mate.
Now I'm wondering if, when using a chest strap, there is an on-screen lag too 🤔
that could help making hypothesis on why there is such a delay.
in any case, the accuracy between the watch and chest strap is amazing!
I got a chest strap for the winter as I got a lot of gaps, the watch struggling to capture my HR while running
Thank you for your amazing reviews! Do you think we can expect the same results with Apple Watch 8?
amazing work a usual. Maybe someone in the community can design a brace to hold the go pro on your arm for future tests
Great Video!
Can you also do some quick tests on this for Apple Watch 8? To see if it’s a common problem. By the way, I love your videos and appreciate your works!🎉
Great content! Does the Huawei GT 3 or 4 have the same delay as the Apple Watch?
Dedication!!! Thank you for this Rob! Yes, interval training...which made me get a chest strap and watch in the first place some 20 years ago.
Will you do a review on Amazfit band 7? Great work btw!
Gold as always. Many thanks!
My pleasure!
Maybe the delay is by design? Like if you looked at your watch and it said 75, 74, 75, 76, 75, 75, 76, 77, 76, 77, 78 all in quick succession it would be a bit annoying and overwhelming so maybe they only update what is displayed on the screen every few seconds just to make it easier to read when you're running or whatever. Just a thought.
As a health tracker, including exercise, daily activity, sleep, comfort and cost, which one or two would you recommend?
Nicely done! I used tesseract for a similar project extracting timestamps from security camera footage. Great video 👍👍
Thanks NeuroApe!
I watch your videos with love and interest and very descriptive detailed videos. Do you plan to do a full scientific review of HUAWEI WATCH GT 3 SE soon? I am very curious. Also, as an iPhone user, what are the watch models you can recommend among the watches you tested, thank you, I look forward to the continuation of your videos.
I love your videos mate.
Glad you like them!
Rob, do you know if the Apple Watch does the same when connected to the H10? Or this delay only exists when using its own optical sensor?
Great work! I also find that the sleep tracking changes after being uploaded to the health app. Specifically the sleep stage amounts change drastically when first uploaded and after about 10 mins in the health app. Not sure if you are seeing something similar
Thank you very much! Will you test polar venture m2, v2, ignite 3?
Rob, you are the best 🎉
Great video, the question is if the health app is reliable? 17:22
Hello Rob, a quick question. How much can we trust the amount of KCals detected by our wearables after different types of workout/exercises?
I'm asking about my GT3 as an example!
There is something similar in the huawei too. If you start the exercise from the app it is calculating closer distance to the treadmill
Did you also check the delay for other apple watches/other trackers?
5 seconds! Thanks for clearing up what I thought was a delay in my heart response. (Turns out my heart isn’t asleep!). Awesome science explained perfectly for non-science people like me. Very interesting.
Ah, I just loved the scientist nerdery in this one. I suggest also accounting for your novelty bias while interpreting the results. You at least appear to be very keen on documenting this "problem", therefore increasing the likelihood of you observing just that. Your results apart from this being sound and verified, I am really amazed at Apple's post-recording algorithms that appear to be at work here. After all, they manage to post-hoc match the HR to the timeframe it belongs to perfectly. This video finally piqued my curiosity beyond the activation potential. I will get an Apple watch Ultra to cross-verify all these interesting results. And because the watch really looks awesome ;-). Cheers from Hannover, Germany.
wow next level of insights that we really need
Thanks so much Milan!
Could the delay be explained by the AW screen refresh rate? Increasing the refresh rate would probably fix it but impact battery life negatively.
Great review. When will we see the Amazfit gtr 4 review.
Can you please compare the ekg functions of smart watches and which one is the most accurate?
That illustration of a use case for cross-correlation was better than any I experienced in a few grad. stats courses.
These companies should hire you! Very detailed video.
Does this delay in screen updates for heart rate only apply to the Ultra, or is it the same for all Apple Watches?
In many reviews where the apple watch and h10 polar have been compared, there have always been clear differences. I've always wondered why that doesn't match your tests and why the apple watch does perform so poorly. Thanks for the clarification
n=1.
11:10 kudos for including this in the video, nice to see everyone makes these kinds of mistakes sometimes
Podrías hacer un análisis del garmin 955.. También podrías incluir en tu prueba la medición de VO2 max en algunos relojes que lo permiten
Few seconds of lag is something that is typical for PPG sensors. There is a lot of post processing going on to filter out noise. From what I see here AW just corrects time stamps in the file so the post-processing lag is removed for post-excercise analysis. Displaying it without lag with high accuracy is not possible. From my experience it is not an issue when excersising. HR is not a metric that provides instant feedback after all.
Whaou! Great job!
I fear this exercise may be useful in your future tests!
Running following closely my BPM, I know 3 to 5 secs may be a issue, not necessarily for the view but for the correction you need to bring. At the end you spend your time making a yo-yo between going faster and slower.
Btw, you already made me buy a Huawei fit, recently a GT runner and now a polar H10!!
I bought the 2 last ones because of their compatibility: you can connect the polar H10 on 2 devices: so one to my pixel 7, one on the GT runner. Probably the best accuracy to get. Time to time I make the comparaisons disconnecting the H10 on the GT runner.
Does the Apple Watch Series 8 have the same delay?
Well... First of all, congrats on the HUGE work, here! Second, very enjoyable video, and great sense of humor. Now one little consideration... Ok for the delay, but it looks to me that the app data are also "smoothened" a bit, which makes sense, they probably process them for removing some of the noise. And probably a first rough smoothing is on the watch itself, thus the on screen delay. This being said... Still impressive accuracy by this watch. Being an Android user that does not like to charge the watch often... I stick to my Fenix, however... Something to think about for the other device makers
how did you like your Fenix ??
A wait of 2 weeks and a review of a watch i cant afford.🙁
Lv ur videos, Love ur research, love ur content
Glad you like them! And yeah, it is very expensive😅
Is it possible that the delay is caused by Apple conducting additional processing on the raw data and put the results into the Health app?
I don't know for sure if it would work because I haven't tested it but there is an accessibility feature where you can control your apple watch from your iPhone and it shows the screen. You could then do a screen recording on the iPhone to have a live video of the Apple Watch. Let me know if that works.
I was doing an endurance run and could feel the precise moment my HR went into my tempo zone. From the moment I felt the tempo heart beat, had the thought “oh I’m in tempo”, turned my wrist over, I could then watch it tick over that 1bpm into my tempo zone haha so yeah, there’s a lag, but nothing so significant that it ruined my workouts. I suggest everyone get so good at feeling your HR zones that a few second delay doesn’t derail your workout.
Haha love the thoroughness of your videos.
Something I've noticed, anecdotally, is that my heartrate at my wrist is delayed from the heartrate at my chest. That may introduce a variable in heart rate accuracy when evaluating wearables vs chest straps.
The recorded heartrate on Apple watch was in correct sync. with chest strap...only live heartrate being displayed was delayed 4-5 seconds! 😉
Have you considered reviewing scales, especially ones that claim to measure lean/fat mass and other comp metrics.. maybe compare them to a dexa scan?
Hi, can sleep tracker improve my sleep ? I'm ready to buy apple watch but I don't want to buy and nothing improved... Can you help me?
Do you plan to test Huawei GT 3SE?
I've bene having the same issue with my Series 7 ever since the update to Watch OS 9. Sadly Re-pairing and resetting the watch and recalibrating the watch hasn't fixed this issue.
My main problem is > Open Fitness app > Under the workouts section, Tap 'Show More' > then Tap on your latest workout. For me it's outdoor walk, when I scroll down to the heart rate section it shows as missing at the very beginning of the workout.. the heart rate dots are more like a flat line ____. and then it shows the dotted heart rate throughout my workout.
Similar thing happens to the actual workout app where it takes at least 10 seconds or longer to register my beginning heart rate. and when I sometimes look at my current heart rate on a walk it processes it kinda like a loading screen
Very interesting! Thankfully, for most of us, there's no practical impact. Looks like a mind-numbing bit of work, congrats and thanks.
I don’t even know what to say. You really outdid yourself. This could be my least personally to me useful, but favorite video of yours ever. I laughed several times. This was amazing. Bravo! - I was thinking, perhaps this is Apple excelling here. Maybe they are using a algorithm over a larger segment (5 seconds?) to more accurately record the overall rate, instead of just taking the average up to the current time. Maybe this is good?
Glad you enjoyed it! And thanks so much for the feedback. Hmmm, not sure actually, since they report the exact same heart rate later in the app with the correct timing (so shifted back), but it is always good to go for reliability
Have you happened to test the delay/lag on other Apple Watches? Thank you for this interesting investigation!
did you also check the heart rate recorded in the app once you export it? as it has create date, start date and end date. The create date may be the one showing on the display as it is a delayed recorded data. I am also curious to know why I do not get heart rate continuosly as I was wearing the watch and just activated the heart rate to measure my pulse and I expected it to continue recording. Does it stop recording if I am not moving? Thanks
Oh man, I wish you'd do some in-depth review of the Galaxy 6/classic 6. I'm a Galaxy user (mobiles), and I'm not buying any watch, not until I see your review and literally been waiting for 2 years for the Galaxy.
Good job, my friend.
Will you review the amazfit band 7?
Hey man can you at some point make an update on the Fitbit because there might have been updates that improved the sleep aglorythm
The dream 2 isn't on the market anymore. Can you please do the muse S sleep tracking accuracy?
Thank You
You're welcome😊
I've noticed this lag in displaying heart rate on other Apple watches as well. So I don't think it's only an Ultra issue.
Where would me the world know without scientist. Keep doing where are trying to understand and make it our decision. Than you
My V02 max scores are way off with the ultra compared to my Apple Watch Series 3. I've tried resetting it and it doesn't work. It's very weird. Other than that, I love the watch.
Hey, are you going to review the new Xiaomi's Redmi Smart Band 2?
best youtuber ever keep it up.
i have fenix 6 it has same delay its normal for weariest heart rate tracking
Why isn’t this just a delay from measurement to on screen display? It’s measuring correct I guess, but due to computational resources it lags to write it on the screen?
When dealing with noisy data in live, ie. you don't know the future values, it's common to run a exponentially weighted moving average in order to estimate the current value. If they go with a low smoothing parameter, that introduces a delay in the response. Later, once they have all the data over the recording time, it's much easier to smooth a line on top of all the data points.
How fast is the watch’s CPU?
I wonder if this issue is also on the Series 8
Wow, so much attention to detail to bring us accurate data, the elevator music made me laugh, I’m guessing all watches will have some lag, I enjoyed watching as always and like you showing your mistake and shows we are all human lol, look forward to more future content 🥳🥳🥳🥳
I would be more than happy if you could review the Garmin Instinct Crossover 😊😊
cool stuff
I can confirm the delay as well. I noticed it as well.
I love your Reviews!!!
Is it possible for you to do Amazfit GTR 4 review?
I know that past Amazfit watches haven't shown any great results especially GTR 2, but many of them are really expecting to purchase GTR 4 for gorgeous design and affordable price.
It will be great to know if GTR 4 is any better to buy , not only in terms of looks but also in terms of data accuracy 😊
Thanks Alot!!!
He said in a recent comment that he's already testing the GTR4...
I think you just have to wait a little more.
I couldn't since my OG Bip died and went on and bought it (was about to buy the huawei gt runner, but found a good deal on a gtr4 racetrack grey, plus it was too good looking :D )
So far I'm really happy for my use, especially for the battery time, gps acc and other features.
Still I'm looking forward for his review on hr & sleep, for me he's the authority on that
please do a review of mi band 7 pro
I cannot understand apple is showing delayed HR instead of more correct healthApp HR
and why there is no comparision between displayed HR and healthapp HR and polar HR?
This clip sounds really interesting but only makes me more curious
Fabulous!
I wonder, is this going to be fixed or can it be fixed??
This guy is on another whole level!