At Eight Sleep, we strive to provide our members with not only an optimal sleep experience, but also highly accurate biometric data during sleep. Our sensing technology, embedded in the Pod’s Active Grid Cover, is designed to monitor a variety of biometrics without any wearable and for up to two people on the same bed. This design presents a set of technical challenges, but it is also a better experience for our members who don’t have to wear a device to bed in order to monitor their sleep and health.
Today, we are launching a key update to our heart rate variability (HRV) algorithm.
The performance of both the heart rate (HR) and HRV algorithms were validated against gold standard electrocardiogram (ECG) on more than 474 nights of data from 163 unique subjects, comprising over 208,600 minutes of sleep.
The Pod’s Sleeping Heart Rate Tracking is 99% Accurate
Key Statistics:
Eight Sleep’s sleeping HR algorithm has a mean absolute error of less than one beat per minute (bpm), a mean absolute percent error of 1.5%, and mean error of less than 0.1 bpm.
One of the best ways to visualize this excellent accuracy is to look at the correlation value (i.e. correlation coefficient) between the ECG HR and the Pod HR. A correlation value of zero means there is no relationship between the two devices, and a correlation value of 1.0 is the highest correlation value and means that the HR values from the Pod perfectly match that of the ECG.
The correlation value between the Pod HR and ECG HR is 0.996 (r2 = 0.99) – a near perfect match (Fig. 1).
Figure 1. Correlation between the Eight Sleep Pod HR and the gold standard ECG HR (r2 = 0.99). Each dot represents a single night of sleep (n=133). The black dashed line represents a correlation value of 1.0 or the best-fit line (1:1 match between Pod and ECG HR).
Sleeping Heart Rate Variability Highly Strongly with Gold Standard ECG
About HRV:
There are several ways to calculate HRV. At Eight Sleep we present your HRV values as the Root Mean Square of Successive Differences (RMSSD) between heartbeats. HRV gives you a measurement of the variability in milliseconds (ms) of your consecutive heartbeat intervals. Generally, a higher HRV is considered healthier, and changes in your HRV can indicate how well-rested you are, both physically and mentally. Note that there is a wide range in “normal HRV,” so we recommend looking at your own HRV trends across weeks and months to evaluate changes in your sleep and recovery.
Key Statistics:
Eight Sleep’s new HRV algorithm has a mean error of 0.782 ms, a mean absolute error of 5.7 ms, and correlation value of r = 0.953 (r2 = 0.91) compared to gold standard ECG (see Fig. 2).
Figure 2. Correlation between the Eight Sleep Pod HRV and the gold standard ECG HRV (r2 = 0.91) for Pod 3 data. Each dot represents a single night of sleep (n=133). The black dashed line represents a correlation value of 1.0 or the best-fit line (1:1 match between Pod and ECG HRV).
The importance of monitoring trends in your HRV
HRV varies greatly based on age, biological sex, and genetics. For example, one study found that 47-64% of your HRV is genetic. Still, ~50% of your HRV is modifiable through sleep and lifestyle factors. That said, it is best to only compare your HRV values to yourself as HRV varies widely from person to person.
The best way to use your daily HRV value is to look at nightly changes in your HRV in response to certain behaviors (like drinking alcohol or exercising), while also monitoring changes in your HRV across time. Monitoring trends in your HRV across time can give you a good indicator of whether your cardiovascular health is changing based on lifestyle changes like exercise, diet, and sleep. If your HRV dips below your normal baseline, it might be an indication that you might be getting sick, or needing more rest to recover from a hard workout, or that you are stressed. Alternatively, an increase in HRV could indicate positive lifestyle changes, like getting good quality sleep, eating healthier, and staying hydrated. The key here is not to focus on absolute differences in HRV values across various monitoring devices, but instead to look at general trends in your HRV over time to better understand how your behaviors are impacting your cardiovascular and nervous system health.
How to interpret HRV across various devices
Each brand calculates their HRV differently. For example, Apple calculates HRV using the formula SDNN, while we use RMSSD (similar to Oura, WHOOP, and Fitbit). Therefore, you will get different HRV values if comparing your Apple Watch’s HRV to Eight Sleep’s HRV. That said, you should still see similar trends in your HRV across all devices if you drink alcohol, are stressed, or don’t recover well from a workout.
Additionally, each of these companies has different methodologies for giving you a single nightly value. For example, one company might take the average of all HRV values, while another might use the median. Thus, you may see some variation in your HRV if you use multiple products that track your HRV. Another factor to consider is the time of day that HRV is measured. Some brands measure morning HRV, which can be very different from your nighttime HRV due to the circadian rhythm of your heart. These differences in methodology across brands are why Eight Sleep compares our Pod HRV to clinical gold standard ECG.
Appendix: Study Details
Total dataset: 474 nights from 163 unique subjects.
Methods: Participants slept on the Pod for 1-4 nights while wearing an ECG device on their chest (VitalConnect RTM or Polar H10 ECG). These ECG devices were used as the gold standard (i.e. ground truth) for sleeping HR and HRV to compare to the Pod’s sleeping HR and HRV. Participants self-selected their Pod temperature settings.
Statistical Analysis: After the study was complete, each participant’s ECG data were compared to their Pod HR and HRV data for each night of sleep.
HR analysis: The Pod calculates HR each minute, and then a 5 min median of Pod HR and ECG HR were compared across each night. With these 5 min values throughout the night, the following metrics were calculated: mean absolute error (the absolute difference between the Pod and ECG HR), mean error (the difference between the Pod and ECG HR), and mean absolute percent error (the mean absolute error divided by the ECG HR for that minute). The correlation value (Pearson r correlation coefficient) was calculated based on the Pod’s single nightly HR value displayed in the app vs. the ECG nightly value.
HRV analysis: Eight Sleep reports a nightly HRV value (RMSSD). This nightly value was compared to the nightly ECG HRV value, where RMSSD was calculated for both devices every minute throughout the night and then aggregated at the five-minute and 15-minute levels. Lastly, all 15-minute values were used to generate a single nightly HRV value. With these nightly values, we calculated the mean error, mean absolute error, and Pearson r correlation coefficient.
By Dr. Nicole Moyen, Brian Schiffer, and Dr. Dave Heinz
