Validation of Wearable Devices to Measure Energy Consumption
Article information
Abstract
OBJECTIVES
The purpose of this study is to verify how accurately wearable devices measure energy expenditure while walking outdoors.
METHODS
A total of 20 people, 10 healthy males and 10 females, participated in the study. After approval by the Institutional Review Board of Pusan National University, the experiment was conducted with the consent of the subject. All subjects wore four wearable devices (Fitbit Surge, Android Phone, iPhone, pedometer) and a portable gas analyzer simultaneously and walked on flat, downhill and uphill road respectively. All subjects repeated these experiments on each slope three times. The validity was verified through correlation analyses and paired <i>t</i>-test between the energy expenditure measured by the wearable devices and by a portable gas analyzer.
RESULTS
Under all three road slopes (flat, downhill, uphill road), the energy expenditure as measured by iPhone, Android phones, and pedometer significantly correlated with the portable gas analyzer. However, all three devices were significantly overestimated or underestimated the energy expenditure as compared to the gas analyzer under all three road slopes. Fitbit Surge did not correlate with the gas analyzer for measuring energy expenditure under any conditions, and significantly overestimated energy expenditure.
CONCLUSION
The validity of energy expenditure measurement during outdoor activities using wearable devices is still low, and more valid motion detection sensors and algorithms need to be developed.
Introduction
It is well known that increased physical activity improves the quality of life of modern people through disease prevention and health promotion. Due to this trend, many people participate in various physical activities, but in order to maximize the effectiveness of exercise, a safe and efficient exercise program suitable for individual characteristics and fitness is required. To this end, monitoring of energy consumption, which indicates the exact amount of physical activity, is necessary. Recently, many wearable devices have been developed to measure energy consumption during exercise. For example, various types of pedometers, smartphone applications, Fitbit products are widely used.
According to the latest forecast from International Data Corporation’s (IDC) Worldwide Quarterly Wearable Device Tracker, the global market for wearables grew up 71.4% from 178 million units in 2018 to 305.2 million units in 2019. From there, total volumes will grow to 489.1 million units in 2023, resulting in a compound annual growth rate (CAGR) of 22.4% [1]. Wearable devices are used not only by professional athletes, but also by people who want to exercise for health. In addition, it is expanding to various medical services such as providing feedback through monitoring physical activity of patients using wearable devices [2,3,4], and rapidly responding to emergencies by monitoring 24-hour physical activity of high-risk patients [5,6].
Accordingly, studies to verify the feasibility of measuring energy expenditure of wearable devices have been actively conducted [7,8,9]. Most of them, however, have been done indoors using treadmills or bicycle ergometers. Some studies have been conducted in outdoor environments, but all in flat road [10,11]. In particular, the effectiveness of measuring energy expenditure of wearable devices on slopes (uphill or downhill) is not yet known. Considering that most of wearable devices estimate energy expenditure using an algorithm, it is important to test if these devices are reliable in different road slopes such as uphill or downhill.
Therefore, this study validated energy expenditure measurements of four wearable devices by comparing to a portable gas analyzer during walking on flat, uphill and downhill road.
Methods
Subjects
Twenty healthy men and women in their 20s (10 for each gender) participated in this study. This study was approved by the Institutional Review Board (IRB) of Pusan National University (PNU IRB/2015_33_HR), and conducted with the consent of the subjects. All subjects checked their health status with PAR-Q & YOU physical activity readiness questionnaire before experiment for their safety.
Procedure
The subjects wore four wearable devices (Fitbit Surge, pedometer, Android Phone, and iPhone) and a portable gas analyzer simultaneously and walked on flat, downhill and uphill road <Figure 1>. All subjects restricted excessive exercise 2-3 days before the experiment and maintained the same diet and physical activity 24 hours before the experiment. The energy expenditure measured by the wearable devices were validated against the energy expenditure measured by the portable gas analyzer.
Example of wearing four wearable devices (Fitbit Surge, Android Phone, iPhone, pedometer) and a portable gas analyzer simultaneously.
Measurements
Subjects walked on flat, downhill, and uphill road in random order. Each walking test was performed after the heart rate had returned to a completely stable state. All subjects underwent the same experiments three times over an average of 2-3 days. The distance of flat road was 800 m and the downhill and uphill road were 530 m. The slope of the downhill and uphill road was 5-18% with an average of 14 ± 1%. The walking speed was determined by the subjects’ preferred speed.
Each subjects wore four wearable devices. Firstly, Fitbit Surge (Fitbit, San Francisco, USA) was worn on the wrist of the left arm. It includes an optical heart rate monitor, a 3-axis acceleration sensor, an altitude sensor, a vibration motor, and GPS. Energy expenditure is automatically calculated by device application based on the subject’s height, weight, age and gender. Secondly, a pedometer (YGH667, Haptime, China) was worn on the waist. It counts the number of walking steps by sensing vibration, and calculates energy expenditure and walking distance based on the subject’s height and weight. Thirdly, Android Phone (Galaxy S6, Samsung Inc., South Korea) was worn on the left upper arm. It contains GPS, gyro sensor, accelerometer, geomagnetic sensor, barometer sensor and heart rate sensor. S Health, a built-in application, provides energy expenditure and walking distance based on the subject’s height, weight, age, and gender. Lastly, iPhone (iPhone 6, Apple Inc., USA) was worn on the right upper arm. It contains GPS, gyro sensor, accelerometer, and barometer sensor. Run keeper, a built-in application, provides energy expenditure and walking distance based on the subject’s height, weight, age, and gender.
A portable gas analyzer (K4b2, COSMED Inc., Italy) was used to measure accurate energy expenditure during walking. Subjects wore masks connected to portable gas analyzer on their backs while wearing four devices.
Statistical Analysis
In this study, the mean (M) and standard deviation (SD) were calculated using SPSS 21.0.
Pearson’s correlation coefficient and paired t-test were performed to verify the validity of measuring energy expenditure by wearable devices. The significance level (α) for all statistics was set at p <0.05.
Results
The characteristics of subjects are shown in Table 1. All subjects completed the experiment without dropout.
Subject characteristics
The results of correlation analyses between portable gas analyzer and wearable devices for energy expenditure of walking on three different road slopes are shown in Table 2. Galaxy S6, iPhone 6, and pedometer showed significant correlation with the gas analyzer for measuring energy expenditure in all three road slopes, flat, downhill, and uphill road. Among these three wearable devices, iPhone 6 showed the highest correlation followed by Galaxy S6 and pedometer. However, Fitbit Surge did not show any significant correlation with the gas analyzer for measuring energy expenditure in all three road slopes.
Pearson’s correlation coefficient between gas analyzer and wearable devices for energy expenditure of walking on three different road slopes
Energy expenditure of walking on flat, downhill, and uphill road measured by the gas analyzer and wearable devices are shown in <Table 3>. Fitbit Surge significantly overestimated energy expenditure of walking on flat (33.1%) and downhill (55.6%) road compared to the gas analyzer. However, there was no significant difference in energy expenditure of walking on uphill road measured by these two devices. Galaxy s6 significantly underestimated energy expenditure of walking on all three road slopes, flat (26.8%), downhill (12.3%), and uphill (59.1%) road. iPhone 6 significantly underestimated energy expenditure of walking on flat (15.5%) and uphill (55.5%) road, but significantly overestimated energy expenditure of walking on downhill road (25.9%) compared to the gas analyzer. Similarly, pedometer also significantly underestimated energy expenditure of walking on flat (21.3%) and uphill (58.8%) road, but significantly overestimated energy expenditure of walking on downhill road (11.4%) compared to the gas analyzer.
Energy expenditure of walking on three different road slopes measured by gas analyzer and wearable devices (kcal)
Discussion
The purpose of this study was to verify the validity of the wearable device for measuring energy expenditure in the outdoor condition. In particular, we included both men and women for subjects, tested on three different road slopes (flat, downhill, and uphill), and repeated the same experiment three times to increase reliability.
The results of this study showed a significant positive correlation between energy expenditure of walking on flat, downhill, and uphill road measured by Galaxy s6, iPhone 6, and pedometer and a portable gas analyzer. However, all three of these devices significantly overestimated or underestimated the energy expenditure of walking in all three road slopes compared to the gas analyzer. This suggests that Galaxy s6, iPhone 6, and pedometer is able to reflect energy expenditure of outdoor physical activity on various road slopes, but is not yet able to accurately measure or estimate them. Fitbit, on the other hand, did not correlated with the gas analyzer for energy expenditure in all three road slopes and overestimated for energy expenditure in flat and downhill roads, suggesting that Fitbit could not reflect or measure energy expenditure of outdoor physical activity.
As far as we know, there are no studies validating smartphones for measuring energy expenditure compared to a gas analyzer, especially during walking outdoors. Smartphones are not medical or research equipment, but the frequency of their use in research is increasing rapidly in recent years. Therefore, it is important to investigate whether it is accurate enough for use in research. Galaxy s6 and iPhone 6 contain similar sensors such as GPS, gyro sensor, accelerometer, and barometer sensor. These devices are able to provide more data during physical activity via these sensors such as movement of speed, distance, time, altitude, and etc. In fact, both Galaxy s6 and iPhone 6 showed relatively strong correlation with gas analyzer for measuring energy expenditure. However, the Galaxy s6 and iPhone 6 use their own algorithms to calculate energy expenditure, so each gives different values. In this study, Galaxy s6 underestimated energy expenditure for all three road slopes, but iPhone 6 underestimated energy expenditure for flat and uphill roads and overestimated energy expenditure for downhill road. This discrepancy may be due to their different algorithms. Nevertheless, both devices underestimated energy expenditures of uphill walking more than 50%. It could be explained by the fact that most of wearable devices underestimate energy expenditure with increasing workload [10,11]. Thus their algorithms need to be made up to increase accuracy to measure energy expenditure.
Pedometer contains relatively simple vibration sensor as compared to other wearable devices. Nevertheless, it had a positive correlation with a gas analyzer for measuring energy expenditure in this study. Although it underestimated energy expenditure for flat and uphill roads and overestimated energy expenditure for downhill road, it showed the possibility of being useful. Pedometer is the simplest wearable device, hence developing a better algorithm can be widely used to monitor energy expenditure in research.
Unlike other devices, there is little research on Fitbit products. Diaz at al. [12] reported that Fitbit One and Fitbit flex are reliable device for measuring step counts and energy expenditure. On the other hand, other studies demonstrated that Fitbit Surge accurately measures heart rate but not energy expenditure [7,8,9]. Considering that these studies were conducted in laboratory, it is not surprising that Fitbit Surge overestimated energy expenditures in flat, downhill, and uphill walking, which were not correlated with a gas analyzer in this study. Since Fitbit Surge contains an optical sensor to accurately measure heart rate, these results might be due to its algorithm. As other devices, it needs to be made up to increase validity to measure energy expenditure.
Conclusions
The most popular wearable devices, such as Android phones, iPhones, Fitbit Surges, and pedometers, do not accurately measure energy expenditure outdoors on flat, downhill and uphill walks.
Acknowledgements
This work was supported by a 2-Year Research Grant of Pusan National University
Notes
The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.