Some researchers are complaining about the inability to use the Apple Watch for studies, pointing out inconsistencies in the data obtained over time with the device. The Verge reports that a Harvard biostatistician is reconsidering the possibility of using the Apple Watch in a research study after finding inconsistencies in the heart rate variability data collected by the device. The problem seems to have to do with Apple frequently changing the algorithms leveraged internally on the device, leading to problems for those who would need certain data for certain studies due to the inability to rely on an algorithm that can guarantee consistently correct data over time
“These algorithms are what we call black boxes, they are not transparent and it is impossible to understand what is inside,” says JP Onnela, associate professor of biostatistics at Harvard T.H. Chan School of Public Health and developer of the open source platform Beiwe, exploited in other fields for the passive acquisition of GPS and accelerometric data derived from the normal use of the smartphone and allow the collection of information relating to clinical status and quality of life.
The inconsistencies in the heart rate variability data are, as noted above, caused by Apple’s continual changes to its algorithms, preventing third parties from pursuing studies (unless Apple specifically states the methodology used). Researchers at the Brigham and Women’s Hospital (BWH) in Boston exported heart rate data read by the Apple Watch for the same period of time but several months apart; this data should have been identical but being filtered by a proprietary algorithm before exporting, it turns out to be extremely different.
Apple, according to the researcher, often changes the algorithm used to determine the heart rate variability but also other algorithms seem to have been changed over the months, including those related to sleep monitoring.
This problem, as you can imagine, prevents the use of these devices for some research, forcing the use of specialized devices.Although this is not a problem for some studies, it severely limits researchers, who are forced to find other products that users do not always have available. The researchers are asking Apple to make known the changes made with the various updates to the algorithms, or to somehow make the raw (raw) data readable by those doing the research.