Last October, PriceWaterhouseCooper’s health research institute released a study that said one in five Americans owned a wearable device — about the same percentage of people that owned tablets back in 2012. But just how accurate are these devices when it comes to gauging how much we exercise? Mitesh Patel, an assistant professor of medicine and health care management at the University of Pennsylvania’s Perelman School, has done some of the first research into just how effective step-counting devices really are.
In an interview with Knowledge at Wharton, Patel discussed what his research revealed about these remarkable bits of mobile technology, and how well their technical strengths stack up against our resistance to exercising.
An edited transcript appears below.
Knowledge at Wharton: You have two papers out — one that was a study, and one that was a follow-up. Let’s start with the study: What did it entail, and what did the data really show?
Mitesh Patel: We were interested in the question of whether or not wearable devices were accurate for tracking step counts. There are hundreds of wearable devices out there on the market. Most of them track how many steps you walk. A lot of them do other things, like tracking heart rate or sleep patterns and so on. But there’s been no evidence to determine whether or not these devices accurately track the behavior that they say they track. We wanted to start with step counts, because it’s one of the most popular things that these wearables do. And it’s something that most people do every day and throughout the day.
So we took a variety of participants and asked them to come in and walk on a treadmill. We asked them to walk two distances: a shorter distance of 500 steps and a longer distance of 1,500 steps. We had somebody there with a tracker manually counting how many steps they walked.
Then, we loaded them up with a bunch of devices. We had them wear a standard pedometer, which is the gold standard for research, on their hip. It’s been found to be very accurate. We also put a Fitbit One and Fitbit Zip on their hips. Those are accelerometers. They use slightly a different technology than pedometers. And then we had them wear three wearable devices, a Fitbit Flex, a Jawbone Up, and a Nike FuelBand. In addition, we had them carry a smart phone, an iPhone in their right pocket, running several tracking apps, and then an Android in their left front pocket. And what we wanted is to see how they compare to the actual number.
Knowledge at Wharton: Were the devices fairly reliable in terms of tracking steps accurately?
“A lot of people think that if we just give someone a wearable so they can track their steps, they’ll get more engaged, they’re going to improve their health. That is actually not true.”
Patel: For the most part, we found that the pedometer and accelerometers on the hip were very reliable, very accurate, and also, very precise. The wearable devices and smartphone apps were all fairly accurate, as well, with the exception of one wearable device, which was the Nike FuelBand, which was about 2% below, and has actually been [taken] off the market since then….
But what was interesting was we actually found that the smartphones had less variability than the wearable devices. That’s interesting for a couple of reasons. Sixty-five percent of people have a smartphone and carry it with them everywhere that they go. It’s one of the few devices that, if you leave home, you’re going to go back and turn around and get. People can leave their wallet, their keys. But if they forget their smartphone, it has their calendar and their contacts and their e-mail. You can’t live without it.
We actually think that the estimate of the proportion of people that carry wearable devices is much lower, probably close to 1% to 2%.
A lot of the industry surveys tend to survey people who are younger or people who are online, and so on, and don’t capture the people who actually have more chronic conditions or are in the hospital or are older in age.
Knowledge at Wharton: I just received a Fitbit from my kids for Father’s Day, and I’ve been using it for the last several days. I feel, at times, like it’s not accurate. Obviously, from your data and your research, it probably is. But there is a perception at times that people may have, wondering if it truly is accurate.
Patel: Yes, that highlights a couple of important factors. Our study was done in a very controlled setting on a treadmill. People were walking. They weren’t doing anything else. When you’re using these in the real world, the motions that you do for everyday things such as picking things up might be similar to what the devices capture as a walking step. So there will be some variability. I think the important thing to take into account is that the average person walks about 5,000 steps. If you want to know whether or not you got enough physical activity, that typically tends to be at least 7,000 steps, but closer to 10,000 if you want to be very active. These devices are fairly good for telling you, “Yes, I got enough physical activity today,” or, “No, I really didn’t.” And a lot of people tend to overestimate how much activity they get. So they can be very revealing and eye-opening at the beginning. But I certainly agree that, as you use them throughout everyday practice, there is going to be some variability.
Knowledge at Wharton: With the number of smartphone apps that are out there that do this, people don’t have to go out and spend, say, $90 on a Fitbit. There are options through your smartphone that may be free or may cost you $3.99, $4.99, whatever it might be. So there’s a financial aspect to this as well.
Patel: Yes. One of the challenges with getting more people to use wearable devices — specifically the people that need them most — is that these wearable devices can be quite expensive. Some of them are $100, $150. Some of the newer ones, the Apple watch and so on, are $350 or higher. The other challenge I think smartphones help with is that most people just are not as engaged in their health as they could be. Certainly, financial limitations are one aspect of that. But if you’re carrying a smartphone, it’s important to know that you can start tracking your steps and engaging in your health within a couple of minutes, and for free. In fact, the new iPhones track your step counts by default. You actually can’t stop them from tracking your steps. You can turn the display off. But Apple was very clever in terms of taking advantage of that default and using the devices. So essentially, every iPhone now is an accelerometer that tracks your steps.
Knowledge at Wharton: As you said, the study that you did was in a controlled area. Is the next step then to expand it into other realms, different settings?
“These devices alone often are not enough to actually take people who have low motivations and get them to change their behavior.”
Patel: I think certainly, as more of these devices come out, there needs to be more rigorous evaluation of their accuracy — because in order for individuals to trust the data and to trust this information, the information’s got to be accurate. But what I think is an even bigger challenge — and what I and my collaborators are focusing more of our time on — is [motivation]. Just giving somebody a wearable device is not alone going to make them more physically active. A lot of people think that if we just give someone a wearable so they can track their steps, they’ll get more engaged; they’re going to improve their health. That is actually not true, especially for the people who are less motivated to start with. If you take someone who’s a “quantified selfer,” who runs a lot, they may get more benefit out of just having the device. But the people who are less motivated probably need other kinds of help.
Knowledge at Wharton: I think we need to make a distinction there, in that there’s a difference between having the motivation to just get up and walk, and the motivation to live a healthier lifestyle. Because that includes your sleeping patterns, what you eat. There’s a lot more factors involved in that than just having a Fitbit and walking more.
Patel: Yes, certainly. I think that improving your health requires that you’re motivated to change multiple aspects of your daily behaviors, including how much you walk, what you eat, if you’re on a medicine, whether or not you take that medicine, and so on.
Knowledge at Wharton: Once people own these devices, how much of an increase do we see in the number who actually will use them for exercise long term? How many use them for a little while and then just give up on them? That long-term question about the desire to use them going forward seems as important as anything.
Patel: Yes, I completely agree. I think one of the most important points is that, as these technologies continue to evolve, there will become better ways to monitor our physical activity or other health behaviors. But these devices alone often are not enough to actually take people who have low motivations and get them to change their behavior. Oftentimes, we need to pair these wearable devices or smartphones or whatever you’re using, with another engagement strategy. That can be something like a workplace wellness program, where you’ve got a competition going on, or there may be financial rewards up for grabs if you’re able to become more physically active. Or it could be a social incentive. We find that people who do things with their spouses or their partners or their friends and family tend to stick with it more than people who do it by themselves.
Knowledge at Wharton: That’s an interesting thing with the Fitbit. You have that ability to contact your friends through Facebook or e-mail contacts. That challenge aspect of it is, obviously, a very important part, because it will also track the friends you are tracking, or who are tracking you. There is a little bit of motivation in that competitive realm.
“We find that people who do things with their spouses or their partners or their friends and family tend to stick with it more than people that do it by themselves.”
Patel: Yes. Each of us has an underlying competitive nature. Things like peer support, accountability to other people on the team — those things really help drive us for what we do in everyday life. You want to fit in within the social norms. Social norms have been found to be very effective. The challenge is trying to figure out how to construct it in a way that can help you without discouraging you. A lot of these apps or competitions use leaderboards, where they’ll show who is at the top of the leaderboard. That tends to be very effective for the couple of people at the top, but not for the other 99% of people that are at the bottom, or not in the top five. So one of the things we often recommend in the workplace, or wellness setting, or a larger setting, is that rather than having competitions, set a minimum threshold — let’s say, something like 7,000 steps — which is approximately equivalent to what the CDC recommends for the minimum level of physical activity to get health benefits. And set the goal to be, instead of everybody walk as much as possible and we reward the top two or three, let’s see what percent of the work place or what percent of your group can meet that minimum threshold and get their baseline of activity. That tends to encourage everyone and really leverages team incentives, as opposed to making it directly an individual competition, which can be discouraging to a lot of people.
Knowledge at Wharton: It would also be interesting to find out if there’s an increase in the number of people who are out walking around at lunchtime because they have a wearable device, and they want to use that half hour or 45 minutes to get that exercise in.
Patel: One of the most interesting opportunities with wearable devices, as their adoption grows, is that this now becomes a valuable data source for us. We can actually use this to try to predict if there are certain things that get more people excited to exercise: it may be the weather or may be the time of the year. Something studied by other folks here at Wharton is called the “fresh start effect,” where on a Monday, beginning of the week, or right after a holiday or New Year’s resolution, people get more excited. It often wears off, but if we can try to leverage those different things through all of this data, that might create new opportunities to help people get motivated.
Knowledge at Wharton: About 10 minutes ago, I just happened to bring up my Fitbit app. And I looked at the number of steps that I had collected already for the day. Well, maybe three minutes ago I looked at it again. There were 15 more steps added to my total count for the day, and I’ve been sitting here talking to you during the whole time. Now, have I moved my feet? Yes. Have I moved my arms? Yes. But realistically, I haven’t done anything. So, although it’s obviously very good technology, I guess there’s still an area to this that needs to be perfected.
Patel: Yes. I think as time goes on, the technology is going to get better and more affordable, and we’re going to be able to track things more accurately. I do think the more important thing, though, is how do we use the information, even as it is now? It will get better, but how do we use it now to help people actually improve their health? So whether knowing the exact number of steps is the right thing I think is one component. But I really think that motivation is probably the key point. Knowing the directionality is going to play a bigger role than a 15-step difference in step counts.
