In the flurry of activity leading up to and lasting through the first enrollment period of the Affordable Care Act, it was a minor miracle in most instances to get websites functioning that would help consumers choose a health insurance plan either from a state-run exchange or the federal website, But two years later, the sites are starting to roll out more robust tools for helping consumers make smart decisions on their insurance picks — although not everyone has the same toolbox at their disposal.

That’s according to recent research by five University of Pennsylvania researchers, led by Charlene Wong, a pediatrician and health services researcher in the Division of Adolescent Medicine at Penn and the Children’s Hospital of Philadelphia. In “For Third Enrollment Period, Marketplaces Expand Decision Support Tools To Assist Consumers,” published recently in Health Affairs, Wong and her colleagues, who include Wharton health care management professors Daniel Polsky and Robert Town, examined the federal exchange that was used by 38 states and the state-based marketplaces used by 12 states and the District of Columbia to compare and contrast the tools that were offered to consumers in the third ACA enrollment period, which began November 1, 2015.

One of most notable new features that were available on more sites were total cost estimators that told consumers how much a plan would cost based not only for monthly premiums but also out-of-pocket expenses for medical care. Other common additions to marketplace sites included integrated provider and drug lookups, quality ratings (which are mandated to be part of the sites starting this year) and pop-up definitions of insurance terms.

“We were encouraged that more exchanges were implementing additional support tools since choosing a health insurance plan is such a complex task,” says Wong, who is also a fellow at Penn’s Leonard Davis Institute of Health Economics.

“We found very positively that more exchanges were implementing additional support tools.”

Window Shopping vs. ‘Real’ Shopping

Even for marketplaces that offered consumer-friendly tools, however, the tools were not necessarily all created equal.

The total cost estimators, for example, utilized a variety of methodologies to help consumers estimate their out-of-pocket spending. Some asked “one or two simple questions” while others “provided extensive, sometimes overwhelming, lists of selectable conditions and treatments,” according to the paper, before assigning a user to a profile of a low, medium or high user.

Wong says the research doesn’t show which method is best or measure the accuracy of the algorithms, and assessing those questions will require more data from multiple exchanges than is currently available to researchers.

“There’s not a lot of transparency on where these estimates come from,” she says. “I do think more transparency is on the horizon as there are calls for improved transparency across health care.”

The choice tools also were different depending on what stage of shopping the consumer was in on each marketplace. Researchers found “substantial differences” between the “window shopping” phase, or browsing plan options anonymously, and “real shopping,” or what consumers saw after creating an account with personal information.

But registering for an account didn’t always bring a consumer more information; in fact, researchers found, on some marketplaces, window shoppers got more information than real shoppers.

The cause for the differences is innocuous — in many cases, marketplaces use different vendors to design the window shopping websites and the real websites, which much be linked into state databases, Wong says — but worth noting given that not all consumers will be both window shoppers and real shoppers, especially if they are reenrolling.

“We found no indication that the decision to offer different choice environments for the two groups of shoppers was based on evidence — or even belief — that such an approach was optimal,” the researchers write.

Even for marketplaces that offered consumer-friendly tools, however, the tools were not necessarily all created equal.

No One Size Fits All

Insurance plans are moving towards required searchable provider directories, which will be useful for marketplace websites to include provider lookup tools. These integrated provider lookups were already offered on eight of 14 plans surveyed for window shoppers and nine of 14 for real shoppers. Wong recommended that marketplaces consider developing a method for flagging the size of the provider network (for example, on a scale from extra small to extra-large) to give consumers an idea of how easily they could access the care they’re considering purchasing.

“These types of indicators would allow consumers — particularly those who do not have preferred physicians or hospitals–– to choose a plan based on network size versus affordability and would minimize surprises when seeking care,” Wong writes.

The research points to three states — Connecticut, Kentucky and Washington — that have had fewer technical challenges with their marketplaces, which may have allowed them to develop their exchange sites and choice architecture more robustly than other states. But overall the trend points to a continued broadening of choices for health care consumers as the Affordable Care Act rolls toward its fourth enrollment period.

“Overall we’re feeling very positive that changes are being made,” Wong says. “Marketplaces are technically working pretty seamlessly when compared to the first enrollment. As states have more time, I would anticipate to see almost all of the sites improve.”