Does Numeracy Skill Affect Choice Consistency in Stated Preference Experiments?

Background: Choice experiments (CEs) are commonly employed in health economics to measure preferences for multi-attribute health services. In CEs, respondents are presented with choices sets, each of which contains two or more alternatives that vary with respect to attribute levels. For each choice task, respondents are expected to face trade-offs between attributes and based on these trade-offs, they state what alternative they would choose. Analysis of responses relies on the Lancasterian theory of attributes-based demand, assuming that individual service attributes (e.g., cost, effectiveness, side effects, etc.) determine individuals’ choices. This assumption allows estimation of marginal rates of substitution across individual attributes and willingness to pay (WTP) or willingness to wait (WTW) measures. However, the consistency of the choice responses is likely to depend on the proper evaluation of each piece of information and the ability to make non-random choice decisions.

Objective: This paper investigates whether there is a link between numeracy skill, choice consistency, and willingness to wait (WTW), using a population of roughly 250 patients enrolled on the waiting list for kidney transplantation in Italy.

Methods: Patients participate in a choice experiment (CE) measuring time and risk preferences in kidney transplantation and answer a three-item numeracy skill test. The experiment included four kidney transplantation dimensions (attributes): two quantitative time attributes (waiting time and expected graft survival), and two qualitative risk attributes (infectious risk and neoplastic risk). Heteroskedastic multinomial logit (HMNL) model was employed to investigate the effect of numeracy skill on choice consistency.

Results: A higher numeracy skill tended to result in more consistent choices and a smaller weight of the error term in the estimated linear-additive underlying utility function. Patients with higher numeracy skill can state decisions that are based more firmly on the content of the choice tasks, namely the attributes and their respective levels. Further, they tend to have a lower WTW for a kidney transplant with better-expected outcomes. The effect of numeracy skill on the WTW is through choice consistency. That is, patients with high numeracy skills tended to provide more consistent responses to the choice questions, and these group of patients have a lower WTW for changes in the kidney transplantation attributes compared to patients with low numeracy skills. These relationships are significant, and even when one takes account of other differences between these patients in terms of their age, education, and gender.

Discussion: The study highlighted the importance of incorporating short numeracy questions to determine the consistency of responses obtained from choice experiments, suggesting the implications for WTW estimates. More research in a different setting is needed to confirm these results.