Table 6 Overview of the quality assessment of the included studies

Criteria

Adams et al. [39]

Arbiol et al. [46]

Bishai et al. [47]

Brown et al. [35]

Brown et al. [36]

de Bekker-Grob et al. [48]

de Bekker-Grob et al. [38]

Determann et al. [49]

Eilers et al. [51]

Flood et al. [52]

Flood et al. [53]

Guo et al. [55]

Hall et al. [41]

Hofman et al. [56]

Hofman et al. [44]

Huang et al. [57]

Lambooij et al. [58]

Ledent et al. [59]

 Attributes and levels grounded in qualitative work with target population

1

0

0

1

1

0.5

1

1

0.5

1

1

0

0

1

0

1

1

0.5

 No conceptual overlap between attributes

0

1

1

0.5

0.5

0.5

0.5

1

0.5

0.5

0.5

1

0.5

1

1

0.5

0.5

0.5

 Uni-dimensional attributes

0.5

1

1

1

1

1

1

1

0.5

1

1

1

0.5

1

1

1

1

0.5

 Opt-out/status quo option or justification forced choice

1

1

1

0.5

0.5

1

1

1

1

0

0

1

1

1

1

0

1

1

 Experimental design optimal or statistically efficient

1

0.5

0.5

1

1

1

1

1

0

0

0

0.5

0.5

1

1

0

0

0.5

 Piloting conducted amongst target population

1

0.5

0

1

1

0.5

1

0.5

1

0.5

0

0

0

0.5

0.5

0

0

0.5

 Target population(s) appropriate for research objective

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

0.5

1

1

 Sampling frame representative of target population

0.5

0.5

0.5

1

0.5

1

1

1

0.5

0.5

1

0.5

0

1

1

1

1

0.5

 Response rate sufficient to minimize response bias

0

0

0

0.5

0.5

1

1

0.5

0.5

0.5

0.5

1

1

0.5

1

0.5

0.5

0

 Any pooled analysis from different subgroups appropriate

1

1

1

0.5

1

1

1

1

1

0

0

0.5

1

0.5

1

0.5

0

1

 Econometric model appropriate for choice task design

1

1

1

1

1

1

1

1

1

0.5

0.5

1

1

1

1

0.5

1

0.5

 Econometric model accounts for serial correlation of choices

1

1

1

1

1

1

1

1

1

0

0

1

1

1

1

0

1

0

 Relative attribute effects compared using a common metric

1

1

1

1

1

1

1

1

0

0

0

1

1

1

1

1

1

1

Total validity score

10

9.5

9

11

11

11.5

12.5

12

8.5

5.5

5.5

9.5

8.5

11.5

11.5

6.5

9

7.5

Validity criteria

Liao et al. [60]

Liao et al. [61]

Lloyd et al. [37]

Marshall et al. [42]

Ngorsuraches et al. [45]

Oteng et al. [62]

Pereira et al. [63]

Poulos et al. [64]

Poulos et al. [65]

Poulos et al. [66]

Poulos et al. [40]

Sadique et al. [67]

Sapède et al. [68]

Seanehia et al. [69]

Shono et al. [70]

Shono et al. [71]

Sun et al. [72]

Veldwijk et al. [34]

Verelst et al. [73]

Verelst et al. [74]

Wang et al. [75]

Wong et al. [76]

 Attributes and levels grounded in qualitative work with target population

0

1

1

0

0

0.5

0.5

0

0

1

0.5

0

1

0

0

0

1

1

0

0.5

0

0

 No conceptual overlap between attributes

0.5

0

0.5

1

0.5

0.5

0.5

0.5

0.5

1

1

0

1

1

1

1

0.5

1

0.5

0.5

1

1

 Uni-dimensional attributes

1

1

1

0.5

1

1

1

1

0.5

0.5

1

0

1

1

1

1

1

1

0

0.5

0.5

1

 Opt-out/status quo option or justification forced choice

1

1

0

1

1

1

0

1

1

1

1

1

1

1

1

1

0

1

0.5

0

0.5

1

 Experimental design optimal or statistically efficient

0.5

0.5

0.5

0.5

0.5

0.5

0.5

1

1

0

0

0

0

0.5

0.5

0.5

0

1

1

1

1

0.5

 Piloting conducted amongst target population

0.5

0.5

1

1

0.5

0.5

0

1

1

1

0.5

1

0.5

1

0.5

0.5

0

1

0.5

1

1

1

 Target population(s) appropriate for research objective

0.5

0.5

1

1

1

0.5

1

1

1

1

1

1

0.5

1

1

1

0.5

1

1

1

1

0.5

 Sampling frame representative of target population

0.5

0

0

1

0.5

1

1

1

1

1

0.5

1

0

0.5

1

1

1

1

0.5

1

0.5

0.5

 Response rate sufficient to minimize response bias

0

0

0

0

1

0

0

0.5

0

0.5

0

0

0

0

0

0

1

0.5

0

1

0

1

 Any pooled analysis from different subgroups appropriate

1

1

1

1

1

1

1

1

1

1

1

0

1

0.5

1

1

0.5

1

1

1

0.5

1

 Econometric model appropriate for choice task design

1

1

0

1

0.5

1

0.5

1

1

1

0.5

1

1

1

1

1

0.5

1

1

1

1

1

 Econometric model accounts for serial correlation of choices

1

1

0

1

0

1

1

1

1

1

0

1

1

1

0

1

1

1

1

1

1

0

 Relative attribute effects compared using a common metric

0.5

0.5

1

1

0.5

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

0.5

1

Total validity score

8

8

7

10

8

9.5

8

11

10

11

8

7

9

9.5

9

10

8

12.5

8

10.5

8.5

9.5