Table 1 Infectious Disease Modeling Reproducibility Checklist elements reported in COVID-19 modeling papers

Computational Environment

1.1) Is the operating system documented?

1.2) Is the operating system version documented?

Analytical Software

2.1) Is the name of the analytical software documented (e.g., the programming language name)?

2.2) Is the analytical software accessible for free?

2.3) Is the version of the analytical software documented?

2.4) Do the authors include a specific identifier (DOI, URL, citation) that points to the analytical software that was used?

2.5) Is the analytical software installation guide accessible online?

Model Description

3.1) Is the complete, structured model description provided in the publication, supplement, or referenced publication?

0.53 (0.38, 0.67)

0.58 (0.51, 0.66)

3.2) Is the model specified in the publication or supplement (contrary to being referenced in other papers)?

Model Implementation (“Code”)

4.1) Is the model implementation (e.g., code, workflow) openly accessible online?

0.86 (0.75, 0.96)

0.86 (0.77, 0.94)

4.2) Does the model implementation (e.g., code, workflow) have a version or modification date?

4.3) Does the model implementation (e.g., code, workflow) have an identifier?

4.4) Is the computer language of the model implementation (e.g., code, workflow) documented?

4.5) Are all model implementation (e.g., code, workflow) dependencies clearly specified in either the publication or supplemental files?

4.6) Are the model implementations (e.g., code, workflow) annotated with comments?

Data

5.1) Does the model in the publication use input data?

0.75 (0.62, 0.87)

0.59 (0.47, 0.70)

5.2) Has the source and content of the input data been described in the publication or supplement?

5.3) Does the paper cite a specific, unique, and persistent identifier to refer to each input dataset?

5.4) Is the input data openly accessible?

5.5) Is the data in a format that can be easily re-formatted (or “parsable”) to meet the input specifications of the model implementation?

Experimental Protocol

6.1) Are all the mentioned parameter values for the model implementation (e.g., code, workflow) documented in a single location (e.g., table or list in the publication or supplement)?

0.65 (0.51, 0.79)

0.69 (0.60, 0.77)

6.2) Is there an explanation of how the described/mentioned categories (computational environment, analytical software, model implementation, and data) were used together to create the results (e.g., figures and/ or tables)?