Rigor and Reproducibility
In January 2016, the NIH implemented requirements for strengthening reproducibility of biomedical research by providing new research strategy instructions for grant proposals, requiring a separate attachment for applicants to describe authentication of key biological and/or chemical resources, and additional rigor and transparency questions study group members are to consider during reviews. It is important to note that applications that do not comply with the directive may be withdrawn from the review process*. Further, authentication of key biologicals/chemicals is given additional review consideration by study sections.
The NSF is, to some extent, also monitoring reproducibility and rigor of research. In 2014, the NSF Reproducibility Framework developed recommendations to encourage robustness and reproducibility in research. Through NSF's DCL (Dear Colleague Letter) announcements, addressing rigor & reproducibility has been a focus for proposals in Computer and Information Science and Engineering, and the Directorate for Geosciences. At this time there is no specific requirement within NSF proposals unless explicitly stated in the FOA.
Below are several helpful links to policy, training and attachment preparation guidelines pertaining to the NIH requirement and the NSF discussions.
- Rigor & Reproducibility Policy Statement: https://grants.nih.gov/reproducibility/index.htm
- Notice: Authentication of Key Biological/Chemical Resources: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-17-068.html
- Research & Training: https://www.nih.gov/research-training/rigor-reproducibility
- Application Instructions: https://www.nih.gov/research-training/rigor-reproducibility/updated-application-instructions-enhance-rigor-reproducibility
- Reviewer Guidance on Rigor & Transparency: https://grants.nih.gov/grants/peer/guidelines_general/Reviewer_Guidance_on_Rigor_and_Transparency.pdf
- *Notice: Rigor & Transparency in NIH & AHRQ Research Grant applications: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-16-011.html
- Rigor & Transparency in NIH and AHRQ Career Development Applications: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-16-012.html
- NIH Video on Background & Goals of Rigor & Reproducibility Program: https://grants.nih.gov/reproducibility/module_1/presentation_html5.html
- One page guide: https://grants.nih.gov/grants/RigorandReproducibilityChart508.pdf
- FAQs: https://grants.nih.gov/reproducibility/faqs.htm
- Notice: Formal Instruction in Rigorous Experimental Design and Transparency to Enhance Reproducibility: NIH and AHRQ Institutional Training Grants, Institutional Career Development Awards, and Individual Fellowships: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-16-034.html
- A Framework for Ongoing and Future NSF Activites to Improve Reproducibility, Replicability and Robustness in Funded Research: https://www.nsf.gov/attachments/134722/public/Reproducibility_NSFPlanforOMB_Dec31_2014.pdf
- Dear Colleague Letter: Encouraging Reproducibility in Computing and Communications Research:
- Dear Colleague Letter: Reproducibility and Robustness of Results: https://www.nsf.gov/pubs/2016/nsf16083/nsf16083.jsp