FSA Quality Assurance Toolkit

Checklist

FSA Quality Assurance Toolkit

Last updated: 03 March 2023

In this guide

On this page

Checklist 1 – Producing research

Question	Completed? Mark the appropriate option with a ✓
Q1. Title, project lead and year
Enter free text.
Q2. Has the project been considered by the appropriate steering group and has an EPI form been adequately completed (for example, has a clear research need been outlined)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q3. Has a precise research question/aim been specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q4. Is the research design…
Experimental
Cross-sectional
Longitudinal
Case study
Comparative
Other
Not specified
Q5. Is the research method…
Qualitative
Quantitative
Both
Q6. Is there a good match between the research question/aim, research design and research method?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q7. Is the study population and setting specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q8a. If Q5 = Quantitative, go to Q8b. If Q5 = Both, go to Q8a and Q8b.
Q8a. Is the sampling method…
Purposive sampling
Convenience sampling
Theoretical sampling
Other
Not specified
Go to Q9.
Q8b. Is the sampling method…
Simple random sampling
Stratified sampling
Quota sampling
Convenience sampling
Other
Not specified
Go to Q9.
Q9. Is the sampling method appropriate for addressing the research question?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to 9a. If Q5 = Quantitative, go to 9b. If Q5 = Both, go to Q9a and Q9b.
Q9a. Is the target sample size justified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q9b. Has a sample size calculation been conducted?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q10. Are the research instruments valid and reliable?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q11a. If Q5 = Quantitative, go to Q11b. If Q5 = Both, go to Q11a and Q11b.
Q11a. Is the analytical approach…
Thematic analysis
Grounded theory
Framework analysis
Other
Not specified
Go to Q12.
Q11b. Is the analytical approach…
Chi-square test, correlation test, t-test or analysis of variance
Linear regression
Logistic regression
Survival analysis
Time series analysis
Meta-analysis
Other
Not specified
Go to Q12.
Q12. Is there a good match between the research question, research method, and analytical approach?
No
Yes – partly
Yes – fully
Free text input to justify rating [mandatory as this is a judgement call]
Q13. Has a relevant checklist from the EQUATOR Network been used to design the study protocol?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q14. Will descriptive data on the characteristics of participants be presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q15. If Q5 = Quantitative, go to Q19. If Q5 = Both, go to Q15 and Q19.
Q15. Will two or more researchers be involved in the analysis process (for example, through double coding)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q16.
Q16. Have the researcher(s) specified the qualitative analysis process that will be used to help ensure consistency between the original data and the identified themes?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q17.
Q17. Will similarities and differences between participants be explored (for example, negative cases)?
No
Yes – partly
Yes – fully
Free text input to justify rating [mandatory, as ‘no’ or ‘yes-partly’ is acceptable for some research]
Go to Q18.
Q18. Will participants provide feedback on the findings (for example, member checking)?
No
Yes – partly
Yes – fully
Free text input to justify rating [mandatory, as ‘no’ or ‘yes-partly’ is acceptable for some research]
Go to Q21.
Q19. Will descriptive data on exposures/interventions and potential confounders be presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q20.
Q20. Will unadjusted and adjusted point estimates and confidence intervals be presented alongside statistical significance?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q21.
Q21. Will generalisability be described in the interpretation of the results?
No
Yes – partly
Yes – fully
Free text input to justify rating [mandatory for qualitative research, as this can be challenging to ascertain].
Q22. Will causality be considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Not applicable
Free text input to justify rating.
Q23. Will uncertainty be considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q24. Will a clear study conclusion be presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q25. Have/will ethical and data governance issues (for example, informed consent, the right to withdraw, confidentiality, data protection) been/be identified and addressed?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q26. If the research relates to, or will potentially affect, vulnerable and underserved respondents, have/will impacts on them been/be identified and addressed?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q27. Have/will risks related to legal, analytical, and operational issues been/be identified and addressed?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q28. Has/will the research plan been/be pre-registered (particularly important if it is a behavioural trial)?
No
Yes – partly
Yes – fully
Free text input to justify rating [mandatory, as it may not always be vital to pre-register the research plan].
Q5. Have/will study materials, data and/or code been/be made openly available?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q6. Have/will key stakeholders been/be identified and involved in the research?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q7. Have/will relevant dissemination outputs and activities been/be specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.

Checklist 2 – Assessing research reports

Question	Completed? (yes or no)
Q1. Title, lead author and year	✓
Enter free text.
Q2. Has a clear research need been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q3. Has a precise research question/aim been specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q4. Is the research design…
Experimental
Cross-sectional
Longitudinal
Case study
Comparative
Other
Not specified
Q5. Is the research method…
Qualitative
Quantitative
Both
Q6. Is there a good match between the research question/aim, research design and research method?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q7. Is the study population and setting specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q8a. If Q5 = Quantitative, go to Q8b. If Q5 = Both, go to Q8a and Q8b.
Q8a. Is the sampling method…
Purposive sampling
Convenience sampling
Theoretical sampling
Other
Not specified
Go to Q9.
Q8b. Is the sampling method…
Simple random sampling
Stratified sampling
Quota sampling
Convenience sampling
Other
Not specified
Go to Q9.
Q9. Is the sampling method appropriate for addressing the research question?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to 9a. If Q5 = Quantitative, go to 9b. If Q5 = Both, go to Q9a and Q9b.
Q9a. Is the target sample size justified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q9b. Has a sample size calculation been conducted?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q10. Are the research instruments valid and reliable?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q11a. If Q5 = Quantitative, go to Q11b. If Q5 = Both, go to Q11a and Q11b.
Q11a. Is the analytical approach…
Thematic analysis
Grounded theory
Framework analysis
Other
Not specified
Go to Q12.
Q11b. Is the analytical approach…
Chi-square test, correlation test, t-test or analysis of variance
Linear regression
Logistic regression
Survival analysis
Time series analysis
Meta-analysis
Other
Not specified
Go to Q12.
Q12. Is there a good match between the analytical approach, the research method and the research question?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q13. Has a relevant checklist from the EQUATOR Network been used in the reporting of the results?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q14. Have descriptive data on the characteristics of participants been presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q15. If Q5 = Quantitative, go to Q19. If Q5 = Both, go to Q15 and Q19.
Q15. Have two or more researchers been involved in the analysis process (for example, through double coding)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q16.
Q16. Is there consistency between the data presented and the themes?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q17.
Q17. Have similarities and differences between participants been explored (for example, negative cases)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q18.
Q18. Did participants provide feedback on the findings (for example, member checking)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q21.
Q19. Have descriptive data on exposures/interventions and potential confounders been presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q20.
Q20. Have unadjusted and adjusted point estimates and confidence intervals been presented alongside statistical significance?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q21.
Q21. Has generalisability been considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q22. Has causality been considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Not applicable
Free text input to justify rating.
Q23. Has uncertainty been considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q24. Has a clear study conclusion been presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.

Checklist 3 – Writing tender specifications

Question	Completed?(yes or no)
Q1. Specification title	✓
Enter free text.
Q2. Has a clear research need been outlined or has the importance of a clear research need been outlined (in the event of the tenderer specifying this)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q3. Has a precise research question/aim been specified or has the importance of a precise research question/aim been specified (in the event of the tenderer specifying this)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q4. Is the research design…
Experimental
Cross-sectional
Longitudinal
Case study
Comparative
Other
Not specified
Q5. Is the research method…
Qualitative
Quantitative
Both
Q6. Is there a good match between the research question/aim, research design and research method or has the importance of a good match between these aspects been outlined (in the event of the tenderer specifying this)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q7. Is the study population and setting specified or has the importance of specifying the study population and setting been outlined (in the event of the tenderer specifying this)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q8a. If Q5 = Quantitative, go to Q8b. If Q5 = Both, go to Q8a and Q8b.
Q8a. Is the sampling method…
Purposive sampling
Convenience sampling
Theoretical sampling
Other
Not specified
Go to Q9.
Q8b. Is the sampling method…
Simple random sampling
Stratified sampling
Quota sampling
Convenience sampling
Other
Not specified
Go to Q9.
Q9. Is the sampling method appropriate for addressing the research question or has the importance of using an appropriate sampling method for addressing the research question been outlined (in the event of the tenderer specifying this)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to 9a. If Q5 = Quantitative, go to 9b. If Q5 = Both, go to Q9a and Q9b.
Q9a. Has the importance of justifying the target sample size been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q9b. Has the importance of providing a sample size calculation been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q10. Has the importance of using valid and reliable research instruments been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q11a. If Q5 = Quantitative, go to Q11b. If Q5 = Both, go to Q11a and Q11b.
Q11a. Is the analytical approach…
Thematic analysis
Grounded theory
Framework analysis
Other
Not specified
Go to Q12.
Q11b. Is the analytical approach…
Chi-square test, correlation test, t-test or analysis of variance
Linear regression
Logistic regression
Survival analysis
Time series analysis
Meta-analysis
Other
Not specified
Go to Q12.
Q12. Is there a good match between the analytical approach, the research method and the research question or has the importance of a good match between these aspects been outlined (in the event of the tenderer specifying this)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q13. Has the importance of using a relevant checklist from the EQUATOR Network in the reporting of the results been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q14. Has the importance of presenting descriptive data on the characteristics of participants been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q15. If Q5 = Quantitative, go to Q19. If Q5 = Both, go to Q15 and Q19.
Q15. Has the importance of involving two or more researchers in the analysis process (for example, through double coding) been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q16.
Q16. Has the importance of there being consistency between the data presented and the themes been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q17.
Q17. Has the importance of exploring similarities and differences between participants (for example, negative cases) been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q18.
Q18. Has the importance of asking participants to provide feedback on the findings (for example, member checking) been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q21.
Q19. Has the importance of presenting descriptive data on exposures/interventions and potential confounders been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q20.
Q20. Has the importance of presenting unadjusted and adjusted point estimates and confidence intervals alongside statistical significance been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q21.
Q21. Has the importance of considering generalisability in the interpretation of the results been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q22. Has the importance of considering causality in the interpretation of the results been outlined?
No
Yes – partly
Yes – fully
Not applicable
Free text input to justify rating.
Q23. Has the importance of considering uncertainty in the interpretation of the results been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q24. Has the importance of presenting a clear study conclusion been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q25. Have key stakeholders provided input into the research objectives?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q26. Have/will tender opportunities been/be promoted through appropriate channels?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q27. Have/will internal and external peer reviewers been/be identified through appropriate channels?
No
Yes – partly
Yes – fully
Free text input to justify rating.

Checklist 4 – Assessing tender bids

Question	Completed? (Yes or no)
Q1. Title, project lead and year	✓
Enter free text.
Q2. Has a clear research need been outlined?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q3. Has a precise research question/aim been specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q4. Is the research design…
Experimental
Cross-sectional
Longitudinal
Case study
Comparative
Other
Not specified
Q5. Is the research method…
Qualitative
Quantitative
Both
Q6. Is there a good match between the research question/aim, research design and research method?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q7. Is the study population and setting specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q8a. If Q5 = Quantitative, go to Q8b. If Q5 = Both, go to Q8a and Q8b.

Q8a. Is the sampling method…
Purposive sampling
Convenience sampling
Theoretical sampling
Other
Not specified
Go to Q9.
Q8b. Is the sampling method…
Simple random sampling
Stratified sampling
Quota sampling
Convenience sampling
Other
Not specified
Go to Q9.
Q9. Is the sampling method appropriate for addressing the research question?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to 9a. If Q5 = Quantitative, go to 9b. If Q5 = Both, go to Q9a and Q9b.
Q9a. Is the target sample size justified?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q9b. Has a sample size calculation been conducted?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q10.
Q10. Are the research instruments valid and reliable?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q11a. If Q5 = Quantitative, go to Q11b. If Q5 = Both, go to Q11a and Q11b.
Q11a. Is the analytical approach…
Thematic analysis
Grounded theory
Framework analysis
Other
Not specified
Go to Q12.
Q11b. Is the analytical approach…
Chi-square test, correlation test, t-test or analysis of variance
Linear regression
Logistic regression
Survival analysis
Time series analysis
Meta-analysis
Other
Not specified
Go to Q12.
Q12. Is there a good match between the analytical approach, the research method and the research question?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q13. Has a relevant checklist from the EQUATOR Network been used to design the study protocol?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q14. Will descriptive data on the characteristics of participants be presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
If Q5 = Qualitative, go to Q15. If Q5 = Quantitative, go to Q19. If Q5 = Both, go to Q15 and Q19.
Q15. Will two or more researchers be involved in the analysis process (for example, through double coding)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q16.
Q16. Will the researcher(s) ensure that there is consistency between the data presented and the themes (for example, have they specified that illustrative quotations will be presented)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q17.
Q17. Will similarities and differences between participants be explored (for example, negative cases)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q18.
Q18. Will participants provide feedback on the findings (for example, member checking)?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q21.
Q19. Will descriptive data on exposures/interventions and potential confounders be presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q20.
Q20. Will unadjusted and adjusted point estimates and confidence intervals be presented alongside statistical significance?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Go to Q21.
Q21. Will generalisability be considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q22. Will causality be considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Not applicable
Free text input to justify rating.
Q23. Will uncertainty be considered in the interpretation of the results?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q24. Will a clear study conclusion be presented?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q25. Have/will ethical and data governance issues been/be identified and addressed?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q26. Have/will impacts on vulnerable and underserved respondents been/be identified and addressed?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q27. Have/will risks related to legal, analytical, and operational issues been/be identified and addressed?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q28. Has/will the research plan been/be pre-registered?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q5. Have/will study materials, data and/or code been/be made openly available?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q6. Have/will key stakeholders been/be identified and involved in the research?
No
Yes – partly
Yes – fully
Free text input to justify rating.
Q7. Have/will relevant dissemination outputs and activities been/be specified?
No
Yes – partly
Yes – fully
Free text input to justify rating.

View entire guide