Reviewing for the Fellowship Awards Program

Welcome

Welcome to this learning module for peer reviewers of the Fellowship Awards Program.

The goal of this module is to ensure that reviewers understand the Fellowship Awards Program and feel prepared to effectively participate in the review process.

Navigation

This course is designed to be self-paced. Use the playbar below to navigate through the course.

• To pause or resume the course, press the Pause button.
• To navigate to the previous slide, press the Previous button.
• To navigate to the next slide, press the Next button.
• To mute or unmute audio, press the Mute button.
• To show or hide the closed captions, press the Caption button.
• To open or close the Table of Contents, press the TOC button.
• To collapse the playbar, press the Collapse Playbar button.

Note that you can also reposition the playbar at any time, should it inadvertently cover any on-screen materials. Simply click and drag it to a set position on the screen. Go ahead and try that now, if you wish.

Objectives

By the end of this module, you will be able to: identify key features of the Fellowship Awards Program, understand the adjudication criteria and rating scale used in the evaluation of applications, and summarize the steps in the peer review process for the Fellowship Awards Program.

Objective 1

In this section, you will learn about the Fellowship Awards Program and the eligibility criteria for applicants.

Fellowship Awards Program

The objective of the Fellowship Awards Program is to provide support for highly qualified applicants in all areas of health research at the post-PhD or post-health professional degree stages, and to add to their experience by engaging in health research either in Canada or abroad. The value and term of the awards varies depending on the recipient’s credentials and location of tenure.

Eligibility of Applicants

Please note that the eligibility requirement that all trainees be within 3 years of having completed their PhD has been removed.

Reviewers are asked to think critically about whether the research training position or program for which an individual is applying for funding will have the desired career benefits and impact compared to other applicants.

CIHR staff have reviewed all applications to confirm the eligibility of the applicants.

Reviewers may flag eligibility cases in the review process if they believe an application that should have been screened out was allowed into the competition.

Objective 2

In this section, you will learn about the adjudication criteria and rating scale used for the Fellowship Awards Program.

Assignment

You will primarily be assessing the applicant and the research training environment. As such, it is not essential for your research expertise to align directly with the research area of the application.

Reviewers are asked to apply their research expertise generally when assessing the diverse array of application assignments.

Adjudication Criteria

The evaluation of the application should be based on the following adjudication criteria: Achievements and Activities of the Applicant, Characteristics and Abilities of the Applicant and the Research Training Environment.

Please take into account the career stage of the applicants when reviewing applications to better assess and calibrate your set of applications.

Adjudication Criteria TOC

In the following slides you will learn more about the adjudication criteria listed below.

Click the button in the top right for additional details on the Adjudication Criteria found in Appendix A of the Reviewers’ Guide for Fellowship Awards.

Note: The program literature has been updated to incorporate the San Francisco Declaration on Research Assessment (or DORA) guidelines. CIHR is a signatory of DORA, which “recognizes the need to improve the ways in which the outputs of research are evaluated”.

Adjudication Criterion 1

Achievements and Activities of the Applicant is divided into 4 sub-criteria:

• Training Expectations;
• Proposed Research Project;
• Honours, Awards and Academic Distinction; and
• Research-Related Contributions and Activities.

For this section review the CCV of the Applicant including the Additional CCV Information, Training Expectations and Research Project Summary.

Adjudication Criterion 2

The 2nd criterion is the Characteristics and Abilities of the Applicant. For this criterion focus your review on the three sponsor assessment forms in the applicant’s application package.

They should offer a perspective on the applicant from individuals who are familiar with the applicant’s characteristics and abilities.

Recognize that positive comments are common while negative ones are not. Keep in mind that applicants have no opportunity within the application to provide a justification for their choices of sponsors.

Adjudication Criterion 3

The Research Training Environment considers the career stage and discipline of the supervisor(s). Expectations of mentoring by a recently established investigator should differ from your expectations of mentoring by a long-established researcher. For this section review the CCV of the primary supervisor and the research training environment sections of the application.

Sex and Gender

Learn more about Sex, Gender, and Health Research by exploring the resources on this slide. All links will open in new windows, so you can return to this module when you are ready.

Rating Scale

The ranking of Fellowship Award applications is completed using a structured review process. In a structured review process, reviewers are asked to provide an assessment for each adjudication criterion.

The score that you submit for each criterion on the 0.0 to 4.9 scale will be weighted automatically in the calculation of an overall score for each application you review in ResearchNet.

Reviewers should be assessing each application based on the information provided and should not be seeking additional information or outside opinion to supplement what is contained in the application.

Remember that all applications with a final rating of 3.5 or higher may be considered for funding.

As such, if you feel that an application should not be considered for funding, it is important that you assign a score below 3.5 to the application. Please use the full rating scale when evaluating your assigned applications.

Question 1

Test your knowledge!

You may refer to Appendix A of the Reviewers’ Guide for Fellowship Awards to assist you.

In the application you are reviewing, the applicant has justified why they chose their proposed training location and what they expect to learn.

Which sub-criterion of “Achievements and Activities of the Applicant” does this apply to?

1. Proposed Research Project;
2. Training Expectations;
3. Honours, Awards and Academic Distinctions; or
4. Research-Related Contributions and Activities

Question 2

Which of the following does not apply when considering “Research Training Environment” in your evaluation?

1. Training record of the supervisor;
2. The appropriateness of research environment including space, personnel and facilities;
3. Sponsor’s assessment letter about applicant; or
4. Supervisor’s research experience.

Question 3

True or False. To ensure that all applications are treated equally, reviewers should NOT complete additional research in addition to their evaluation on the content of the application.

Objective 3

In this section, you will learn about the review process for the Fellowship Awards Program.

Peer Review Process

The peer review process of the Fellowship Awards consists of assigning applications to committees, identifying conflicts of interest, conducting reviews of assigned applications, submitting reviews and ratings, and participating in re-review, if required.

Distribution of Applications

Applications are assigned to a specific committee based on the following: the theme of the proposed research activities; the applicant’s educational background at the time of taking up the award; and the applicant’s licensure status, if applicable, at the time of the competition’s application deadline date.

A diagram summarizing the peer review committees for Fellowship awards is displayed on your screen. Please visit the web link provided if you would like more information.

Identify Conflicts

Follow along with this simulation.

In ResearchNet, you will first need to agree to the Conflict of Interest and Confidentiality Agreement.

The expanded version of the Equity, Diversity and Inclusion Self-identification Questionnaire is a requirement for peer review committee members, enhancing our ability to understand the degree of diversity in peer review committees.

The questionnaire is accompanied by a Privacy Notice Statement which outlines CIHR’s intended purpose and uses of the self-identification data.

Next, to start conducting your reviews, you will click on the “Manage Conflicts/Ability to Review” task.

This will bring you to the set of applications that have been assigned to you.

You’ll select each application by clicking on the hyperlinked application numbers.

You will have access to the relevant information to determine if you are in conflict.

This is also where you will indicate whether you are in conflict or not in conflict with the application.

Conduct Reviews

Please ensure you are familiar with the adjudication criteria for the Fellowship Awards Program. You can now start working on your reviews by clicking on the “Conduct Reviews” task.

To assess the applications, click on the hyperlinked application numbers.

From this screen, insert your numeric rating and provide written feedback on the application’s strengths and weaknesses.

After saving your scores in ResearchNet, it is important to provide comments that justify your ratings, highlighting strengths and weaknesses for each adjudication criterion. Note that written feedback is mandatory for the Fellowship program.

Review Quality Expectations

CIHR has established a set of guiding principles to help support high quality in peer review.

For each application, reviewers must provide a concisely written assessment highlighting the strengths and weaknesses for each adjudication criterion.

When entering written feedback reviewers are asked to consider the following review quality criteria:

1. Appropriateness - Review comments are fair, understandable, confidential and respectful
2. Robustness - Review is thorough, complete and credible
3. Utility - Review provides feedback that addresses the needs of reviewers, applicants and funders

CIHR has developed a checklist as a practical tool to assist both applicants and reviewers to apply the review quality criteria to their received or submitted reviews.

Click on the button to see the checklist.

Participate in Re-Review

When you are ready to submit your reviews, you will first need to select the reviews and then click on “Submit Selected Final Review”.

When all reviews are submitted, a calculation is applied to determine which applications have received discrepant scores.

In this instance, CIHR will ask these reviewers to discuss the application in order to reconcile their scores.

If you are requested to do a discrepancy review on an application, you will be notified by e-mail with the contact information of the other reviewers and your access to that specific application on ResearchNet will be re-opened.

The goal of this process is to have all three reviewers listen to and consider each other’s opinions. In the end, you are not obliged to change your score but we ask that you re-submit.

Once all re-reviews are submitted, the final score for each application will be determined by taking the average of the three scores.

Finally, CIHR will generate ranking lists for each committee.

Applications are recommended for funding by committee from the top down in order of ranking as far as the budget will allow.

Summary

Congratulations! You have now completed the learning module for peer reviewers of the Fellowship Awards Program.

You should now be able to: identify key features of the Fellowship Awards Program, understand the adjudication criteria and rating scale used in the evaluation of applications, and summarize the steps in the peer review process for the Fellowship Awards Program.

Survey

Before continuing on to explore the Additional Resources, please complete the survey to assist CIHR in tracking progress and improving the quality of our learning. The survey will open in a new window.

Additional Resources

Click on the additional resources below to view important links for peer reviewers of the Fellowship Awards Program.

Date modified:

2024-05-30