The process of peer review may not be perfect, but it is the best solution we have to judge research projects and determine which projects to support in a world with limited resources. In particular, we intend the review of CCAS internal proposals to be a developmental process, whereby everyone who participates gets useful, constructive feedback. We ask that reviewers provide narrative comments, in addition to numerical scores, in order that the PI can learn how to strengthen their proposal. In order to provide meaningful feedback, reviewers need to offer clear criticism where appropriate. Overall, reviews should answer the solicitation and review rubric, while highlighting the strengths of the proposal and making constructive suggestions for improvement. Reviewers should aim to write a review which they would appreciate receiving as a PI, and which offers ideas to strengthen the proposal.
The following guidelines are provided to make the review process more productive and effective for both reviewers and PIs. We ask that you read them through carefully before reviewing proposals, and that you adhere to them. In considering the impact of implicit bias and providing guidelines for review language, we seek to minimize the potential that a well-intended comment be perceived as a micro-aggression. We also believe that a more inclusive review process is both fairer to PIs and leads to investment in a more impactful and diverse portfolio of supported projects at the college level.
Implicit Bias
In structuring the reviews and defining the scope of review we are trying to limit the impact of implicit bias: when people unconsciously hold attitudes toward others or associate stereotypes with them. Implicit bias results in poorer decision making and the perpetuation of historic marginalization across all fields. We are committed to work to actively reduce its impact in all aspects of CCAS research.
If you want to learn more about implicit bias and its potential impacts upon judgement, there are many resources, for example:
- Goldin, Claudia and Cecilia Rouse. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians." The American Economic Review 90.4 (2000): 715-741.
- Moss-Racusin et al. “Science faculty’s subtle gender biases favor male students.” PNAS 109 no. 41 16474-16479 (2012) https://doi.org/10.1073/pnas.1211286109.
- Implicit Bias tests: https://implicit.harvard.edu/implicit/
Scope of the Review
As a reviewer, you are asked to review only the proposal in front of you. While we try to avoid having reviewers from the same department of the PI, you may still know the PI or be aware of their work from other venues. We ask that you do your best to set aside this knowledge and judge the merits of the proposal purely on the content of the proposal and relevant, generally available, scholarly literature. This is the only fair way to proceed as not every PI in the competition will be known by their reviewer, and we endeavor to have all proposals reviewed on an even footing. To that end, please resist the temptation to seek additional information on the PI, beyond the biographical information within the proposal package, before completing and submitting the review. The information you might find may bias your review, and should not be taken into account in your deliberation of the merits of the project.
Criteria for the Review
Please adhere strictly to the review guidelines given. They have been constructed with the aim of obtaining the best selection of high impact projects, and to minimize the impact of bias on our decision making. It may be that the guidelines for submission, or the proposal call could be improved. If you identify such deficits in the criteria, please make us aware and share your suggestions for improvement with ccasres@gwu.edu. However, proposals may only be judged against the proposal call and review rubrics under which they were submitted. PIs cannot be penalized for their failure to address things which were not asked of them.
Language of the Review
When serving as a peer reviewer, we ask that you be kind in the way in which you construct your reviews. Constructive criticism is welcomed, indeed it is the best way to help improve our work, but the criticism should always be delivered in terms which do not demean the PI. Here are some key suggestions.
- Clearly identify and praise positive aspects of the proposal.
- Where you identify issues, describe how the proposal could be improved rather than focusing on where the proposal is lacking.
- Select your language carefully: for example, calling an assumption naïve could be interpreted as ad hominem, explain that the assumption missed some essential complexity, and suggesting which additional aspects could be considered changes the dynamic of the comment while addressing the same issue.
- Direct your suggestions / commentary to the proposal or idea rather than the PI: instead of “The PI has not performed sufficient work in this field”, “The proposal should provide more evidence to demonstrate the PI’s appropriate expertise”. Remember your review is of the proposal, not the individual.
- After writing the review, save it, leave it for some time, and reread it carefully before submission.
For some more ideas for alternate phrasing and examples of good practice see https://plos.org/resource/how-to-write-a-peer-review/ under the heading “Giving Feedback”.
We ask that you take particular care to observe the above guidelines in cases where the proposal addresses sensitive societal issues and / or the proposal reveals the PI to be a member of an historically marginalized group. It is crucial that we as a College of Arts and Sciences address complex societal issues, and find words to debate ideas and address challenges. We should, however, do so respectfully in order that the tone of our communication does not hinder its message. This does not exempt the proposal from constructive criticism; indeed to do so would be a disservice to the PI. However, it does mean that extra care should be taken to avoid an unintended microaggression and place an additional burden on a faculty member who may already have faced additional challenges.
Procedural Implementation
We aim to read all reviews before releasing them to the panel and to the PI under review. Should we find issue with a review which appears to be aggressive, demeaning, or contains language which could be interpreted as discriminatory, we will approach the reviewer, explain our concerns, and ask them to revise the language of their review. If, in the flood of review reading, we accidentally overlook such language, please make us aware of the situation, whether you are a panelist or the PI under review. We will take steps to ensure that the reviewer is made aware of our concern, and offer guidance as to how to review more effectively in future.
