Peer Reviewer Manual

Peer review is a type of quality assurance process that helps ensure the integrity of the scientific dissemination process. The quality of HLRC depends on a sound peer review process, since, with the help of peer reviewers, editors evaluate whether a manuscript is appropriate to the journal’s focus and scope, assess the manuscript’s merits, and decide whether or not to publish the manuscript. As such, reviewers serve an important role in carefully examining manuscripts submitted for publication to prevent the dissemination of irrelevant findings, unwarranted claims, incorrect interpretations, and opinions as fact. Furthermore, reviewers help authors and researchers to improve the overall quality of their presentation of research methods, findings, and conclusions, thus ensuring the scientific output is relevant and up-to-date.

There are several ways to conduct peer reviews, the most common being a blind review. In a single-blind review process, the names of the reviewers are hidden from the authors, but reviewers may know the names of authors. In a double-blind review process, both reviewers and authors are anonymous to each other. The HLRC uses a double-blind review process. Other, more recent review methods, which the HLRC may adopt in the future, include open reviews, in which the names of authors and reviewers are known to each other; collaborative reviews, in which groups of people conduct the review together (this process may still be open, single blind, or double blind); and post-publication reviews, in which the manuscript is published as submitted and reviewed openly by others.

Invitation to Review

Process.

The editors identify and formally invite peer reviewers who have expertise with the manuscript’s topic or methodology. Once invited, reviewers should carefully determine whether they are able to complete the review, considering the following issues.

  • Can the reviewer complete the assignment within the specified review period? This is very important, given that editors want to get information out quickly to potential readers. HLRC requests completed reviews be returned no later than 3 weeks from the date of acceptance.
  • Is the article within the reviewer’s specific expertise?
  • Can the reviewer confirm that he or she has no biases that could influence a decision regarding the publication of the manuscript?

Based on the answers to these questions, reviewers decide whether to accept or decline a given review. The HLRC editors ask potential reviewers to accept or decline a review within 10 days from the time of the request. Once reviewers decide that they can accept the review, they have several important responsibilities in the process.

Reviewer Responsibilities and Ethics

Reviewer responsibilities toward editors.

Reviewers should ensure that their HLRC profiles are updated regularly. This helps the editors to improve assignment efficiency. Reviewers who accept an assignment but then find they cannot complete it must inform the editors immediately. When an assignment is accepted, reviewers should provide careful and specific feedback, as this helps the editors to make a truly informed decision; the role of reviewers is critical to this process. Reviewers need to send specific feedback as well as a completed rubric with a recommendation to Accept Submission, Revisions Required, Resubmit for Review, Resubmit Elsewhere, Decline Submission, or See Comments (refer to “The HLRC Review Form” section below for specific details). The editors may not follow a specific recommendation, but the recommendation is important in informing the editors’ final decision.

Reviewer responsibilities toward authors.

It is critical to provide timely, unbiased feedback on the merits and scientific value of the manuscript. Avoid personal comments and criticism outside the scope of the manuscript; keep the tone objective and professional. Reviews should be constructive and encouraging to authors; the goal of providing meaningful feedback is to improve manuscripts and motivate authors to enhance their research and writing skills. Finally, specific feedback will help authors to revise their manuscript for future publication, even if the recommendation is to reject the submission. Finally, it is not in the authors’ best interests that reviewers focus on copyediting; rather, what is most useful to them is constructive feedback on content. The manuscript will undergo copyediting prior to publication.

Ethical considerations.

Peer reviewers are the gatekeepers of academic quality. Observing the basic principles of academic integrity is of utmost importance. It is crucial to respect the confidentiality of the process; reviewers should not discuss unpublished data with third parties or use access to unpublished, confidential data to their own advantage or to the disadvantage of others. Reviewers should destroy copies of manuscripts and associated data once the review is complete.

The HLRC Article Review Form

HLRC reviewers complete the online HLRC Article Review Form. Scrutinize each section of the work under review and provide meaningful feedback in complete sentences. Constructive criticism aides and encourages authors to improve their research skills and reporting. When writing the report, stay factual and objective. Assist the authors in understanding where they can enhance the manuscript to provide sufficient detail. It is good practice to offer feedback per section; in other words, offer meaningful comments and observations regarding the literature review, methodology, analysis, and conclusions. This makes it easier for authors to follow and attend to all comments.

The review form seeks feedback on the various aspects, described below, of each manuscript. Aside from the comment entries, reviewers rank each aspect using the following scale:

  • Not Applicable / Not Relevant for Review – The aspect is not appropriate or necessary for the particular manuscript.
  • None – The manuscript fails to include or support this aspect.
  • Limited – There is minimal support of the aspect within the manuscript but improvements are required.
  • Adequate – There is sufficient support of the aspect within the manuscript, but improvements would enhance the work.
  • Good – The authors achieve this aspect, although improvements or additions could be made.
  • Excellent – The authors clearly meet or exceed the indications of this aspect. There is no need or possibility for improvement regarding this aspect.

For any review item ranked lower than Good, reviewers must elaborate on their reasoning (100 characters or less for each instance) in the Comments-only-for-Editors field. This helps the editors to provide valuable feedback to the author(s).

Review form categories.

The review form includes the following questions with the ranking selected from a drop-down menu. A short explanation is given for each category.

  • Contribution to existing knowledge.
    The research adds new information or new findings/insights to the available body of scientific work.
  • Clarity in theory, conceptual/theoretical framework and consistency with literature.
    Authors are unambiguous in presenting the rationale and supporting evidence of their research.
  • Adequacy of research design and methods (e.g., design, procedures, context, instrumentation, and sample size) and data analysis procedures.
    The research achieves standard expectations for conducting social science research.
  • Data presentation and interpretation of results (tables, figures, summaries, etc.).
    Data support the thesis, and data presentation is clear, accurate, and complete.
  • Discussion integrates study results into the current literature in the field.
    The authors quote, paraphrase, cite, and respond to supportive or contradictory findings from other published research.
  • Conclusions follow from but do not extend beyond the study findings.
    The authors summarize and apply their findings without introducing new information or applying their findings beyond what their research actually supports.
  • Quality of writing (e.g., basic spelling and grammar, organization, clarity, style, and referencing, among other considerations).
    Select the rank appropriate for limitations to or errors within the writing that detract from the manuscript’s scientific value. Explain concerns in the Comments only for Editors field.
  • Significance to higher education researchers.
    The work provides new information or insights of use to other scientists.
  • Significance to higher education practitioners.
    The work provides new information or insights of benefit to those working in the field.
  • Comments for Author and Editors.
    Provide a narrative critique of the work, section by section, identifying areas for improvement within the research and the manuscript.
  • Comments only for Editors.
    Report praise of or concerns about the work. Feedback in this section is not appropriate for direct communication to authors but provides the editors significant insight regarding the reviewer’s suggestion to accept or decline the work.

Once the form is completed, the reviewer must make a recommendation (as detailed in the “Reviewer responsibilities toward editors” section above).

Reviewer Training Resources

To learn more about peer review, begin with the webinar Reviewing Manuscripts for Academic Journals (Burkholder, 2017). This session, conducted by HLRC Editor in Chief Dr. Gary J. Burkholder, “offers guidance and advice … on how to perform professional peer review for a research manuscript or essay.” The session also includes an example peer review.

Additional resources include Reviewing a Journal from Wiley Author Services (n.d.), Peer Review: Navigating the Peer Review Process and Becoming a Reviewer (Elsevier Publishing Campus, n.d.) a collection of online lectures, interactive courses, videos, and downloads; and Peer Review: A Global View (Devine & Frass, 2016), an insight supplement released by Taylor & Francis.

Publons (see below) offers Publons Academy, which is a free peer-review training course available to Publons-registered individuals. Publons intention is "to teach you the core competencies and skills needed of a peer reviewer" (Publons, n.d.).

Being recognized for peer-review work

The HLRC editors encourage reviewers to promote their peer review work and experience. HLRC is a recognized journal with Publons (2017a), a peer-reviewer profile service that allows reviewers to publicize their peer review work. Publons works in partnership with publishers to advance the commitment to and recognition of peer review. When accessing Publons reviewers will be asked to set up a profile. Once a profile is established, researchers can add their HLRC reviews by forwarding their HLRC confirmation-of-review receipt to reviews@publons.com (2017b).

References

Burkholder, G. [HLRC Editor]. (2017, February 15). Reviewing manuscripts for academic journals [Video file]. Retrieved from https://www.youtube.com/watch?v=VUawLMBVSkk

Devine, E. & Frass, W. (2016). Peer review: A global view [Research supplement]. Retrieved from Taylor & Francis website: http://authorservices.taylorandfrancis.com/peer-review-global-view/

Elsevier Publishing Campus. (n.d.). Peer review: Navigating the peer review process and becoming a reviewer. Retrieved from https://www.publishingcampus.elsevier.com/pages/69/Colleges/College-of-Skills-Training/Peer-review.html?PAGE_NR=69

Publons. (n.d.). Welcome to Publons academy. Retrieved from https://publons.com/community/academy/?utm_source=PublonsUsers&utm_campaign=14778ca78b-Mixed_reviews_issue_11&utm_medium=email&utm_term=0_d203ec3f11-14778ca78b-137171629

Publons. (2017a, Feb 24). How does Publons work? [Online FAQ entry]. Retrieved from https://publons.freshdesk.com/support/solutions/articles/12000009179-how-does-publons-work-

Publons. (2017b, March 7). How are reviews added to Publons? [Online FAQ entry]. Retrieved from https://publons.freshdesk.com/support/solutions/articles/12000009183-how-are-reviews-added-to-publons-

Wiley Author Services. (n.d.-a). Reviewing a journal. Retrieved from https://authorservices.wiley.com/Reviewers/index.html