The hidden layer in peer review: how editors actually check your references

You click submit. Months of work, compressed into a single button press. What do you think will happen next?

Most researchers picture their manuscript landing in the inbox of two or three expert reviewers who will read it carefully, engage with the argument, and return detailed feedback. That picture is not wrong, but it is incomplete. Before any reviewer sees your paper, a handling editor reads it first and makes an early judgement about whether it is ready to go further.

Editors assess many things at this stage: scope fit, methodological plausibility, language quality, and adherence to the journal's formatting requirements. This article focuses on one element that authors frequently underestimate, and that editors consistently notice: the reference list.

What editors assess during desk review

The editorial pre-screening, sometimes called desk review, is rarely described in journal author guidelines. In practice, a handling editor will form an initial view across several dimensions at once: whether the paper addresses a question the journal cares about, whether the study design is credible, and whether the manuscript looks like it was prepared with care.

That last question is where the reference list becomes diagnostic. A bibliography assembled carelessly signals something about the rest of the manuscript. Conversely, a well-constructed reference list, current, comprehensive, and formatted correctly, suggests an author who has done the work properly throughout.

"A reference list is a map of a researcher's intellectual neighbourhood. Editors can tell, within moments, whether the author lives there or is just visiting."

What editors look for in a reference list

Here is what tends to draw editorial attention, positively or negatively:

• Currency: Are the most recent key papers cited? A manuscript that references nothing from the past five years raises questions about whether the author is genuinely engaged with where the field currently stands.
• Seminal works: Does the list include the foundational studies the field expects? Their absence suggests either unfamiliarity with the literature or a bibliography assembled at speed.
• Self-citation balance: A reference list where 30 to 40 per cent of citations point to the authors' own prior work is a soft flag. Some journals have informal thresholds; others pass it to reviewers to assess.
• Format consistency: Inconsistent capitalization, missing volume numbers, or author lists that trail off arbitrarily signal that the manuscript was not carefully prepared for this specific journal.
• Citation-claim alignment: Editors with subject expertise will spot-check whether references actually support the claims they are attached to. Misattribution, even accidental, is a credibility issue.

When citation errors make it into print

Research has consistently shown that citation errors are common and often go undetected until it is too late.

The scale of quotation inaccuracy

A systematic analysis by Mogull found that around 14.5 per cent of cited assertions in biomedical research articles were inaccurate. Most of those errors were major, meaning the cited source either failed to support, was unrelated to, or directly contradicted the claim being made. [1] These were not marginal papers: they had gone through peer review. The errors survived because no one verified the citation against the original source.

The problem of citing retracted work

Bolland and colleagues found that papers citing publications with integrity concerns continued to do so even after those concerns had been formally documented, with errors propagating through the literature uncorrected. [2] Citation of retracted publications, the authors concluded, is a persistent and structurally difficult problem for scientific self-correction.

What "below threshold" sometimes means

When a manuscript is genuinely out of scope for a journal, an editor will say so, and mean it. Scope decisions are straightforward. What is less visible to authors is what sits behind decisions framed as "below the threshold for review." That threshold reflects the overall readiness of the manuscript, and a weak reference list is part of that assessment. It is not a hidden reason substituting for the real one; it is one of several legitimate criteria that together determine whether a paper is ready to be reviewed.

Editors are not looking for a reason to reject. They are looking for evidence that the work is ready. A well-constructed reference list is part of that evidence.

Signals that concern editors

Heavy self-citation; nothing cited from the past five years; inconsistent formatting; references that do not match in-text claims; absence of foundational studies

Signals that build confidence

Style consistent with the journal guide; recent and seminal works balanced; references clearly supporting their claims; scope neither sparse nor padded

What this means for your submission practice

Your reference list deserves the same revision pass as your methods section. Three habits make a consistent difference.

First, do a literature check within two weeks of submission. A key paper published after you began writing can still be incorporated before you submit. Editors notice when a manuscript cites work from the year of submission.

Second, verify every reference against the original source, not just the abstract. Mogull's analysis found that the majority of citation errors were major misattributions, not trivial formatting slips. [1] Citation managers pull metadata from databases that contain errors; treat their output as a first draft and check each entry yourself.

Third, match the journal's style precisely. Formatting inconsistencies add editorial burden and signal carelessness. Getting this right is courtesy that costs little and communicates a great deal.

Citation hygiene is not just about accuracy. It is about the story your manuscript tells about you before a single reviewer opens it.

A tool worth knowing

Manually checking a reference list against the source, verifying DOIs, and ensuring formatting consistency across dozens of entries is time-consuming and error-prone. CiteOrbit is a tool built specifically for this step. It matches every in-text citation to the corresponding bibliographic entry, cross-checks references against Crossref, Google Scholar, OpenAlex, and Semantic Scholar, adds missing DOIs, flags styling inconsistencies, and detects hallucinated or non-existent references. It produces a report with a fix-list before you submit.

The broader point

Peer review exists to filter for quality, but editors are the gatekeepers before peer review begins. Their first impression of a manuscript is formed across several dimensions, and references are among the most legible of those signals. A reference list is not a bibliography you append at the end. It is part of the argument. Build it like one.

References

[1] Mogull SA. Accuracy of cited "facts" in medical research articles: a review of study methodology and recalculation of quotation error rate. PLoS One. 2017;12(9):e0184727.

[2] Bolland MJ, Grey A, Avenell A. Citation of retracted publications: a challenging problem. Account Res. 2022;29(1):18-25.