Reference Letters and Predatory Journals

Reference Letters and the Specter of Publications in Predatory Journals

Jacqueline K. Owens and Peggy Chinn

Nurse Author & Editor, 2018, 28(1), 2

In the course of your professional duties, you may be asked to provide a recommendation or review the scholarly work of a peer. Nurses in academic or editorial roles often serve as peer reviewers either because they have knowledge about the area of scholarship of the person under review, or they serve as a blind reviewer for submitted manuscripts or activities demonstrating scholarship. Writing review letters or serving as a peer reviewer of manuscripts is prompted by a desire to serve the discipline. The intent is to provide expert commentary that assists decision-makers to render an informed conclusion about the outcome under consideration.

Recently each of us, external to our roles as editors, agreed to serve as a reviewer for nurse faculty under consideration for promotion, tenure or retention, only to subsequently discover a difficult situation. Knowing the rigor inherent to the academic review process, we had every reason to expect that the person’s achievements had already undergone internal scrutiny, putting forward this individual as worthy of consideration for promotion, tenure, and/or retention. However, these individuals had published articles in journals identified as predatory, disreputable publications. This article describes our experiences and offers potential considerations and solutions to this uncomfortable dilemma.

External Reviews

External reviewers serve the purpose of providing expert assessment of an individual’s record of scholarship, with evidence of contributions to the discipline beyond local boundaries. These are never cursory reviews; external reviewers are called upon to provide specific analysis and rationale supporting the quality and significance of the person’s work, and the potential of this person to continue to make significant contributions to the discipline.

As the term “external” suggests, reviewers may or may not have personal experience or knowledge of the individual under review, but they are in a position to provide an assessment from the perspective of the broad disciplinary community. These reviewers are selected because they are not directly involved with the individual in their workplace. However, because external reviewers are selected for their expertise and familiarity with a given area of scholarship, it is not unusual for them to already know about the person’s work. They may have encountered the person in some professional capacity, such as in the context of a professional organization, as a former student, or other past association. Letters of review always include a statement summarizing any prior association or knowledge about the person under review in order to establish the extent to which the reviewer is “external.”

The Review Process

Scholars in the local setting have already vouched for the person by forwarding the dossier to external review. Thus, reviewers typically expect to find an abundance of evidence in the dossier supporting a well-qualified candidate for promotion, tenure, or retention. The institution often sends the criteria on which the person is being assessed; occasionally they simply ask for an assessment of the dossier in consideration of general expectations for rank typical in similar institutions.

As reviewers, we first approach the task by examining the personal statement in which the individual typically, and rightfully, highlights the good things they have accomplished to deserve a positive review. We examine the curriculum vitae (CV) to assure it reflects the details in the personal statement, and note any aspects reflected in the CV that the personal statement does not show. We look for evidence that the person’s work is widely recognized; that other scholars draw upon the work of this person; and indications that the person’s influence is reaching national and international audiences. We may read student and peer evaluations of teaching, if they are included. And finally, we read the published articles submitted in the dossier. If we find any “red flags” we examine the material again to be sure we have not missed anything.

Our Red Flag: Predatory Publications

Given the exhaustive, rigorous process that occurs to the point to external review, we rarely find red flags. When we each (independently) recently found ourselves in the situation of discovering articles published in predatory journals, we were stunned. We completed our reviews, as described below. Likely we would have not addressed this further had we not shared our experiences during discussions at the annual meeting of the International Academy of Nursing Editors (INANE). We then realized that this is not an isolated situation.

INANE members first brought the problem of predatory publishing in nursing to public attention in 2014 when we published a collaborative statement about predatory publishing (INANE “Predatory Publishing Practices” Collaborative, 2014). We participated in the INANE-led initiative to write editorials about this concern in our respective journals (“Editorials Published: Open Access & Editorial Standards,” 2014). We have written, led workshops, and lectured at various gatherings to alert faculty to the challenges and confusion surrounding legitimate open access publishing, predatory publishing, and the important distinctions between these digital-age phenomena (Nicoll & Chinn, 2015). All of these efforts over the past 3 years, however, have only scratched the surface to educate nurses about unethical publishing practices.

When we made this discovery during our external reviews, each of us had similar experiences and reactions. Our overall assessment of the dossiers we reviewed provided evidence of accomplishments that were worthy of a positive review. This included the articles published in predatory publications. But the fact that these articles, even though worthy, were published in what we also call “illegitimate” or “disreputable” publications casts serious doubt about the person’s record of accomplishment.

Predatory publishers like, and eagerly accept, articles that lend credibility to their enterprise, but this does not diminish the fact that this is a serious problem for the author(s) of such articles. There are several serious concerns. First, in all likelihood, the article has not undergone the rigorous peer review prior to publication that assures readers that experts in the subject matter and methodology have found the work credible and authentic. This vetting is vitally important for the integrity of the literature in the discipline, and incredibly significant in terms of possible patient outcomes if the work ultimately influences nursing practice. The published work, while available on the journal website at the time of publication, is not properly indexed or archived; it may vanish from view at any moment. There is no long-term assurance it will remain available to an interested reader.

Given these concerns, the red flag escalated to a dilemma. In our role as external reviewers, we are expected to affirm that the person under consideration for promotion, tenure, or retention is engaging in scholarship and producing scholarly work expected to influence the discipline in a positive way now and in the future. Even if we are convinced that the published work is sound and important, we know that because of where it is published, the article likely will never be discovered by other scholars searching the literature for information on this topic. Affirmation of this work lends credibility to questionable published practices that have been deemed unethical by many (INANE “Predatory Publishing Practices” Collaborative, 2014) and serves to diminish the trustworthiness each nurse places in the body of published literature.

Our Solutions

Each of us reached the painful conclusion that in all good conscience, we must address this issue in our letters of recommendation. We include here (see below) excerpts of our responses in the hope that these examples can be useful to others making this same discovery during review work. A tactful and thoughtfully composed response can provide an excellent opportunity to educate others who may be unaware about the concerns inherent to unethical predatory publishing practice.

Example 1 (Chinn)

I am concerned that one of the publications appears in a journal published by a deceptive or predatory publisher . . . . In my view, this publication could qualify for publication in a credible journal, but the problem of predatory publishing is still not well understood and many nursing scholars remain vulnerable to making this choice in the quest for publication. For more information on this problem see “Caught in the Trap: the Allure of Deceptive Publishers.” Also refer to the recent “early view” article titled “Quality of Articles Published in Predatory Nursing Journals” in which the authors (of which I am one) explain the hazards of publishing in these journals, noting that there are occasional articles of high, even excellent quality. In my assessment Dr. ______’s article, it falls in this category. While there is a trend for promotion and tenure committees to disregard publications in predatory journals, I highly recommend that this concern not be seen as a barrier in her promotion and tenure consideration. In part, this is due to the timing of this publication when the problem of predatory publishing is still emerging as a concern in nursing, and in part because the published article does have merit. However, it is important for the College faculty to be aware of this issue, and assure that future publications for all nursing faculty appear only in journals that assure sound editorial practices. For more information about delineating adequate publishing outlets see “Study of Predatory Open Access Nursing Journals” and most recently “Cabell’s New Predatory Journal Blacklist: A Review.”

Example 2 (Owens)

I do have some concern with the 20XX publication, “[removed].” The [journal name] is published by [publisher name]. This publisher is listed on Jeffrey Beall’s List of [predatory] Publishers indicating that it is a potential, possible, or probable predatory scholarly open access publisher. I did also confirm with Mr. Beall via email that this journal is one of a fleet of journals offered by predatory publisher, [publisher name]. The term “predatory publishers” was created by Beall, an academic librarian whose focus of study is unethical, open access publishing practices. Beall has created a blog that lists both predatory publishers and standalone predatory journals, called Scholarly Open Access; it can be found at Briefly, some major concerns related to predatory publishing include lack of appropriate and credible peer review, discoverability, and archiving. Mr. Beall’s excellent work has enlightened many related to the concerns of predatory publishing. However, in my conversations both as a faculty member and a journal editor, it is evident that even a great many well-respected, senior scholars in many disciplines are unaware of this practice. These publishers have become increasingly sophisticated in the presentation of the journals, and it is sometimes very difficult to detect predatory practices.

I could not locate this journal in either of the two most commonly used database indexing services in healthcare, MEDLINE, or the Cumulative Index of Nursing and Allied Health Literature (CINAHL). This is likely because of predatory status of the publisher. The potential outreach of [person under review]’s work is significantly diminished if it cannot be located in the major databases used in healthcare. In sum, I believe that [person under review] conducted credible post dissertation work and has good information to share with researchers and clinicians interested in [topic]. However, it is my opinion that at best, this is not a top-tier journal, and at worst, it is likely a journal using unethical, predatory publishing practices (per Mr. Beall’s, and my own, evaluation).

NOTE: At the time of this review, Beall’s Lists were the most current resources related to predatory publishers. Beall’s lists have been removed and are no longer a current resource; however, there are resources available in the literature to identify red flags for unethical publishing practices and predatory publishers.


Having engaged in discussions at INANE meetings around the concerns of predatory publishing for several years, we understand that there remains a long way to go before these issues are clearly understood. This problem is not going away—it is growing. Even still, it was startling to discover nursing scholars presenting work published in disreputable publications as part of dossiers for promotion, tenure, and retention.

Should you, as an external reviewer, encounter the dilemma of writing a recommendation for a person with a publication in a predatory journal, or perhaps a national or global level presentation at a conference sponsored by one of these publishers, we hope that this article will inform your response. The profession of nursing has been at the forefront of education about predatory publishing (“Editorials Published: Open Access & Editorial Standards,” 2014). It is important that nurse leaders and scholars continue this effort whenever the occasion arises. As education about unethical publishing practices continues, it will soon no longer be acceptable to claim naïveté on this matter and unknowingly publish in a disreputable journal. The time is approaching when it will be only reasonable to assume that an author has deliberately chosen this option, disregarding the potential devastating effect on the credibility of all scholarly literature (Kolata, 2017).


  1. Editorials Published: Open Access & Editorial Standards. (2014, November 9). Retrieved November 14, 2017, from
  2. INANE “Predatory Publishing Practices” Collaborative. (2014). Predatory publishing: what editors need to know. Nurse Author & Editor, 24, 1. Retrieved from
  3. Kolata, G. (2017, October 30). Many academics are eager to publish in worthless journals. The New York Times. Retrieved from
  4. Nicoll, L. H., & Chinn, P. L. (2015). Caught in the trap: the allure of deceptive publishers. Nurse Author & Editor, (4), 4. Retrieved from
  5. Oermann, M. H., Nicoll, L. H., Chinn, P. L., Ashton, K. S., Conklin, J. L., Edie, A. H., … Williams, B. L. (2017). Quality of articles published in predatory nursing journals. Nursing Outlook.

About the Authors

Jacqueline K. Owens, PhD, RN, CNE is Editor-in-Chief, OJIN: The Online Journal of Issues in Nursing and Associate Professor and Director, RN to BSN Program, Ashland University, Ashland, OH, USA. She is also a member of the Authors-in-Residence for Nurse Author & Editor.

Peggy Chinn, RN, PhD, FAAN is the Editor-in-Chief of Advances in Nursing Science, author of a few books, and manager or co-manager of several websites/blogs, including INANE. She is an Author-in-Residence for Nurse Author & Editor.

2018 28 1 2 Owens Chinn

Copyright 2018: The Authors. May not be reproduced without permission.
Journal Complication Copyright 2018: John Wiley and Son Ltd.