Rejection Handout Final - Posgrado en Ingeniería de...

Rejection

Page | 1 © 2012 American Journal Experts® | www.journalexperts.com

American Journal Experts is dedicated to helping international researchers succeed. In addition to providing services such as editing, translation, peer review, journal recommendation, and manuscript and figure formatting, we also want to share our expertise through workshops and educational materials. We are delighted to present this seminar in Colombia as part of our mission to empower scientists around the world to publish their research!

If you want information about AJE’s services, are interested in a custom workshop, or have more questions about writing or publishing, please e-‐mail us at:

[email protected]

Responding to reviewers CONTENTS

Publication process 1 Author 2 Journal 3 Editor 4 Peer Review 5 Galley proofs and uncorrected proofs 6 Publication 7 Cost of publishing a scientific paper 8 When rejected 9 Responding to reviewers 10 Other information 11 References 12

Rejection


Rejection

Rejection has become the norm in scholarly publishing; directly related to editorial standards and authors manuscript presentation. Most rejections are not related to the scientific contribution of the manuscript but caused by issues in the presentation of the manuscript. 1. Publication Process

a) The process

b) Steps leading to rejection

Rejection


c) Steps leading to acceptance

2. Author

a) Author’s basic premise: If a manuscript meets scientific standards and contributes to the advancement of science it should be accepted for publication.

b) Author’s objective: to publish in the high-‐status, competitive journals with a wide circulation

c) Author: i. Does the research ii. Writes the papers iii. Submits to a journal iv. If asked to: makes changes, works on rewriting and performing new experiments

if necessary v. If rejected, resubmits to as many journals as necessary until the paper is

published d) Author’s worries:

i. The peer review system is random and a barrier to the progress of science ii. A novel result may not be published before the competition publishes iii. Knowing when to publish follow-‐up studies if the first paper is still in the process

of being published iv. For young researchers, time is of the essence: they need publications to start

their career

Rejection


3. Journal

a) Prioritization of the literature b) The price of a journal is more dependent on the perceived quality of its content than on

the number of papers published c) There is no guarantee that the criteria of the journal or of their reviews will be similar to

the author’s d) High impact journals weigh high on novelty (a subjective concept)

4. Editor Stephen Lock was editor of the BMJ. He conducted a study in which he alone decided which of the consecutive series of papers submitted to the journal he would publish. He then let the papers go through the usual process. There was little difference between the papers he chose and those selected after the full process of peer review. (Richard Smith, 2006)

a) Type of editor roles: a. Editor-‐in-‐Chief (EIC)

i. The Editor-‐in-‐Chief is the face of the journal ii. Also called “editorial head” -‐ has the final responsibility for all operations

and policies iii. Editor-‐in-‐Chief can overrule reviewers and publish any manuscript

b. Associate Editor (AE)

i. Looks over manuscripts for selection ii. Finds reviewers for the journal iii. Advises the Editor-‐in-‐Chief on the acceptance or rejection of manuscripts

based on reviewers’ decisions iv. Helps prepare journal volumes for publication v. Corrects, revises, and adapts manuscripts for publication

c. Managing Editor (ME)

i. Senior member of a publication management team ii. Oversees and coordinates the publication’s editorial activities (in the US) iii. Manages budget, staffing, and scheduling of a publication (in the UK) iv. Oversees a group of the publisher’s journals

d. Editorial Office Assistant/Peer Review Coordinator

i. Day-‐to-‐day e-‐mail responses to authors’ questions ii. Keeps track of the peer review process iii. Provides quality control for the peer review process

Rejection


e. Editorial Board i. Consulting committee ii. Formed by associate editors and commonly used to attract high-‐quality

manuscripts to the journal iii. Provides advice to the Editor-‐in-‐Chief on manuscripts and reviewers

b) Desk Rejection

a. Rejection creates a market for lower-‐tier journals to publish rejected papers

b. Rejections and resubmissions to the next best journal will naturally lead a paper to land in the journal whose perceived quality matches that of the manuscript (which means money for the sister journal)

c. The editor in-‐chief makes decisions on acceptance/rejection depending on: i. Number of papers/volume

1. How many manuscripts are lined up for the next volume ii. Number of volumes/year iii. Area of study iv. Traditional journal vs. open access v. Scope vi. Facts

d. Don’t waste your time arguing – change journals

e. Look for the real reasons your manuscript got desk rejected (science, writing,

English)

f. Take time to work on your manuscript, look for a journal that fits your M&M

c) What are the editor-‐in-‐chief and associate editors looking for? a. Sound science b. Importance c. Originality d. Relevance e. Clear and engaging writing f. Papers that will be well received and frequently read and cited g. Ethics (e.g., are clinical trials registered?)

I was challenged by two of the cleverest researchers in Britain to publish an issue of the BMJ composed of papers that had failed peer review and to see if anybody noticed. I wrote back, “How do you know I haven’t done it?” -‐Richard Smith, BMJ Editor-‐in-‐Chief, 2006

Rejection


5. Peer Review

Ø A reviewer’s task is to guide the editor in making their decision on publication of the manuscript. It is not to decide by a majority vote what should not be published. a) Why we need peer review:

a) Pre-‐publication review provides the publisher with a preview of how the article can be received by the field

b) To exclude papers of poor quality i. After the peer review process, the journal selects the top manuscripts for publication (peer review provides an expert opinion)

c) Peer review is “seen as highly accurate in calculating the quality of science” i. Safeguarding the scientific standard ii. Keeping with the “wow” factor, improving science, and printing strong and

novel science

d) To detect fraud

Since peer review can be used to detect fraud, the BMJ did a number of studies where they inserted major errors into manuscripts that were sent to many reviewers. The results: not one peer reviewer spotted all errors, some peer reviewers spotted only a quarter of the errors, and others did not spot any of the introduced errors. (Richard Smith, 2006)

b) Types of reviews

a) Internal review (by editorial staff) i. Desk rejection ii. Usually done by an in-‐house editor familiar with the area of study; used for

a quick “okay” on the science before an external peer review

b) External review (by experts in the field, usually not paid) i. Tradition since the 18th century (Royal Society of Edinburg in 1732) ii. Usually, peer reviews perform this service to help their field and progress of

science

c) Different review options a) Single-‐blind review

i. Reviewers remain anonymous, but the reviewer knows the author’s identity ii. Some critics argue that the anonymity hinders accountability, promotes

excessive reviews, and promotes irresponsibility and malice iii. The majority of journals use this system iv. For some journals, authors can request double-‐blind review

Rejection


• Authors may not know thath they have this option or may not request it because of the potential for reviewer bias

v. Used in the biomedical and hard science fields

b) Double-‐blind review i. The identity of both the reviewer and author are hidden ii. Some journals have this service available upon request iii. Not very common in the biological sciences iv. Not much evidence to argue for double-‐blind review

c) Open peer review

i. Both the author and the peer reviewer know each other’s identities

Example: British Medical Journal (BMJ) • 50% of BMJ’s submissions are desk rejected by an in-‐house editor • The rest of the manuscripts are sent out to 1 or 2 peer reviewers who receive

compensation for their services • The final decision is made by the editor “hanging committee” (a statistician, an

external editor advisor, and the editor-‐in-‐chief) d) Hybrid Systems

i. Upon submission, relevant remembers of the standing editorial board are asked to look over the manuscript for any technical problems. If the manuscript meets the basic journal guidelines, it is uploaded onto the website

ii. Registered researchers can post comments anonymously or after registering online, and the authors can publicly respond

iii. The action editor moderates the online discussion. iv. After a period of time (~8 weeks), the author has the option to make

changes or to submit for traditional peer review v. Reviewers for this stage have been selected before manuscript was

uploaded; they can comment during the initial stage vi. Manuscripts that pass all stages are posted immediately

Examples: Atmospheric Chemistry and Physics (ACP) http://www.atmospheric-‐chemistry-‐and-‐physics.net/ Electronic Transaction on Artificial Intelligence (ETAI)

http://www.ida.liu.se/ext/etai/ There have been variations of this approach used by:

Lancet, BMJ, and BioMed Central

Rejection


d) How reliable is the peer review system?

The level of agreement between reviewers on whether a paper should be published is little better than you’d expect by chance. -‐ Stephen Lock, Editor, BMJ, 1985 Robbie Fox, editor of the Lancet, who was no admirer of peer review, wondered whether anybody would notice if he would swap the piles marked “published” and “rejected.” He also joked that the Lancet had a system of throwing a pile of papers down the stairs and publishing those that reached the bottom.

-‐ Richard Smith, Editor of the BMJ and chief executive of the BMJ Publishing Group

e) How many peer reviewers are necessary?

a) In the 1950s, editors would use no more than two reviewers, and by the 1990s,

many journal editors were pressed to use three to four reviewers

b) Approximate numbers of manuscripts at each stage (BMJ group): i. 1000 rejected by one editor within 48 hours ii. Another 3000 rejected by a second editor iii. Within one week of submission, 3000 read by senior editor; another 1500

rejected iv. 1500 sent to two reviewers; then 500 more rejected

c) Kravitz and Baker, 2011 study of neuroscience manuscripts

i. Average reviews before publication: 6.3 ii. Average number of journals before acceptance: 2.1

d) Inter-‐rater reliability (Bornmann and Daniel, 2010)

i. Manuscripts reviews have high inter-‐rater reliability when two or more independent reviewers of the same scientific manuscript agree.

ii. In some cases, inter-‐rater reliability is seen as undesirable: • Too much agreement is, in fact, a sign that the review is not working

well (reviewers were not properly selected for diversity, and therefore, some of the reviews were unnecessary)

iii. When the peer review process differs between the reviews working on the same manuscript, giving a low inter-‐rate reliability:

• It can be a result of selecting complementary reviewers (e.g., a generalist and a specialist). The author may gain a broader and better review.

• It has been suggested that an increase in the number of reviewers could increase the inter-‐rater reliability

Rejection


Example: Inconsistency between peer reviews (Richard Smith, 2006) Reviewer A: I found this paper an extremely muddled paper with a large number of deficits. Reviewer B: It is written in a clear style and would be understood by any reader.

f) Author-‐related bias by peer reviewers and/or journals

a) Author/Institution prestige • Peer review rewards prominent scientists (senior scientists or researchers

from prestigious institutions), while junior scientists and less elite institutions are held to a much more rigorous review

A very prominent physicist’s manuscript was rejected because his name was left out of the manuscript in error. After adding his name into the manuscript the editor’s decision was revoked! (Merton, 1968) Twelve articles were resubmitted with different author and institution names after being published in high-‐impact journals. Of the 12 papers submitted, only three journals realized that they had already been published (and in 1982, there were fewer submissions than in 2012, so this should have been easier to detect!). Of the nine remaining papers, eight were rejected not for novelty but for “serious methodological flaws.” (Peters and Ceci, 1982)

b) Country of Origin Examples: Floyd E. Bloom, the editor-‐in-‐chief of Science (1995-‐2000), once stated, “Researchers that make mistakes when editing their manuscripts in the English language should make the same type of mistakes during their research.” Óscar H. del Brutto, editor-‐in-‐chief of Revista Ecuatoriana de Neurología replied, “It is important to remind Dr. Bloom that our mother tongue is not English. It is logical to assume that English-‐speaking authors would commit more language errors when writing in Spanish than we do when writing in English. This however does not mean that the English-‐speaking authors do not know how to do science.” (Del Brutto OH, 1996) The Editor-‐in-‐Chief of the New England Journal of Medicine, Jerome P. Kassirer stated, “Third world countries should be given advice about nutrition and immunizations before receiving advice on text editing. Poor countries should be worrying about this type of stuff before doing high quality science” (Cabral and Kraus, 2009)

c) Reviewer gender and/or author gender • Favoring one gender over another

g) Manuscript-‐related bias by peer review and/or journal

a) Positive results (mostly those that go against other studies) b) Negative results c) Controversial science (more likely to receive harder reviews)

Rejection


d) English language quality

h) Peer review bias a) Personal interest in the topic of the manuscript b) Scientific competition (unjustified harsh review to block or slow down publication) c) Peer reviewer-‐author friendship d) The author-‐editor-‐peer reviewer cycle creates a burden on the reviewer, as the

same paper requires various exchanges among the participants without economic compensation

i) How editors use peer review a) To coordinate the process of review and publication and make decisions about

publishing or rejecting manuscripts b) Based on the opinion of the reviewers, editors decide whether to allow author

revisions to address the reviewers’ comments or to simply reject the manuscript

j) Editorial decision

a) Top journals have little interest in publishing more papers as submissions increase; by maintaining the number of accepted papers, the exclusivity of the journal increases and, with it, the perceived quality and price

b) The editor’s responsibility is to send back an off-‐topic, uncompetitive, badly written and formatted manuscript

k) Potential weakness of the peer review system a) Reviewers take the position of “gatekeepers” looking for weakness in the submitted

manuscript instead of finding the positive contributions of the paper b) Peer review does not provide feedback about scientific merit (just “novelty”) c) Peer review is slow and delays publication d) Peer review can become biased e) Acceptance or rejection can be biased f) Peer review can be influenced by personal interests

a. Preferences b. Worldview

g) The chance to publish is placed under the control of a small minority h) Results of bad peer review:

• Systemic bias • Unreliability (lack of inter-‐rater reliability) • Hindrance of the propagation of science to the world

i. arXiv (mathematics, physics, computer science) 1. Highly automated electronic archive and distribution server

for research articles 2. Maintained and operated by the Cornell University Library

Rejection


3. The absence of peer review and open exchange might be seen as a negative: in particular, how is good science supposed to be distinguished from bad science?

4. arXiv is self-‐policing in regard to quality (they know what is good and what is bad science)

5. Most authors publish in arXiv and in traditional journals

The example of arXiv helps to differentiate between the kinds of information scientist need “on the ground” of daily scientific work versus the kind of information used by administrators and granting agencies. Physics community may have features that make arXiv more appropriate for them than for others scientific communities. They are described as strongly collaborative and have “little stress over priority and intellectual property.” (K. Gunnarsdóttir, 2005)

l) Arguments for peer review a) Safeguarding the scientific standard b) The Editor-‐in-‐Chief can prevent reviewer bias c) Improving the quality of the manuscript d) Helps science be disseminated around the world

m) How to improve peer review

a) Guidelines i. Scientific significance (Does the manuscript represent a substantial contribution to our field?)

• New concepts? • Ideas? • Methods? • Data?

ii. Scientific quality • Are the scientific approach and methods valid? • Are the results discussed in an appropriate and balanced way? • Are the references appropriate?

iii. Presentation quality • Are the scientific results and conclusions presented in a clear,

concise and structured way? • Number and quality of figures • Appropriate use of the English language

b) Standardize the peer review system c) Formal training

• Give suggestions about how to improve the manuscript • Be civil and respectful

d) Feedback i. Constant feedback on:

• How to write in a “positive” way to authors

Rejection


• How thorough to be • Avoiding personal biases

e) Professional review agencies

6. Galley proofs and uncorrected proofs

a) Once the paper has been accepted by the journal, the first copy of the manuscript will

be sent to you for proofreading.

1. Galley proof: 1. Originally, this was the name for a typeset copy of a document used to

permit correction of errors before the type was made up into formal books, journals, etc.

2. Used in traditional publishing for advanced reading copies 3. The author’s final chance to look over the manuscript before printing

2. Uncorrected proof:

1. The author’s final chance to look over the manuscript before printing, sent out in a digital format

7. Publishing

Ø When a paper is accepted, it is published in a journal and becomes available to the field.

a) Two trends

1. Traditional journals 2. Open access journals

8. Cost of publishing a scientific paper a) To author:

1. > 1,500 USD (color photos, black and white) 2. Some journals charge no page charges, some do 3. Research (experiments) 4. Researcher’s time 5. Subscription to read the journal of your field

b) To publisher: data unknown

Rejection


9. When rejected

a) Be positive, you have at least received reviews, so the journal likes your science enough

b) Give yourself time to figure out what to change in the manuscript c) Rejection is part of the learning process d) Rejection can lead to a better manuscript e) Always write nice responses to reviewers f) Do not write excuses; focus on solutions g) Try to make the changes asked for; reviewers have the upper hand

10. Responding to reviewers

a) Tips for writing responses (see Annesley, 2011, for more tips.) 1. Do not use an aggressive or defensive tone 2. Never use one reviewer’s response against another 3. Thank reviewers for the good suggestions 4. Say things like “we agree” or “this is an excellent point” if you are going to change

your manuscript as suggested 5. Pick your battles! If you can make some changes easily, go ahead. Then, if you

refuse other suggestions, it looks like you are compromising with the reviewer instead of fighting them.

6. For each response: 1. Number your responses to each reviewer 2. Restate the reviewer’s question or concern OR quote the reviewer’s

comment 3. Try to acknowledge something that can be improved

a. Say your text may have been unclear b. Say that you could provide more detail c. Say that the suggestion is valid, but would belong in another paper d. etc.

4. Write each response so it can be read by itself; never refer to other responses (don’t say “see the response to Reviewer 2”)

d) Examples of responses to reviewers:

What you want to say: You just didn’t understand what we wrote. What you should say: Several statements that we made were more ambiguous than intended, and we have adjusted to the text to be clearer. What you want to say: No one knows the answer to that question.

Rejection


What you should say: This is a valid question, and we are actively pursuing the answer in our lab. OR

This is a valid and important question, and we are curious what the results would be. However, we are unaware of any studies that provide the answer. What you want to say: That experiment would take forever! What you should say: The suggested experiment is interesting and would provide additional information about…, but we feel that it falls outside the scope of this study. What you want to say: We’re not saying we proved anything – that’s just our hypothesis! What you should say: We agree that this explanation is speculative at this time, and we have edited the text to state that our conclusion is only suggested by our results. Note: Try to make some changes to the text to clarify your thoughts. What you want to say: You didn’t even read what we wrote! What you should say: We did not intend to indicate [insert reviewer’s mistaken assertion here], and we have therefore altered the text to specify that [insert correct conclusion here]. Note: change some text to appease the reviewer What you want to say: You are being so picky about grammar or formatting! What you should say: We apologize for this error, and we have corrected the text as suggested.

11. More information

Publication process

Desk rejection 70-‐90% Accepted upon initial submission 2% Average total number of journals ~3 Average total revisions ~3 Average total reviews ~7 Time under review 3-‐4 months Time to publication 6-‐12 months (Kravitz and Baker, 2011, and Suls and Martin, 2009)

Rejection


12. References

1.-‐ Navarro F. El inglés, idioma internacional de la medicina. 2001 (http://www.medtrad.org/panacea/IndiceGeneral/n3_FANavarro.pdf) 2.-‐ Langdon-‐Neuner. Hangings at the bmj: What editors discuss when deciding to accept or reject research papers. 2008 (http://www.emwa.org/JournalArticles/JA_V17_I2_Langdon_Neuner1.pdf) 3.-‐ Bornmann L. and Daniel H-‐D. The luck of the referee draw: the effect of exchanging reviews. 2009 (http://www.lutz-‐bornmann.de/icons/BornmannDanielLB.pdf) 4.-‐ Bornmann L. and Daniel H-‐D. Reliability of reviewers’ ratings when using public peer review: a case study. 2010 (http://www.psh.ethz.ch/research/publications/reliability2010.pdf) 5.-‐Bornmann L., Mutz R. and Daniel H-‐D. A reliability-‐generalization study of journal peer reviews: a multilevel meta-‐analysis of inter-‐rater reliability and its determinants. 2010 (http://www.plosone.org/article/info:doi/10.1371/journal.pone.0014331) 6.-‐ Lloyd M. Gender factors in reviewer recommendations for manuscript publication. 1990 (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1286270/) 7.-‐ Suls and Martin. The air we breathe. 2009 (http://pps.sagepub.com/content/4/1/40.short) 8.-‐ Kravitz D. and Baker C. Toward a new model of scientific publishing: discussion and a proposal. 2011 (http://www.frontiersin.org/computational_neuroscience/10.3389/fncom.2011.00055/full) 9.-‐ Smith R. Peer review: a flawed process at the heart of science and journals. 2006 (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/) 10.-‐ Birukou A. et al. Alternatives to peer review: novel approaches for research evaluation. 2011 (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3237011/) 11.-‐ Cabral A. and Kraus A. TERCER MUNDO: sinónimo de incompetencia. 2009 (http://www.ejournal.unam.mx/cns/no40/CNS04008.pdf) 12.-‐ Annesley T. Top 10 tips for responding to reviewer and editor comments. 2011 (http://www.clinchem.org/content/57/4/551.full) 13.-‐ Gunnarsdóttir k. On the role of electronic preprint Exchange in the distribution of scientific literature.2005 (http://sss.sagepub.com/content/35/4/549.short)

Rejection Handout Final - Posgrado en Ingeniería de...

Documents

Transcript of Rejection Handout Final - Posgrado en Ingeniería de...