An initial submission to the RR section contains an Introduction and a Methods section only, which is first assessed by the Editorial team on the scientific merit and fit with the journal, and the availability of all the supplementary materials. If a Stage 1 (S1) submission passes triage, the submission will be fully peer-reviewed on its merits and feasibility of methods. A S1 manuscript can receive the following decisions: desk-reject, reject after review, revise, or accept. If the paper is in principle accepted (IPA), the manuscript enters stage 2 (S2) and the authors are asked to collect data. The authors inform the editorial office about their final time schedule with regard to data collection, i.e. the estimated deadline for their S2 manuscript. The authors are required to communicate any changes in this schedule to the editorial office, in order to keep them informed.
Preparation of the manuscript:
Stage 1 (S1):
Cover letter: In the cover letter, the authors should make clear what the rationale for their proposed study is. If their study is a replication, they should specify, (1) whether it is a direct or conceptual replication, (2) whether they expect they will replicate the original findings, and (3) in the case of a direct replication, whether there has been any contact with one or more of the original authors, and whether any of them has been involved in the design of the current study or agreed to some form of collaboration. If not, the original authors will be contacted (if possible) by the editorial office and invited to comment on the manuscript produced in Stage 2. This comment will then be published independently with the replication report.
Introduction: Provide an overview of the relevant literature and include a motivation for the study. The review should logically lead to the rationale of the current study. In keeping with APA guidelines, state the purpose of the study, a brief summary of your approach and the hypothesis/es tested at the end of the review of the relevant literature.
Methods (no word limit):
Stimulus materials and related specifics should be made available to reviewers so that pre-registration can be checked independently (e.g., https://osf.io/). Include a full description of the intended sample, and criteria for the exclusion of participant data, including a definition of outliers. Include a justification of the sample size, which can be based on power analysis, or a desired accuracy in estimates. Plan both for the presence, as the absence, of a true effect. Be careful when relying on effect sizes from past research, as these are likely upwardly biased – previous effect sizes can be a starting point, but not the endpoint of power analyses. The study should be designed such that a prior statistical power of 0.9 or higher can be met, or a reasonably narrow parameter estimate can be reached. In the case of an interim data analyses plan (e.g., Frequentist of Bayesian sequential analyses) a final stopping rule (e.g., Bayes factor, alpha-level) for the data collection should be outlined.
- Provide sufficient details of the planned materials and experimental procedure such that a full assessment of the study can be performed, and that researchers have all the information they need to independently replicate the study. All materials should be available to the reviewers in S1 via pre-registration specifics (e.g., OSF).
- Provide a complete description of all measures taken, even if some of the measures are not central to the main hypothesis or included in planned analyses. Provide a rationale for collecting additional variables to those required for the main test of the hypothesis.
- Incorporate a description and justification of planned analyses. Please note that Post-hoc tests or additional and exploratory analyses will be allowed in the final manuscript, as long as these are clearly identified as such, where possible with appropriate Type 1 error corrections.
- We advise that you include manipulation check and/or any other verification check of the outlined procedure. Please provide a full description of such a check, as the reviewers will assess the extent to which this check is feasible and could help explain any unpredicted results obtained after data collection.
Reviewers will be asked to assess the S1 manuscript on significance of the study, clarity and rationale for the aims and predictions, soundness and feasibility of proposed methods, and in the case of a (conceptual) replication, how well the original experiment is reproduced, or the motivation for the difference(s) in experimental procedure. The decision can be to ‘reject’, ‘revise’ or ‘in principle accept (IPA)’ the S1 manuscript. An IPA means that the manuscript is accepted for publication irrespective of the findings, as long as the agreed methods and procedures are adhered to and the outcome neutral condition has been met.
Once the S1 manuscript has been accepted in principle, the manuscript enters Stage 2 (S2) and the authors can start with data collection, in accordance with the approved methods and procedure. Should any deviations from the agreed methods and procedure occur due to unforeseen circumstances, this needs to be reviewed and approved by the editorial board prior to data collection. A substantial change to the methods may result in a decision to reject and invite to resubmit the manuscript at S1. It is important that the authors do not collect data ahead of this time, and time-stamped data or log files will be requested upon the submission of the full article.
Stage 2 (S2):
Submit a full report of the study, in accordance with APA guidelines, and include a cover letter.
Cover letter: In the cover letter, authors must collectively certify that all data was collected after admission to Stage 2. A basic laboratory log or electronic data files must be provided or made available along with the pre-registration document, outlining the range of dates during which data collection took place.
Introduction and Method: Note that the Introduction and Methods should not change, unless prior approval from the Editorial team has been obtained. Highlight such changes in the text.
Results: Report the results of the analyses that were registered at S1, using exact p-values and effect sizes. Do not deviate from the analytic strategy outlined in S1, unless good reason can be brought forth and prior approval from the Editorial team has been obtained. If any additional, exploratory analyses have been performed and are considered worthwhile reporting, please include these under a heading of “post-hoc analysis” or “exploratory analysis” and justify the need for these in the manuscript, including appropriate correction for multiple comparisons if possible. Note that the review process may lead to alterations or suggestions of omission of the reporting of these exploratory analyses.
Discussion: Return to the registered hypotheses. In the case that post-hoc analyses were performed, clearly indicate that the findings were obtained from post-hoc statistical tests.
Submission of time-stamped data and/or log-files: We strongly encourage authors to make their raw data freely available to the science community (either via OSF, or another data repository, as long as this repository will be permanently accessible). Data files should be time stamped to show that data were collected after S1 approval and not before. Other than pre-registered data, no data acquired prior to the date of IPA (In Principle Acceptance) is admissible in the S2 submission. Raw data are to be accompanied by guidance notes and relevant analysis scripts, where required, to assist other scientists in replicating the analysis pipeline. If the authors cannot – for ethical or other practical reasons – make the entire raw dataset freely available, a subset of time-stamped data can be accepted for verification purposes. If the latter is applicable, please seek prior approval from the Editorial team.
The Editorial team will invite the original reviewers, although new reviewers may be asked in case one or more of the original reviewers are not available. The reviewers will be asked to assess whether conditions for the outcome-neutral checks are met and findings are interpretable, whether the Introduction and Methods are the same, whether any post-hoc analyses are sound, and whether the conclusions are justified (see Instructions for Reviewers for more detail). If the authors decide to withdraw their submission after the completion of S1, they agree that Cognition and Emotion publishes this information with a short explanation written by the authors.