The mishandling of scientifically flawed articles about radiation exposure, retracted for ethical reasons, impedes understanding of the scientific issues pointed out by Letters to the Editor

1. Introduction

In this commentary, we examine the treatment by the scholarly journal Journal of Radiological Protection (JRP) of critical comments on two papers published by JRP.^1,2 The two papers (henceforth the Papers, and Paper 1 and Paper 2 for separate reference) were both jointly authored by Makoto Miyazaki and Ryugo Hayano (hereafter abbreviated as M&H). The papers examined individual external radiation doses of Date City residents after the Fukushima nuclear power plant accident.

The Japanese government utilized additional yearly radiation doses to determine which areas were contaminated and to design decontamination plans. The additional doses were estimated on the basis of the ambient dose (μSv/h) with the assumption that a resident would be spending eight hours outdoors and sixteen hours in a wooden house with a shielding coefficient of 0.4 each day, as expressed in the following formula.⁵

This formula has important social and political consequences, because if the coefficient were smaller than $0.6,$ the necessary amount of decontamination effort would be reduced (as a political decision). In Paper 1, M&H claimed the actual coefficient to be $0.15,$ by comparing individual doses measured with Glass Badges and the ambient doses estimated using airborne surveys. Based on this result, in Paper 2, they estimated the lifetime dose by extrapolating the ambient dose to seventy years.

After the Papers had been published, a number of serious issues were pointed out, which can be broadly classified as “scientific” and “ethical”:

• Scientific issues include, among others, anomalies in box-and-whisker plots, existence of figures for which no corresponding data were confirmed to exist in official records, unfounded underestimation of lifetime doses, unsupported denial of the effect of decontamination, and confusion between median and mean. These were pointed out in four Letters to the Editors^6–9 submitted to JRP. The first Letter (Letter 1) was officially accepted. The other three Letters (Letter 2, Letter 3 and Letter 4) were all provisionally accepted after peer review.

• The main ethical issue concerns the use of personal dose data without consent. This was pointed out by local residents of Date City, who were the subject of the studies of the Papers.

JRP retracted the Papers on 28 July 2020, citing as grounds only the ethical issue noted above. It then refused to publish the already formally or provisionally accepted Letters. The argument against publication of the Letters by JRP was thin and inconsistent, as we will see in Section 5.1. The scientific issues are serious, they are distinct from the aforementioned ethical issue, and they were not resolved by the retraction of the Papers due to this ethical issue. The treatment of the Letters by JRP hindered healthy scientific communication, the promotion of which is supposed to be the mission of scholarly journals. The handling of the Papers and the Letters by JRP is at best questionable. We give a summary of the problems involved in this process in Box 1.

The study reported in the Papers was carried out upon request by the then Date City mayor,¹⁷ and the results were sent prior to publication to the Chairperson of the Nuclear Regulation Authority of Japan.¹⁸ Following publication, the invalid conclusions in the Papers were mentioned by the National Council of Radiation.¹⁹ Furthermore, the Date City mayor also referred to the results of the Papers (while not explicitly citing them) and suggested that no decontamination was needed for a large part of Date City.²⁰ This shows that there was a huge political interest in such results, and indeed, the Chairperson of the Nuclear Regulation Authority insisted that the collected data in the Papers were still valid even after many problems with them were pointed out.²¹ In this situation, it was not only scientifically essential but also socially critical for JRP to carefully scrutinize the content of the Papers. If it turned out that the results were invalid, this should have been clearly recorded and shared. However, as we will see, JRP failed to do so.

The rest of the present paper is organized as follows. In Section 2, we review the relevant background and the status of the Papers and the Letters and clarify the ethical issues in the study, including the one that led to the retraction of the Papers. Section 3 summarizes the main scientific problems in the Papers that are reported in the Letters. In Section 4, we summarize the investigations into the Papers that were carried out at three relevant institutions. These sections together give the background against which and framework within which the main issue, i.e. the handling of the Papers and the Letters by JRP, is evaluated, which is addressed in Section 5. Section 6 concludes our discussion.

2. The Original Papers and Letters to the Editor

We give here an overview of the status of and issues in Papers 1 and 2,^1,2 along with a summary of the events related to them. It is necessary to reconstruct the course of events based on primary sources, including official Date City documents produced before the issues in the Papers were revealed, because the Papers contain a number of errors and incorrect statements, the official statements by the universities that carried out investigations into the Papers are insufficient and in some parts incorrect, and the statements by JRP and its publisher IOP Publishing (IOPP) turned out to be incorrect or inconsistent (as we will see in the later sections). See also the timeline in the supplementary material.

2.1. Background

The Fukushima Daiichi Nuclear Power Plant accident, triggered by the Great East Japan Earthquake on 11 March 2011, released a large amount of radioactive material that contaminated a wide area. Many residents in the areas close to the power plant evacuated, while some municipalities did not order evacuation and the residents remained in their homes. Date City, in Fukushima Prefecture, is situated about fifty kilometers northwest of the power plant. A part of the city was heavily contaminated and was declared to be one of the “specific spots recommended for evacuation” (estimated 20mSv/y as of 30 June 2011), while in other parts of the city the radioactive level remained relatively low. In August 2011, Date City distributed individual radiation dosimeters (Glass Badges), supplied by Chiyoda Technol Co., to children and pregnant women. This monitoring program was then extended to citizens of all ages.^{22 (Chapters 2,3)}

M&H used the data collected from Glass Badges to write the two Papers. The protocol for this research was submitted by Akira Ohtsuru (the then head of the department to which Makoto Miyazaki belonged) to Fukushima Medical University (henceforth FMU) on 2 November 2015, and it was approved by FMU’s ethics committee on 17 December 2015. Paper 1 was submitted to the Journal of Radiological Protection (JRP), the official journal of the Society for Radiological Protection (of the United Kingdom) on 18 August 2016 and was published on 6 December 2016. Paper 2 was submitted to JRP on 8 January 2017 and was published on 6 July 2017. These Papers were retracted on 28 July 2020 on ethical grounds. However, as we will see below, the course that the Papers took is characterized by many anomalies and unusual events that go beyond the ethical problem that led to their retraction.

2.2. Noncompliance with the Protocol

If a study involves human participants, the researchers are required to write a protocol, to submit it to the ethics committee of their institution for approval, and to follow it closely in the course of the study. In Japan, the Ministry of Education, Culture, Sports, Science and Technology (MEXT) and the Ministry of Health, Welfare and Labor (MHLW) have issued Ethical Guidelines for Medical and Health Research Involving Human Subjects.²³ In addition, universities and research institutes usually have their own local rules. Any study involving human participants can start only after the approval of the protocol by the ethics committee of the research institute. This rule was violated by the authors of the Papers.

First of all, Glass Badge data were provided to M&H by Chiyoda Technol Co. on 20 February 2015,¹⁷ well before the protocol was submitted. These data contained the Glass Badge data from August 2011 to June 2014, together with personal information such as the name, date of birth, sex and address of the participants. Officially, the data were provided to the authors to improve the analysis methods of the company. However, M&H analyzed these data and communicated the results to Date City using figures almost identical to those that later appeared in the Papers.

On 12 August 2015, Date City provided Glass Badge data from July 2012 to June 2014 together with personal information related to decontamination to M&H (these data were supposed to have been anonymized, but it was suggested that Ryugo Hayano had data including precise addresses of the individuals.^{17 (p.8)} A letter requesting that the data be analyzed and the results written up in academic papers was sent from the then mayor of Date City to Makoto Miyazaki on 23 October.^{17 (p.3)} Interestingly, the letter was dated 1 August, i.e. before the data were actually given to M&H by Date City. That the date had been falsified was confirmed.^{17 (p.5)}

On 21 October 2015, Hayano sent preliminary results to the then Chairperson of the Nuclear Regulation Authority (NRA) of Japan,¹⁸ saying that they were writing up papers based on the data from Date City and that he would provide these to the Chairperson of the NRA because the results could be useful for regulation purposes.²⁴

All this occurred before the authors submitted the research protocol to FMU (2 November 2015), indicating that M&H had started and finished most of the analysis for the paper before they obtained the ethical approval. Although FMU confirmed this fact,²⁵ it decided that these acts by M&H did not constitute a grave infringement or negligence and cleared them of the allegations.

In addition, M&H had planned in their protocol to study the correlations between personal external and internal doses. This study has never been published, despite the fact that Miyazaki said that they did not find significant correlations.^{26 (e30021, meeting minutes)} This is puzzling, because the absence of correlations was the expected result according to the protocol. Not publishing the result of a conducted study is also a violation of the research protocol, and also goes against the ICMJE recommendations²⁷ as well as the Declaration of Helsinki.²⁸

2.3. Ethical Problems

Apart from the incongruence with the research protocol, the Papers have many serious ethical problems. The most important one is that many of the citizens of Date City whose radiation dose data were provided to the authors did not give consent for the data to be used for research. The dose data given to M&H (that have been made available through a Freedom of Information (FOI) request²⁹) contained a column that shows consent. The negligence of the column clearly goes against any ethical standard, and violates the research protocol, FMU’s ethics code and MEXT and MHLW’s Ethical Guidelines for Medical and Health Research Involving Human Subjects.

In addition, although M&H had promised in their protocol²⁹ to inform the citizens about the study through the Date City website and the city journal, they did not do so. The citizens of Date City were not informed of the study or given opportunities to revoke their consent for their data to be used for research.

Furthermore, M&H did not disclose their conflicts of interest properly. Makoto Miyazaki was a policy adviser to Date City, and he analyzed data and published papers in international journals to publicize the results at the request of the then Date City mayor. The former is not declared in Paper 2, while the latter is not stated in either of the Papers. (Paper 1 states that “Date City mayor’s office entrusted the data to the authors” in its Ethics statement, but it is unclear whether the research was requested by the City mayor or not). As with any reasonable editorial policy, IOPP, the publisher of JRP, requires authors to disclose conflicts of interest, including consultancies, in an acknowledgements section.³⁰

It should also be noted that the authors declared in the protocol that they would delete the research data after the completion of the study, rather than conserving them. Although earlier guidelines on epidemiological research by MEXT and MHLW required deletion of data after a study, by the time of the submission of the protocol to FMU (2 November 2015), these guidelines had been replaced by the Ethical Guidelines for Medical and Health Research Involving Human Subjects (whose article on data conservation had taken effect on 1 October 2015).²³ The new guideline requires the conservation of research data for as long as possible. Accordingly, FMU updated the rules for its Ethics Committee and for research misconduct in May 2015. This means that the authors had to follow the national-level and university-level guidelines and to conserve their research data. Instead of following these binding guidelines, however, they claim that they deleted the data on 23 October 2019.²⁵

2.4. Letters to the Editor

In August 2019, Shin-ichi Kurokawa (SK) submitted Letter 1 to JRP, pointing out several issues in Paper 2 including the contradiction between Figures 6 and 7, inconsistencies in various parameters and possible underestimation of the lifetime doses. It went through a review process and was provisionally accepted in November 2018. As the authors of the criticized paper, M&H were invited to respond to Letter 1. Instead of submitting an answer Letter to clarify the points raised in Letter 1, however, M&H proposed to JRP that they would submit a corrigendum. JRP agreed to their proposal and decided not to publish Letter 1. This is unusual, because this would mean that JRP would not publish a (provisionally) accepted work on request by the authors of the original papers that the work criticizes. Upon protest by SK, JRP reversed its decision and promised that it would publish Letter 1 even if the original paper were retracted. Letter 1 was formally accepted on 18 March 2020.

In the meantime, investigations by the University of Tokyo (UTokyo) and FMU had begun. The investigations by these universities concluded in July 2019.

Letters 2, 3 and 4 were submitted to JRP between January and March 2020. The Letters pointed out numerous technical issues in Papers 1 and 2 (details are given in Section 3). These Letters were all provisionally accepted by April 2020. Although JRP told the Letter authors that it was waiting for a reply from the original authors (M&H), it later claimed on various occasions that the Letters had been “on hold” because the investigation by Date City was ongoing.

Eventually, despite the fact that all the Letters were officially or provisionally accepted, JRP reversed its decision and withdrew all Letters, claiming that “it was not felt possible to independently verify with certainty whether or not the four Letters were free of the use of the ethically inappropriate data.” The fact is that it did not conduct any verification, as we will see in Section 5.1 in detail (see also the timeline in the supplementary material).

3. Errors in the Original Papers

Before elaborating on the issues with JRP’s editorial processes, we summarize here the technical problems in Papers 1 and 2. The main conclusions of the present paper do not depend on the details in this section, and the reader may skip it if s/he is mainly interested in the problems with the editorial processes. The Papers contain obvious errors and design flaws that can be verified by checking only the figures and tables in the retracted papers and public documents released by Date City, without looking at the original unconsented data. These issues were pointed out in the Letters.^6–9 As explained in 2.4, all the Letters had been provisionally accepted (the first Letter⁶ was formally accepted), and the referees agreed that the authors would have to provide explanations for the issues pointed out in the Letters. Yet, JRP retracted the original Papers solely on ethical grounds and without mentioning most of the scientific errors.

We will outline below some of the most important flaws pointed out in the Letters that affect the main conclusions of the Papers. Table 1 summarizes the major issues. These are in no way exhaustive.

3.1. Inconsistencies in the Box-and-whisker Plots

A box-and-whisker plot displays the distribution of data by showing certain percentile values. In the six panels in Paper 1,^{1 (Figure 4)} each plot in the panel shows the distribution of the Glass Badge data of the participants who are registered in an area with a certain ambient dose rate in a period, and the upper whisker shows the 99th percentile, as claimed in Paper 1.^{1 (First paragraph, Section 3)} Outliers are those who had higher than the 99th percentile value or lower than the 1st percentile value.

In Figure 4a, the bins with 1.35 and 1.45 µSv/h contain less than two-hundred and four-hundred participants respectively. Nevertheless, there are four and six points indicating upper outliers above the upper whisker. This is impossible, because there should be only two and four outliers, respectively, for these bins. There are similar issues in other figures. This shows that the plot of these figures is unreliable. Similar inconsistencies in the percentile values are found in several other parts of Papers 1 and 2.

3.2. Figures for a Period with no Official Record of Data Provision

Figure 4f of Paper 1¹ is supposed to be a plot of the Glass Badge data for the period 2014 Q3 (from October to December). In Table 1,¹ “2014 3Q N $= 21080$” is written above the age-distribution histogram that corresponds to the period 2014 Q3. However, a public document of Date City³¹ states that the number of Glass Badges distributed to citizens of Date City for the period October 2014 to December 2014 (2014 Q3) was $16,037$ and the percentage for these citizens who returned the Glass Badges was about 90%. The number of the participants should then be about $14,500.$ This inconsistency with the claimed N $= 21080$ suggests that the data used to plot Figure 4f were incorrect.

A closer look at Figures 4e and 4f suggests that the data for Figure 4f can be partly the same as that for Figure 4e, as pointed out in Letter 3.^{8 ((2)-(5))} This is a serious issue, because these indicate that Figure 4f cannot have been created from the correct dataset. If the authors used the same dataset also for Paper 2, then Figure 5 therein which contains the period 2014 Q3 must be wrong as well.

The fact that there is no public record that the data for 2014 Q3 were provided to the authors corroborates this point. An email from a Date City clerk to Miyazaki on 3 August 2015 said that only the Glass Badge results until June 2014 had been included in the database and suggested that Date City provide the latest data (July 2014 to June 2015) in autumn.³² The latest data provided to the authors was dated 12 August.¹⁷ The authors thus should not have had the data for 2014 Q3. This is supported by another document,³³ most probably created by the authors and given to Date City, showing a histogram identical to that of 2014 Q3 of Table 1 in Paper 1 whose title reads “GB period 2013/10-12: The 9th airborne monitoring (2014/11/7).” This document clearly states that the Glass Badge data in 2013 and the airborne monitoring data in 2014 are compared. This strongly suggests that M&H knew that the Glass Badge data were not for the period 2014 Q3, contradicting JRP’s claim that “the authors were not aware of this mistake in advance of publication of the article” in their retraction notice.³

3.3. The Wrong Estimate of Lifetime Doses

It has been pointed out³⁴ that the plots of the Glass Badge data in Figure 7 of Paper 2² were wrong and smaller by a factor of 0.46 compared with the actual value. This has been acknowledged in the investigations by the universities.^25,35 JRP issued a statement on this issue by modifying the Retraction Notice⁴ without the record of revision histories and without clarifying whether this affected the main results of Paper 2 or not. (It does, as we will see immediately.)

FMU claimed that this error did not affect the estimates of lifetime doses.²⁵ The error does indeed affect the estimate of lifetime doses, as pointed out by some of the Letter authors.³⁶ As the estimate of lifetime doses is given by integrating the theoretical function that predicts the reduction of the ambient dose rate, if the cumulative dose data were multiplied by $2.2 = 1/0.46,$ then the theoretical function must be multiplied by 2.2 as well in order to fit the median of the cumulative doses. Consequently, the estimates of lifetime doses should also be multiplied by 2.2. It seemed that the members of the FMU investigation committee failed to understand the logic of estimating the lifetime doses. Their incorrect recognition that there were no mistakes in the lifetime dose estimates was one of the reasons why FMU judged that the error was not intentional. As it is evidently false, their judgement that there was no scientific misconduct loses its basis.

3.4. Unsupported Conclusions on the Effect of Decontamination

In Paper 2, the authors assess the effect of decontamination in Zone A of Date City carried out in 2012 Q3, but their arguments ignore most of the data available. The authors conclude, by picking only two periods among ten (before and after the decontamination) and assuming a single reduction function throughout the whole period, that “effects of decontamination on the reduction of individual doses were not evident.” This conclusion is unreasonable: the authors should have compared the dose rates before and after the decontamination and fitted each period by an a priori different function.

As a matter of fact, the effect of decontamination is visible if one compares the upper whiskers of 2012 Q2 and Q3 (17th and 20th months) in Figure 6 of Paper 2.² One of the authors (Ryugo Hayano) in fact reported in a symposium on 13 September 2015 that there were effects of decontamination in Zone A, by showing the same graph.^{37 (13:00~)} Hayano also stated that the effect of the decontamination in Zone A was ∼ 60% in their slides sent to the Chairperson of the Nuclear Regulation Authority,^{24 (p.14)} before publication of Paper 2. This shows that M&H changed their conclusion by omitting the analysis that showed a reduction of 60%.

3.5. Other Issues

The Letters pointed out a number of other issues in Papers 1 and 2.^6–9 Some of the issues that substantially affect the conclusions and/or suggest further ethical issues are:

• Some of the participants in the study were evacuated, while their official residence remained in Date City. Therefore, their Glass Badge data cannot be compared with the ambient dose rates of their registered addresses.

• It was reported that a majority of the participants left the Glass Badge at home, hence the Glass Badge data did not reflect the actual radiation doses of the participants.

• The Ethics Statement of Paper 1¹ is wrong: the household addresses were not “pseudo-anonymized” as claimed.

3.6. The Referee Reports and the Replies from the Authors and JRP

As we have seen, both Papers^1,2 contain numerous errors, inconsistencies, logical and design flaws, and ethical issues. The Letters pointing out these issues were submitted to JRP and they have been either (provisionally) accepted without revision or have received positive reviews and then been (provisionally) accepted. The reviewers recommended that the authors should clarify the issues pointed out by the Letters. Therefore, JRP was and is aware of all these problems, yet it failed to acknowledge most of the issues publicly. Instead, when Letter 5 (which points out the issues of the Retraction Notices and the Publisher’s Note) was submitted, JRP refused to consider it for publication, while admitting that at least one of the points in the Letter was correct.

4. Three Investigations: FMU, UTokyo and Date City

A Date City resident submitted letters of allegations, requesting UTokyo (in December 2018) and FMU (in January 2019) to conduct formal investigations into research misconduct by the authors of the Papers. The allegation contained claims concerning violations of the ethical guidelines and some scientific anomalies in Figures 5A, 6, and 7 of Paper 2. These investigations concluded on 22 July 2019, without addressing fully the raised questions. Date City started its investigation on 4 February 2019 on the handling of personal information of residents, and concluded that the data provided to M&H contained the dose data of the residents who did not give consent to the study. This section summarizes these three investigations.

4.1. Investigation by FMU

The allegation by the Date City resident contains claims regarding both violations of the ethical guidelines and technical issues concerning Figures 5A, 6, and 7 in Paper 2. Specifically, it pointed out that Figure 7 and Figure 6 contradict each other, as do Figure 5A and Figure 6, and Figure 5A contain more outliers (defined to be above the 99th percentile) than the 1% of the participants (see Section 3 and Table 1 for details).

The following is an extract from the FMU report concerning these Figures²⁵ (the translation is from article³⁸):

In comparing the claims of the alleger and the alleged, the following was determined.

(1) Review of Paper 2 shows that the error pointed out by the alleger corresponds to Fig. 7.

(2) When creating Fig. 7, the authors converted the individual dosimeter data from the 3-month cumulative dose to the dose rate per hour $(/3/24/30.5 ∗ 1000(= 0.455))$ just as they did in Fig. 6, even though the conversion was unnecessary for Fig. 7.

(3) The value of the estimated lifetime doses shown in the conclusion of Paper 2 is reasonable, and there is no underestimation of individual radiation doses as claimed by the alleger.

This conclusion is problematic for several reasons. Firstly, the issue of outliers is ignored. Secondly, Figure 5 contains the same mistake of multiplying the three-month cumulative dose by a factor of 0.455 (or about 0.5), but FMU does not admit this. Thirdly, the estimate of lifetime doses is wrong (see Section 3.3). Furthermore, FMU cites one of the articles written by two of the Letter authors³⁴ as a reference in which many other issues in both Papers were pointed out, but it did not discuss any of them. FMU then decided that the problem in Paper 2 was just a simple error rather than a fabrication. As for the ethical guidelines, FMU admitted in its report that the research protocol had not been followed closely,^{25 ((6)1)} but largely blamed Date City for the improper handling of the data. FMU concluded that the acts of M&H did not constitute research misconduct.

Altogether, the report by FMU failed to recognize the scientific issues, including conclusions not supported by the data. The reason why the researchers were judged not guilty, i.e. there is no underestimation of lifetime doses, is inconsistent with the graphs in Paper 2.

4.2. Investigation by UTokyo

The UTokyo Committee only published a very brief summary of its conclusions.³⁵ It did not investigate the ethical issues, with the excuse that they were not within the mandate of the investigation committee on scientific misconduct. Regarding Figure 6, they stated³⁵ (the translation is from an article by two of the Letter authors³⁸):

Concerning the discrepancy between the slides from the seminar (cited as 1 in the Appendix) and the paper (cited as 3), because the vertical axis in the slides was intended to show the individual dose rate (Sv/h), the values obtained from the raw data (cumulative doses for 3 months in mSv) should have been multiplied by 0.455 (/3(months)/30.5(days)/24(hours) ∗ 1000). But we confirmed that this was not done. We further confirmed that this conversion was done for the values of the vertical axis in Fig. 6 in the paper cited as 3 in the Appendix.

As for the discrepancies among data in the paper cited as 3 in the Appendix, the values of the vertical axis in Fig. 7 represent the Cumulative Dose [sic] (mSv) and they should have been multiplied by 2.2, which is the inverse of 0.455 mentioned above. We confirmed that the alleged researchers forgot this multiplication when conducting computations from Fig. 6.

The report by UTokyo, in the same way as the report by FMU, failed to recognize the error in Figure 5 and ignored the underestimation of the lifetime doses, which had been clearly alleged by the Date City resident (see the expository article³⁸ for details). Similarly to the FMU report, the UTokyo report also trivialized the scientific problems in Paper 2 as a single, unintended mistake. They did not make sufficient effort in their investigation and therefore overlooked many issues that they should have identified. This should have been clear from the large number of serious issues pointed out in the Letters (which are partially covered in Section 3 in this paper). The UTokyo committee did not investigate ethical problems, without clarifying who was responsible for dealing with ethical issues.

4.3. Date City Investigation

Date City set up an investigation committee on the handling of the personal data. The investigation started on 4 February 2019. The scope of the investigation was the examination of administrative procedures and handling of personal information by Date City. The committee’s mandate did not include the examination of the contents of the papers. The investigation concluded on 17 March 2020. It did not determine which data had been provided to the authors and where the CD-Rs containing the personal data were now (this was followed up by an additional investigation by Date City council, although the latter did not find much more because most of the people questioned by the council did not answer its questions). In any case, it confirmed that the data provided to the authors contained that of individuals who did not give consent for the study, and that personal information was illegally provided to M&H.¹⁷

5. Retractions and Consultation with COPE

5.1. JRP’s Handling of the Papers and Letters

As we saw in Section 4, the university investigations did not sufficiently examine either the problems in the allegation or the problems pointed out in the Letters and brought to their attention. In such a situation, it is the duty of JRP to verify the claims made in the Letters and to determine whether the original publications contain scientific problems, given that the Letters were all officially or provisionally accepted. This did not happen.

Overview of what took place in the handling of the Papers and Letters by JRP

The handling of the Papers and the Letters took a rather unusual course (see supplementary material for the timeline). As we already saw in Section 2.4, after the publication of Papers 1 and 2, Shin-ichi Kurokawa (SK) submitted a Letter to the Editor (Letter 1) about Paper 2 on 18 August 2018. This Letter was provisionally accepted, or became “ready to be accepted,” and the authors of the Papers (M&H) were invited to respond to the Letter on 16 November 2018. After some correspondence between JRP and SK, JRP promised to publish Letter 1 even if Paper 2 was retracted. The university investigations started on 18 December 2018 and 19 January 2019, and the Date City investigation started on 4 February 2019. Meanwhile, JRP issued an Expression of Concern about both Papers on 11 January 2019. The university investigations concluded on 19 July 2019. Letters 2, 3 and 4 were submitted between 30 January 2020 and 1 March; all of them were provisionally accepted by 7 April 2020, while Letter 1 was officially accepted on 24 March 2020. The authors of the Letters were notified that the authors of the Papers had been invited to reply, with no reference to anything about the Letters being “on hold” (see Section 5.2) or to the Date City investigation. Meanwhile, the Date City investigation concluded on 17 March 2020. JRP retracted Papers 1 and 2 on 28 July 2020 on the basis that they used unconsented data, and decided to withdraw the four Letters as well.

As we mentioned in Section 2.4, it was already unusual that the authors of the Papers proposed writing a corrigendum, instead of submitting a point-by-point reply to Letter 1, and that JRP accepted it. If that had happened, the content of Letter 1 would have been published with M&H as the authors, even though it was SK who pointed the content out in a provisionally accepted Letter to the Editor. This was only reversed after SK wrote to JRP several times.

JRP’s handling of the Letters

JRP’s handling of the Letters after the Papers were retracted was even more inappropriate. JRP resorted to various inconsistent reasonings not to publish the Letters. We identify several serious problems in JRP’s explanations for its decision:

• JRP’s justfications for withdrawing the Letters changed over time and were inconsistent. On 8 July 2020, JRP wrote to the Letter authors that they would withdraw the Letters “on the same basis, i.e. that they are drawing scientific conclusions based on a dataset that was given without participant consent.” This was simply false in two ways. First, the Letter authors did not have the original dataset nor refer to it. Second, the Letters pointed out problems in the Papers but did not draw scientific conclusions on the radiation issues. Upon inquiry by the Letter authors, JRP changed its explanation and said, on 14 July 2020, that “there is a serious problem in that they comment upon, and draw conclusions about, papers that have been retracted due to the use of unconsented data.” This reasoning is hard to understand, to say the least, because accepting this reasoning would make it possible to hide scientific flaws by resorting to ethical issues in data.

• The Letter authors then appealed to IOP Publishing (IOPP, the publisher of JRP), but it withdraw the Letters. JRP stated in its Publisher’s Note¹⁰ that “it was not felt possible to independently verify with certainty whether or not the four Letters were free of the use of the ethically inappropriate data.” The fact is that JRP made no effort to “verify with certainty” whether the Letters used the unconsented data or not. They did not ask the authors of the Letters any questions regarding how the Letters were written. There was no confirmation that JRP consulted the reviewers of the Letters, either.

• The Letters did not use the original unconsented data, but only the figures, tables and the text of the Papers and public documents obtained through Freedom of Information requests. This can be confirmed from the content of the Letters themselves.

As of April 2022, JRP has acknowledged only three of the scientific issues pointed out in the Letters^3,4: the one regarding the data of 2014 Q3 (pointed out in Letter 3, see Section 3.2), errors in the plot of Figures 6 and 7 of Paper 2 (pointed out in an article in Japanese³⁴ and in Letter 4, see Section 3.3), and a missing normalization factor (pointed out in Letter 1 and the explanation about the missing factor is wrong; only a normalization factor without replacing $t$ by $t - 0.65$ is needed). These confirmations were added without the permission of the Letter authors, but also are inconsistent with JRP’s claimed policy that the unconsented data cannot be referred to in any situation. Furthermore, it is unclear when these issues were added to the retraction notices. (JRP modified them without a clear record, and the initial version of the retraction notice of Paper 2 did not contain the issues of the normalization factor and the plot). All the other issues pointed out in the Letters remain unaddressed.

JRP’s Retractions and Publisher’s Note

The Retraction Notices^3,4 and Publisher’s Note¹⁰ contain a number of incorrect statements about the Letters, the investigations, the ethical policy of FMU, and other issues¹¹. The most important ones are as follows:

• The summary of the report by the Date City Citizen’s Exposure Data Provision Investigation Committee is incorrect, and the links in the Retraction Notices^3,4 to the reports are wrong. This clearly shows that JRP did not verify the report, but rather trusted what the authors of the Papers told it without necessary – or any – verification.

• The retraction notice³ claims that a number in a figure in Paper 1 was wrong. This was one of the issues pointed out in Letter 3. The Letter authors told a journalist about this, who subsequently made an inquiry to Miyazaki (FMU admitted that Miyazaki came to know about this through the journalist³⁹). Here, JRP included a finding in the Letters in its Retraction Notice without the permission either of the Letter authors or of the journalist. Worse still, this issue with the number means that a part of the dataset used for both Papers 1 and 2 was wrong, which affects almost all findings in the Papers. The Letter authors informed JRP of this,¹¹ but JRP refused to acknowledge it.

• As we summarized in Section 2.3, FMU updated the rules for its Ethical Committee and for research misconduct in May 2015. M&H were thus required to conserve the data for at least ten years. The claim that “[the authors] said that this was required by the ethical policies of Fukushima Medical University” only shows that JRP trusted them without minimal verification, again.

• JRP claimed that it had put the Letters “on hold” while the Date City investigation was ongoing. This cannot be true, at least for Letter 1, because it had been provisionally accepted before any of the three investigations started. Furthermore, Letter 1 was formally accepted on 23 March 2020. (This would be reversed later, when JRP learned of the existence of the Date City investigation.)

As the Letters were provisionally accepted, JRP should have known that the Papers contained a number of serious scientific problems that critically affected the Papers’ conclusions, many of which were not identified in the university investigations. Nevertheless, it did not scrutinize most of them, but rather published what the authors told JRP as is. Such a handling of issues is inappropriate, as we have shown, because the statements of the authors were at least partially untrue.

Given this situation, the Letter authors submitted Letter 5, which pointed out these problems in the retraction notices and the Publisher’s Note. JRP rejected it without peer review, claiming that “the matter is now closed.” However, the course of events outlined above shows that JRP understands neither the content of the Letters nor the nature of the three investigations. It is entirely illegitimate to call a case closed when the editors have no grasp of it.

5.2. IOPP’s Consultation with COPE

After some correspondence with the Letter authors concerning the withdrawal of the Letters, IOPP consulted the Committee on Publication Ethics (COPE) for advice on 4 September 2020. COPE is an organisation “committed to educating and supporting editors, publishers and those involved in publication ethics with the aim of moving the culture of publishing towards one where ethical practices become a normal part of the publishing culture.” COPE’s members consist of scientific journals and companies. JRP is a member of COPE, and IOPP states in its “[E]thical policy for journals” that IOPP applies “the principles of publication ethics outlined in the COPE Core Practices.”⁴⁰ On 26 October 2020, IOPP informed the Letter authors of COPE’s advice following IOPP’s consultation with COPE.

IOPP’s request to COPE for advice, the resulting document of which the Letter authors received from IOPP, was vague and contained misleading descriptions. It claimed that the Letters “were placed on hold while the investigation was being completed,” which was not the case, as we saw in Section 5.1. Most surprisingly, IOPP did not tell COPE that the Letters pointed out errors in the Papers, but instead said that the Letters “were heavily based on the papers and cite data, figures, and equations.”

Based on these incorrect and misleading explanations of the status and content of the Letters, IOPP asked COPE for its advice on two points:

1. “Was the journal right to withdraw the accept offers for the four letters?”

2. “Is there any specific guidance that covers articles that comment on retracted papers and what is permissible?”

The Letter authors received the statement of advice made by COPE from IOPP on 26 October 2020. (The original statement was not dated.) It said “[i]f the letters are directly and solely based on the results of the affected papers, rejection should be straightforward,” while “[i]f the letters cite the papers in the context of journalology, or criticize the same issues that underlie later retraction, it could be argued the letters could stay, with an explanatory editorial note.” IOPP, first without disclosing the content of its consultation with COPE to the Letter authors (such disclosure was eventually made on 9 November 2020), claimed:

In summary, we take COPE’s response to suggest two options:

1. The withdrawal of the Letters by IOP Publishing stands

2. The authors revise the Letters removing the affected material and references

It is difficult to interpret COPE’s reply in this way. It clearly allows the Letters to be published in some cases, either “in the context of journalology, or [if the Letters] criticize the same issues that underlie later retraction,” while it recommends rejection when letters are “directly and solely based on the results of the affected papers.” The Letters are not “based on the affected papers” but scientifically invalidate the Papers, at the same time they implicitly raise questions about the editorial decision to publish the Papers, which are full of errors, in the first place. Among these two categories, therefore, the Letters belong to the first category (journalology) and hence should be published.^[1]

5.3. The Letter Authors’ Consultation with COPE

Interactions among COPE, IOPP and the Letter authors

As the description given to COPE by IOPP misrepresented the situation and IOPP’s decision seemed an arbitrary interpretation of COPE’s opinion, the Letter authors consulted COPE about JRP’s decision not to publish the four accepted and provisionally accepted Letters. Their claim clarified the fact that the JRP/IOPP explanations not to publish the Letters had changed twice (see Section 5.1).

In relation to this consultation, COPE made two inquiries to IOPP, first on 23 November 2020 and then on 1 January 2021. Yet, in this inquiry, COPE ignored all the inconsistencies in IOPP’s explanations included in the Letter authors’ consultation document. In IOPP’s reply to COPE, JRP wrongly claimed again that the Letters had been “placed on hold pending the outcome of the investigation.”

The second inquiry from COPE to IOPP concerned “the circumstances and process” and “the error in Table 1.” On 12 January 2021, IOPP made a response, which again contained multiple anomalies, including the following:

• IOPP stated that the investigation confirmed that some of the data used in the Papers were gathered without participant consent, but the actual ethical issue was the use of the data not consented for research.

• IOPP claimed that the allegations made by the Letter authors were “unproven,” while in reality the Letters were accepted or provisionally accepted and the reviewer(s) recommended that the Paper authors should explain the issues, which the Paper authors have never done.

In addition, IOPP stated:

We are in the process of updating the retraction notice for the second article to acknowledge and correct two errors that were identified in the first Letter for which the authors have now provided some corrections.

As we saw above, this had been done without the permission of the Letter authors. IOPP admitted that the contents of the Letters needed to be recorded, and published them as “corrections.” It is hard to see a sensible reason why it was possible to publish the corrections but not the Letters. At some point in 2021, JRP modified the Retraction Notice⁴ of Paper 2 without a clear record and stated that the cumulative doses in Figure 7 should have been multiplied by 0.455. This was one of the findings of the Letter authors, first identified by SK in Letter 1, published with more details in an article in Japanese,³⁴ cited by FMU, and included also in Letter 4. The Retraction Notice mentions none of these but only thanks “the readers that brought these issues to their attention.”

COPE’s final report

From COPE’s inquiries to IOPP, it had already become clear that COPE had no intention of examining anomalies involved in the handling of the Papers and the Letters or the content of the Letters.

The final report made by COPE stated that

the journal followed an adequate process to handle the concerns raised about the reversal of the acceptance decision for the letters to the Editor.

The journal took the decision to reverse the acceptance following an institutional investigation which led to the retraction of the publications that the letters were related to, and in consideration of the journal’s documented processes which note that acceptance decisions can be reversed in certain circumstances. The publisher apologized to the authors of the letters and offered them the possibility of submitting a new letter to the Editor that did not rely on data reported in the retracted articles. The publisher has taken steps to update the retraction notice where this was established to be necessary to provide accurate information to readers. The publisher also sought advice from COPE in relation to the decision not to publish the letters.

The Facilitation & Integrity subcommittee agrees with the publisher that the processing of the letters should have been placed on hold when the journal was informed about an institutional investigation about the published studies, and welcomes the steps taken by the publisher to update their processes so that in future, if an investigation is ongoing, the processing of related submissions is paused.

Several extraordinary points can be observed in this judgment. First, COPE claimed that the journal “followed an adequate process to handle the concerns,” but this claim breached COPE’s own guidelines. According to “What to do if you suspect fabricated data (b) Suspected fabricated data in a published manuscript,” the editors must “Contact author’s institution requesting an investigation,” and if they obtain “no or unsatisfactory response”, they must “Contact regulatory body.”⁴¹ However, neither JRP nor IOPP followed this process. COPE nevertheless claimed that they followed “an adequate process.”

Also, COPE avoided the issue of the arbitrariness of the decision made by IOPP/JRP by using the vague expression, “the retraction of the publications that the letters were related to,” without clarifying the nature of how the Letters were “related to” the retracted papers. The statement “acceptance decisions can be reversed in certain circumstances” is also very vague. In sum, COPE decided not to check the arbitrariness and anomalies involved in the decision by JRP not to publish the Letters, ignoring all the evidences provided by the Letter authors.

COPE, in stating its final reply to the Letter authors that “[i]t is beyond the scope of this process to evaluate the scientific content of publications, or the content of notifications posted on an article that pertain to the content,” effectively admitted that it gave up on fulfilling its stated mission and responsibility; the validity of the formal process cannot be evaluated without minimally understanding the content of publications and of notifications. Without checking the validity of the dates given in the notification or the links, which COPE did not refer to in its conclusions, what one claims happened and what happened in reality cannot be distinguished. Even if COPE has no competence in evaluating the scientific issues, it is still puzzling that it avoided reference to the problematic aspects of IOPP/JRP’s handling of the case, including the fact that IOPP/JRP’s explanations of its decision not to publish the Letters changed, the date and link information given in the Retraction Notices were in part wrong, and some of the explanations in the Publisher’s Note and Retraction Notices were originally given in the Letters. It must have been clear, even with a superficial consideration, that IOPP/JRP did not “follow an adequate process.”

In addition, when the Letter authors asked COPE to specify its criteria for the use of “unconsented data,” COPE answered that “some of the data were unconsented such that none of the data should be used in any future analysis.” This again obstructs debate on scientific issues in papers retracted on ethical grounds.

5.4. Current Status of the Papers

Following this chain of events, the Publisher’s Note remains in place with a number of errors, and with no reference to any of the scientific issues in Papers 1 and 2 except three that were added without any credit to or permission by the Letter authors (see Section 5.1). The only official reason for the retractions remains the ethical issue, i.e. the Papers used unconsented data, and JRP has admitted none of the technical issues except the above three.

Although it had called the matter “closed,” following the provision of information from Date City Council, JRP made corrections to the Retraction Notices and Publisher’s Note^3,4,10 in January 2022. No detail was given about what had been corrected.

As the last example of arbitrariness in JRP’s editorial decision making, we point out that another paper was published by the same authors of the Papers,⁴² in which the results of Paper 1 and UNSCEAR data were compared. Although this paper depends entirely on the results of Paper 1 (because their main point was comparison), JRP has not retracted it, but instead agreed to keep it by removing references to Paper 1 and citing other papers with similar results, hence by changing the subject of the paper.

6. Conclusions and recommendations

Let us summarize the problems with the handling of the Letters by JRP:

• The original Papers contain a large number of errors that are easy to spot. This suggests that the peer-review process did not work properly.

• When JRP received Letter 1, it allowed the authors to publish a corrigendum, instead of requiring them to reply to the Letter point-by-point. This enabled the authors to ignore the issues pointed out in Letter 1 arbitrarily.

• Although the referee reports of Letters 2 and 4 recommended clarifying some of the issues, JRP has never done so.

• The excuse for not publishing the Letters, i.e. that any finding itself based on unethically obtained data cannot be published, is unreasonable. Indeed, JRP itself mentioned some of the issues pointed out in the Letters in the Retraction Notices. What is worse is that JRP did so without mentioning the Letters, as if it had identified the issues by itself.

All these (in)actions obstructed sound scientific debate, and the journal has failed to record the scientific problems of the retracted Papers, including possible misconduct.

Scholarly journals play an important role in the sound development of science. The established procedure of peer review, though not without problems, is essential in maintaining the quality of published papers.⁴³ As peer review gives certain authority to papers published in scholarly journals, peer-reviewed papers can influence policy decisions and public opinions as “scientific findings.” In this regard, it is very important that peer reviews are carried out in a reasonably rigorous manner. Perhaps even more importantly, when problems surface after the publication of papers, the journal that published the papers should provide an adequate space for scholarly debate, or allow public peer review. Depending on the outcome of this process, the paper should be corrected if the main body of the original paper is still valid after corrections, or retracted if its problems are serious. Scientific misconduct that goes beyond errors is sometimes revealed in this process. Most journals have a section designated for criticisms on papers they publish, typically “Comments” or “Letters to the Editor.” The authors of the criticized papers are generally expected to provide point-to-point responses to each of the specific items in the criticism. These procedures are essential for maintaining the scholarly standard of the journal and of the research.

In order to maintain the integrity of scholarly publishing, we believe that the following acts are necessary:

• In some journals, authors are allowed to list preferred and non-preferred referees during submission. Authors could utilize this system strategically to list referees who would provide favorable review for the manuscript. To prevent such abuse of a referee-nomination system, if a referee was selected from the list, this fact and the reviewer’s name should be disclosed. Moreover, in recent years, some journals, including Nature, have published review reports and rebuttals from authors to advance scholarly discussion.⁴⁴ Industry journals may be perceived as publishing only papers that are advantageous to the industry. To avoid such evaluations and to contribute to scientific progress, it is necessary to improve the review system and make it more transparent.

• Letters that point out methodological and analytic problems in retracted papers should be published, in order to prevent cases where the journal is not willing to disclose such problems.

• Accepted letters should be published even if the authors do not reply. There should be a clear deadline for the authors to reply, in order to prevent cases where the authors deliberately block the publication of the letters.

• Journals should not be allowed to publish the content of letters without publishing the actual letters.^[2] Such publication should be clearly identified as plagiarism by the journal.

• Journals should declare when authors have failed to disclose conflicts of interest.

When a journal does not take proper action, there should be an independent organization that investigates the case. In the present case, COPE should have played that role. As we have seen, however, it failed to do so. While it is understandable that COPE was not able to examine the technical issues of the paper, it was not difficult to confirm that JRP did not even minimally check the claims made by the authors of the Papers.

To prevent such a superficial procedure, we recommend the following:

• The independent organization should make sure that the journal followed a due process.

• The independent organization should accept appeal, where the alleger can point out that the inspection of the case was not satisfactory.

Disclosure statement, including conflicts of interest and funding

YH contributed articles related with this topic to a popular science magazine, received honoraria for them and is a member of Citizen’s Commission on Nuclear Energy: an independent research and advocacy body that seeks nuclear power phaseout since 2018. KK contributed an article related with this topic to a popular science magazine and received an honorarium for it. SK contributed articles related with this topic to a popular science magazine and various news magazines, received honoraria for them, received 1travel allowances and conference fees through KAKENHI (Japan Grants-in-Aid for Scientific Research) managed by other researchers, and is partially supported by the Takagi Fund for Citizen Science. JM contributed articles related with this topic to a popular science magazine and a news magazine and received honoraria for them. MO and YT declare no competing interest related with this work.

Ethics Approval

To write this commentary we used only figures and tables in published (and then retracted) papers and documents obtained through Freedom Of Information requests, and no ethics approval is necessary. Although the retracted papers use data unconsented for research, the Committee on Publication Ethics (COPE) states that it is possible to publish letters on such retracted papers in the context of journalology.

Acknowledgements

We thank Ms. Akemi Shima for providing us with the public documents obtained through her Freedom Of Information requests. We are also grateful to Mr. Taro Tanaka (the KAGAKU Editorial Office, Iwanami Shoten, Publishers) for opportunities to discuss this work.