Terms of reference

When I was asked to chair the Investigation Commission appointed by Rikshospitalet- Radiumhospitalet MC and the University of Oslo (the appointing institutions) in January 2006, it probably mirrored the desperation of these institutions to identify anyone from the outside who could give legitimacy to the claims that everything should be done in order to clarify the circumstances surrounding the article by Jon Sudbø published in The Lancet during the fall of 2005 (1). However, my experience of investigating scientific misconduct were limited although not non-existent, but the omnipotence characterizing most physicians made it possible for me to accept the role as chairman and also the terms of reference formulated jointly by the appointing institutions.

The appointing institutions had chosen, or had been forced to choose (?), an uncommon pathway when an institution faces a case of suspected scientific misconduct. Instead of taking the normal route of hiding and/or downplaying the problem, the appointing institutions made an early commitment that all findings should be made public and the investigation would be carried out by individuals without any ties to any of the appointing institutions. As the chairman to be, I had accepted the terms of reference before the rest of the members of the Commission had been identified, which in hindsight was a mistake, especially as some terms were vague and the Commission was given less than three months to file a report. Eventually the composition of the Commission was finalized with the following members:

• Professor Anders Ekbom (Chair) Karolinska Institute Stockholm

• Special Advisor Gro Helgesen, the Research Council of Norway

• Researcher Tor Lunde, the Faculty of law, Bergen

• Professor Aage Tverdal, The Norwegian Institute of Public Health

• Professor Stein Emil Vollset, Bergen, and

• Research fellow Sigmund Simonsen (Secretary), Master of Law, Trondheim.

In addition, the National Cancer Institute (NCI), USA, was offered a seat on the Commission but did not accept the invitation.

The chairman eventually realized that he was to be congratulated on the choice of the members of the commission, who turned out to be an extremely hardworking group and dedicated to the task. In all the Commission met 13 times, had 11 telephone meetings, and in addition some members made site visits on different occasions. Another interesting note is that no leakage to the media or any other party occurred during its work. The experience and knowledge of legal procedures of two of the members of the Commission was an additional essential part, although the Commission was not a «public investigation commission». Their input made it possible to define the rules for our work which to a large degree adhered to the rules laid down for a public investigation commission. In addition, the Commission placed a great emphasis on independence in its contact with the appointing institutions, which was possible as no restrictions were placed on the use of resources. For instance all meetings were held outside the appointing institutions and the secretariat was placed in Trondheim.

The Commission’s primary task was to clarify the facts with the aim of discovering whether and to what extent breaches of standards for scientific research and other blameworthy acts had occurred. One of the first tasks for the Commission was therefore to establish the degree of proof that should be required as a basis for criticism. Taking into account the serious legal consequences and sanctions that might be triggered by such an act for an individual, the Commission decided that the degree of proof to be applied in order for it to rely on a particular fact as proven should be proof by a so-called qualified preponderance of probability. In the interaction within the Commission between those with a medical research and those with a legal background both parties eventually managed to conceptualize what the other party meant, although a p-value would had been helpful. In other words the Commission applied a very high threshold for criticism of persons. Moreover, the individuals who were investigated were notified of this and also informed that they might be subjected to criticism. The individuals who were subjected to criticism were allowed to read memos, documents, and finally a draft of the report and were also given the opportunity to respond and make contributions.

In relation to institutions, the Commission had a somewhat different approach. Two institutions were notified that they might be subjected to criticism and were given the opportunity to read the criticism, but not the full draft report, and had therefore only a limited possibility to contribute. Finally, the Commission chose not to notify the appointing institutions in order to uphold its independence and to prevent the risk of any unfortunate influence from these institutions. Moreover, the opinion of the Commission was that institutions, to a completely different degree than individuals, must be prepared to put up with public criticism.

The role of the co-authors

As mentioned previously the terms of reference formulated appointing institutions were vague and broadly stated, which meant that there were no restrictions imposed on the Commission, but also that the Commission had to prioritize. It became obvious early on that the entire scientific activity and production of Jon Sudbø had to be investigated, in all 38 publications according to the PubMed database. This meant that all co-authors who had contributed, 60 individuals, were approached. All were treated equally and were notified in writing that they were subjected to investigation and formally notified that this could result in criticism. They were also asked to make a written statement about their involvement of the research they had conducted with Jon Sudbø. In addition certain individuals who were named in Acknowledgements were approached in a similar manner as the co-authors. All co-authors and those other individuals approached by the Commission responded, and in quite a few instances there was a followup correspondence or a face to face interview.

One of the main interests of the media coverage of the Commission’s work was on the role of the co-authors. It therefore became clear that that the Commission had to choose a strategy to adhere to the so called Vancouver rules for authors.

The Vancouver Rules set forth three key conditions for authorship:

• 1. Substantial contributions to conception and design, OR acquisition of data, OR analysis and interpretation of data,

• 2. drafting the article, OR revising it critically for important intellectual content, and

• 3. final approval of the version to be published.

All three criteria must be met.

However, early on it became obvious that some individuals had not adhered to the Vancouver rules. For instance some co-authors were not aware of their status as a co-author, and in some instances they had even objected to being included both before and after publication. On the other side of the spectrum, there were some co-authors who undoubtedly fulfilled the criteria, but the majority of co-authors were in a «grey zone», where further investigation was needed in order to establish to what extent the criteria were fulfilled. Keeping in mind, as mentioned above, the potential serious consequences for an individual, who would be criticized in a final report from the Commission for not fulfilling the Vancouver rules, and the stringent standard which had to be applied for such criticism for 60 different coauthors, the Commission chose not to name any specific individual. This decision was made, to some extent, based on the impression that there are different perceptions of the authority as well as knowledge of the Vancouver rules within the medical research community in Norway. This is not a unique phenomenon from an international perspective, as similar problems have been documented in other countries.

Although the management of the appointing institutions could demonstrate a clear attitude with regards to the Vancouver rules, manifested by internal work instructions and other measures, the Commission was left with the impression that these measures had not been followed up well enough. The Commission therefore in the end chose to raise criticism against the appointing institutions for failure to create guidelines and follow-up systems with regard to authorship.

It also became obvious that at least for some articles a more active involvement of the co-authors in the handling of the manuscripts would have led to an earlier discovery of the use of fraudulent data. The Commission therefore believes that in order to contribute to a better compliance with prevailing rules, medical journals should introduce and practice a system in which all co-authors are made part of the communication with the journals. This includes a confirmation message to all co-authors that the paper has been submitted and also copies of review statements. In this way the individual co-author’s awareness of his/her responsibility would be strengthened and avoid the possibility that researchers are listed as co-authors without any knowledge of this fact.

The scientific production of Jon Sudbø

There was a need to evaluate the whole scientific production of Jon Sudbø. This evaluation was greatly helped by the Norwegian Cancer Registry, which ran a parallel investigation. Early on in this process it became obvious both for the Commission and the Norwegian Cancer Registry that the data which was the core in most of the articles published by Jon Sudbø did not add up. However, to establish beyond any doubt that the data were fraudulent was deemed by the Commission to be impossible within the time frame which was originally in the terms of reference. In order to keep to the time schedule we therefore initially tried another strategy.

The core of Jon Sudbø’s subsequent scientific production appeared already in three papers in his PhD – thesis. In these studies 150 individuals with premalignant changes in the oral cavity had been identified through different centers in Norway and then followed up through linkage with the Norwegian Cancer Registry for a subsequent cancer occurrence. In two separate papers, which both were included in the thesis work, the histopathological evaluation process of those premalignant changes was of particular interest. In the article published in The New England Journal of Medicine in 2001 (2) it was stated: «All histological sections were reviewed by four separate pathologists working at three different institutions (Department of Pathology, Haukeland Hospital, University of Bergen; The Norwegian Radium Hospital; and Department of Oral Pathology, University of Oslo).» In the article published in Journal of Pathology 2001 (3) it was stated: «All histological sections were subsequently reevaluated by four pathologists according to the guidelines of the World Health Organization. Consensus on the classification of dysplasia was reached in the case of 196 of the 242 patents (81 %).» No information was given about the identity of those four pathologists and there were seven potential candidates among the authors or those listed in the acknowledgements.

After interviewing all seven potential candidates it was clear that none of them had seen all specimens and subsequently all of them believed that they did not belong to the group of four pathologists. The members of the Commission then confronted Jon Sudbø and his lawyer during a face to face interview with these facts thinking that we had a good case to prove that these articles were a result of scientific misconduct. Their response was to point out (and rightly so) that we had misinterpreted the texts. An alternative interpretation was that four pathologists were involved and that nowhere in the text was it stated that any of four pathologists had had access to all specimens. Although the chairman of the Commission persisted in his opinion that the two texts were misleading, the Commission decided that this was not good enough in order to make the case that any of these articles could be classified as fraudulent.

It was then obvious that the Commission could not file a report at the end of March, 2006 and we therefore approached the appointing institutions with two alternatives. They could find themselves a new Commission or extend the time limit. If the institutions decided to choose the latter alternative, the Commission would not be able to provide a final date. We were not particularly surprised when the appointing institutions decided to keep the Commission intact, but then we faced the dilemma how to chart our subsequent work. The Commission decided that there was no more room for shortcuts and in essence went back to all original files from the different institutions, in order to explore to what extent the different studies could be replicated. This was done in close collaboration with the Norwegian Cancer Registry, which provided invaluable help, but we also received good support from the Rikshospitalet- Radiumhospitalet MC and the University of Bergen.

After a time-consuming process the Commission became convinced that the original patient material published in The New England Journal of Medicine in 2001 (2) contained data which did not correspond to the data which we and the Norwegian Cancer Registry were able to retrieve. The Commission was of the opinion that «these errors and defects which were exposed was to numerous, too great, and to obvious to be attributed to random errors, incompetence or the like; and that the raw data therefore appear to have been fabricated, manipulated and adapted to the desired finding». The Commission therefore recommended retraction of the majority of Jon Sudbø’s scientific production as it suffered from errors and flaws caused by scientific dishonesty.

Conclusions and future perspectives

What did I learn? There were at least three things I realized after my involvement with this case and taking into account other instances of scientific misconduct:

• 1. It will happen again!

• 2. It will happen again!

• 3. It will happen again!

In other words the scientific community has to be prepared to deal with situations when there are suspicions of scientific misconduct. Ideally one should have an organization in place, but if this is not the case at least use the experiences from this commission and similar ones. Other investigators should not be forced to re-invent the wheel! My take-home message to them can be summarized in four sentences:

• 1. You need lawyers, or access to good legal advice.

• 2. It is a costly and time consuming process.

• 3. You have to be an outsider.

• 4. Investigating is different from research.

Finally, a personal note. Doping among athletes has been used as a metaphor for fraudulent research, but in my opinion that is a dangerous and misleading comparison, especially if it is used in order to give legitimacy for the introduction of more rules and regulations as preventive measures. What one should not forget is the two entirely different goals characterizing these two activities. Among athletes the underlying aim is at a given moment to produce results which will surpass those of the opponent(s). In order to make that happen, the athletes or their coaches will use their creativity, and sometimes they will stray from that which is allowed. The rules and regulations are there to provide fair competition. In research, on the other hand, the underlying aim is to increase knowledge, sometimes in a competitive way. But creativity is unfortunately a scarce commodity and anything which dampens that creativity will potentially become an obstacle in the research process. Therefore the research community does not need more rules and/or regulations. We must learn to adhere to those which have emerged over time such as the Helsinki declaration and the Vancouver rules.