ISSN: 1743-1026 (Print); 1743-1034 (Online); Volume 28, No. 3 June 2017
Robin Blake, CSI Senior Consultant and Editorial Board Member, is the lead author of an editorial published in the June 2017 issue of Outlooks on Pest Management – an international journal covering the management of weeds, pests and diseases through chemistry, biology and biotechnology.
The title of the editorial is, Scientific Rigour – Are Any Studies Truly Independent?.
EDITORIAL: Glyphosate has long been regarded as one of the most benign herbicides and has been used successfully for over 40 years. However, its controversial reclassification by the International Agency for Research on Cancer (IARC), part of the World Health Organization (WHO), as “probably carcinogenic to humans” forced a delay to the European relicensing decision and has put scientific conduct firmly in the spotlight. Whilst the European Commission granted an 18-month extension in July 2016 to the glyphosate approval, a proposal for full license renewal met opposition from member states with the Commission accusing its members of failing to take an open stance on controversial issues and instead choosing to hide behind Brussels. The European Food Safety Authority (EFSA) and the European Chemical Agency (ECHA) have since published reports concluding that glyphosate is unlikely to pose a cancer risk to humans, and a final decision from the European Commission is expected later this year.
The glyphosate debate represents more than just scientific interpretation and has become a political battleground for trust, independence and transparency issues within EU institutions. On the one hand is the pesticide industry, represented in this case by the Glyphosate Task Force, which has submitted hundreds of studies to demonstrate the safety of glyphosate to human health and the environment, and farmers who are already concerned about the lack of options to control weeds; on the other hand are non-governmental organisations (NGOs) such as Friends of the Earth and Pesticide Action Network who receive funding from the organic food industry and tend to lobby against anything pesticide-related. Politicians and the wider public are caught somewhere in the middle. Whilst some commentators, notably anti-pesticide groups, believe the industry is choosing to report only positive data on glyphosate and other pesticides, the wider problem is that the public has no way of putting what they hearinto context. They tend to read newspapers and social media rather than scientific journals, and thus get a distorted view on reality from a media industry that sells through scaremongering and focusing on negative stories whilst downplaying or even omitting the full picture.
A sizeable portion of the nearly $300 million necessary to bring a new crop protection product to market is spent on highly regulated studies conducted under strict standards of performance and documentation to ensure the product does not present any unacceptable risk to human health or the environment. These studies are often conducted under Good Laboratory Practice (GLP) which gives assurances that study data submitted to government assessors are accurate, valid and of sound integrity. GLP is focused on ‘traceability’ i.e. the ability of a study to be repeated many years after it was first conducted using the same parameters, materials and methods that were used in the original study, to create a similar result. Reports of data generated under GLP are generally kept confidential and the data protected from third party commercial use for limited time periods under data protection laws. Maintaining confidentiality in a regulated environment is intended to produce sound and repeatable results; however, this secrecy does little to help convince the anti-pesticide campaigners, politicians or wider population that there is nothing to worry about regarding study interpretation.
The European Free Alliance Group within the European Parliament has called on Commission President Juncker to revise the pesticides legislation to ensure that the scientific evaluation of pesticides for EU regulatory approval is based only on studies that are independent, replicable, peer-reviewed and published, for example in academic journals. Whilst these study attributes are considered to represent perfect science, how difficult are they to achieve and do studies like this represent unbiased findings?
To define an independent scientific study is quite difficult – after all, the money to fund a study must come from somewhere. The best definition is a study that is financially independent and without affiliation to an academic, (non-) government, or industrial body. Whilst this may be an option for a few researchers the majority will require financial help for purchasing equipment, use of facilities, and their time. If it is assumed that most studies are funded by someone else, this can lead to funding or sponsorship bias, i.e. the tendency of a scientific study to support the interests of its financial sponsor. In addition, the methods or test parameters used in studies that are not financially supported are more likely to be less rigorous and repeatable. Whilst industry studies are often criticised for their prejudice towards the company funding the research to ensure that their products are portrayed in a positive manner, funding bias also exists elsewhere, and academic studies receiving funding from e.g. a research council, government department or an NGO, may be designed in such a way to portray the subject in a positive or negative manner depending on the objective. Therefore, the assertion that non-industry funded studies are more likely to be independent is questionable. The importance of author affiliation is highlighted by the IARC itself. Christopher Portier was one of the co-authors on the glyphosate monograph even though he had a role working for an anti-pesticide NGO. Whether Portier’s involvement in the glyphosate review made a difference to the outcome will probably never be known; however, there is a clear conflict of interest here that should have been made public from the start. Recent allegations of Monsanto ghost-writing research and then later attributing it to academics do not help convince the public that science is trustworthy and credible, and whilst Monsanto have strongly denied these allegations, it is clearly imperative that scientists on both sides declare professional conflicts of interest that may be seen to confer a bias on the work that they conduct to maintain credibility.
Within academia, the ability to replicate or reproduce research is extremely important – to allow the scientific community to accept a finding, other investigators must be able to duplicate the original investigator’s findings. Setting appropriate hypotheses, recording key parameters including materials and methods, and undertaking appropriate statistical analysis help achieve quality and credible research. However, many studies are the results of poorly conducted or poorly analysed experiments which makes reproduction impossible – in 2012 the Biotech firm, Amgen, found that they could only reproduce 6 of 53 landmark academic studies on cancer research (Begley & Ellis, 2012). The importance of reproducibility was underlined by the infamous fraud involving the now discredited Dr Andrew Wakefield and team on a supposed link between autism and the MMR (Measles-Mumps-Rubella) vaccine which was published in the highly respected journal The Lancet (Wakefield et al., 1998). Despite the small sample size (n=12), uncontrolled design, and speculative nature of the conclusions, the paper received substantial media coverage. Concerns were raised when other researchers were unable to reproduce their results, and whilst the paper was eventually retracted it took more than ten years to do so. The public had good reason to believe that there was a credible link because the media distorted the scientific evidence – in 2002, less than a third of UK broadsheet newspapers reporting on the subject mentioned that the MMR vaccine was safe, despite the many hundreds of studies available showing just that. Nearly twenty years later, untold suffering to human health continues with a global drop in MMR vaccination rates because of parental concerns about possible autism links despite none existing. Unlike industry where regulation tends to ensure authenticity of results, academic scientists rely on peer review prior to publication to assess the quality and integrity of the science; however, the Wakefield example demonstrates that this is a far from perfect solution.
The goal of peer review is to assess the quality of articles submitted for publication in a scholarly journal. Before an article is deemed suitable it is submitted to the journal editor who forwards it to experts in that field to review the paper for accuracy, quality, assess the method validity and claims made, and if appropriate suggest revisions or reject it. Whilst the peer review process should eliminate poorly conducted studies, it is not without its flaws. For example, when a prominent medical journal ran research past other experts in the field, it found that most of the reviewers failed to spot mistakes it had deliberately inserted into papers, even after being told they were being tested. From the author’s perspective, defining and subsequently testing robust hypotheses is critical to performing credible science, and yet failures to prove a hypothesis are rarely offered for publication, let alone accepted – negative results now account for only 14% of published papers, down from 30% in 1990 (Anon, 2013). Given that less than 10% of papers received by leading journals are accepted for publication, the most striking findings have the greatest chance of making it into print so it is perhaps not surprising that conclusions are sometimes exaggerated and results cherry-picked. Sensational or controversial papers often get more reads and thus citations and can boost the researcher’s profile and potential to secure future funding and tenure opportunities. This academic incentivization to produce supposed ‘quality’ papers in top journals ensures that inferior science will continue to overshadow credible science. If an author wishes to deceive then it can be difficult for journal editors and reviewers to detect as the Wakefield study shows; however, journals do have a key role to play in ensuring the research that they allow to be published is accurate and credible, regardless of who authored it, who funded it or what the title claims to suggest. Mistakes in peer-reviewed papers are easy to find yet difficult to fix with little formal guidance existing to address post-publication corrections and with editors often unable or reluctant to take fast and appropriate action (Allison et al. 2016).
Bernhard Url, the Executive Director of EFSA, recently commented that studies on the safety of pesticides such as glyphosate could enjoy higher levels of trust from the public if there were stronger guarantees that the science behind them is really independent. In an ideal world that would be preferable; however, as outlined above that is not attainable. Certainly, industry needs to continue to work hard to show the public the significant effort that goes into ensuring products are safe to use. However, the substantial cost burden of conducting regulatory safety testing should be borne by the company wishing to bring the substance to market. EU Research programmes including Horizon 2020 require scientists to collaborate with industry to receive funding, and, in an era when public funding for science has been dramatically cut, it is imperative that industry can continue to underwrite scientific research. In the US, the National Institute of Environmental Health Sciences has invested over $30 million on supposed ‘independent research’ to assess the safety of a single chemical, bisphenol A, following similar concerns of industry cover-ups, with the result that its investigations agreed with the original industry findings. Implementing a similar strategy to investigate each pesticide active ingredient is clearly not affordable. If European institutions such as the EFSA and the ECHA are to be impartial and objective then they should examine themselves for bias against industry to ensure they are not further confusing the public and inhibiting safe and efficient decisions about chemical regulation. Finally, the role of the media should be examined to ensure good science is reported over sensationalism.
Allison, D.B., Brown, A.W., George, B.J. & Kaiser, K.A. (2016). Reproducibility: A tragedy of errors. Nature 530, 27–29.
Anon (2013). Problems with scientific research. How science goes wrong. The Economist. Published October 21st 2013.
Begley, G.C. & Ellis, L.M. (2012). Drug development: Raise standards for preclinical cancer research. Nature 483, 531–533.
Wakefield, A.J., Murch, S.H., Anthony, A., Linnell, J., Casson, D.M., Malik, M., Berelowitz, M., Dhillon, A.P., Thomson,
M.A., Harvey, P., Valentine, A., Davies, S.E. & Walker-Smith, J.A. (1998). RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet 351 (9103), 637–641.