Design, analysis and statistical power of the Farm-Scale Evaluations of genetically modified herbicide-tolerant crops

1. The effects on British farmland wildlife of the management of four genetically modified herbicide-tolerant crops are currently being studied in a 5-year trial termed the Farm-Scale Evaluations (FSE), the first 4 years of which are completed. The FSE is controversial and extensive. There has been intense scrutiny of the experimental design and proposed analysis, and of the estimated statistical power to detect effects of a given magnitude, should any exist. 2. For each crop, the FSE is a form of on-farm trial with a single composite null hypothesis and a simple randomized block experimental design. This has statistical implications for the imposition of treatments by growers and the need for proper randomization. The choice of a half-field experimental unit was based on field availability, the focus on herbicide management, the need to reduce variability and efficiency gains in sampling effort. Farms and fields were selected to represent the range of variability of geography and intensiveness across Britain for each crop. 3. Results of a power analysis suggested that the planned replication of the FSE of about 60 fields per crop over 3 years would be sufficient to provide useful information, from which valid statistical inferences could be drawn. The achieved replication for spring crops in the FSE exceeded, by more than threefold, that in any of 82 comparable terrestrial manipulative ecological experiments undertaken previously. 4. Here, we exemplify a range of analyses including covariates, interactions between various factors including years and treatments, diagnostic procedures to aid selection of the most efficient statistical model, the estimation of power from coefficients of variation, a novel and apparently robust test statistic and the calculation of overall variance from within- and between-unit variability. Preliminary results indicated that a simple log-normal model appeared adequate for most analyses. 5. Synthesis and applications. Statistical challenges created by the scope of the FSE were resolved from a sound knowledge of good experimental design. There is an urgent need for further statistical studies to develop experimental designs or modelling approaches that allow similar studies of genetically modified (GM) crops, at reduced cost. However, this power analysis has shown that this cannot be achieved at the expense of adequate replication, essential for all risk assessment studies.