T.The oldest behavioral journal in the world, Ethology, announced on Jan. 4 that it will introduce a new experimental design and data reporting framework called STRANGE to remove bias in animal behavior and cognitive research. In the future, authors who send manuscripts to the journal will need to examine their test animals for possible biases – factors such as genetics, personality differences, or previous research experience – and discuss how these facets may influence the results of the study.
“Everyone knows that there are certain sampling biases that can affect the reproducibility and generalizability of research into animal behavior, but these are often not declared,” says Christian Rutz, behavioral ecologist at the University of St. Andrews in Scotland and a member of the editorial board of Ethology Who helped shape the STRANGE framework? “We thought it was time to bring all of this together and develop a framework that would help our community.”
About a decade ago, the field of human psychology struggled with the realization that most studies favored Western, educated, industrialized, rich, and democratic (WEIRD) populations. People who live in these societies make up around 80 percent of the research participants, but according to the EU only make up 12 percent of the world’s population American Psychological Association. The recognition of a tendency towards WEIRD subjects led to wider efforts to diversify study participants and to new discussions about how scientists report, reproduce, and generalize their results.
STRANGE, in combination with other existing frameworks, seeks to replicate this success in the areas of animal behavior and animal cognition by asking researchers to consider the sources and effects of bias in their studies.
Rutz and his St. Andrews colleague Michael Webster presented STRANGE in one for the first time nature June 2020 comment setting out several possible causes of bias that could affect animal behavior during experiments, including their social background (S); Captivability and Self-Choice (T); Rearing history (R); Acclimatization and habituation (A); natural changes in responsiveness (N); Genome (G); and experience (E).
For example, it is known that animals caught in the wild behave differently than individuals of the same species that are bred and raised in a laboratory. And traps used to catch animals in the field can affect a specimen by only including those individuals brave enough to approach. For animals raised in captivity, prior exposure to research tasks can alter the response to subsequent experiments.
It’s a really interesting framework to evaluate the suitability of the animals used.
– Nathalie Percie du Sert, National Center for the Replacement, Refinement and Reduction of Animals in Research
Submitted in future manuscripts at EthologyResearchers are asked to add a short section on their methodology or a supplementary table listing the qualitative and quantitative “STRANGE” of their study cohort, as well as a short section in the discussion that provides an appropriate context for their findings in the possible biases experimental design. The conclusions from a single study should be closely linked to the population of animals included in the research, as per STRANGE guidelines, and should not be extrapolated to other populations or taxa.
Animal behavior, like many other areas of science, suffers from a “reproducibility crisis” that makes it difficult to judge how reliable or universal the results are. In addition, animal studies sometimes generalize their results from just a few people. By including more details, STRANGE could make replicating experiments easier, says Webster. “Undeclared STRANGE effects can explain why some experiments repeat and others don’t.”
In addition to EthologyRutz said two other magazines – a niche animal behavior and a larger, interdisciplinary magazine – are currently changing their submission guidelines to include STRANGE, although both refused to be named prior to their official announcements.
STRANGE is not the first such framework that aims to increase transparency about how experiments are designed and how their results are shared in the literature.
Nathalie Percie du Sert, a researcher at the National Center for Animal Replacement, Refinement and Reduction in research dealing with experimental design and reporting, first recognized the need for new guidelines while doing a review of the ferret system she was using did her doctoral thesis. In analyzing the literature, she tried to apply the same quality metrics that were used to evaluate human clinical trials. For example, studies that were not randomized or blinded would normally have been excluded from their review. In the case of animal testing, however, I would have “had no studies to include in my systematic review” if I had followed the same rules. “It was so bad.”
Working with her colleagues, Percie du Sert helped develop ARRIVE, a set of reporting guidelines approved by more than 1,000 magazines and promoted by several funding agencies since it was launched in 2010. ARRIVE provides a checklist of 10 “essential” items researchers should include to ensure their studies are reported in sufficient detail, including information about the species, strain, substrain, sex, weight and age of each animal.
Strange, says Percie du Sert The scientist, is “fully compatible” with ARRIVE and goes one step further to solve more detailed problems that only arise in the area of animal behavior. “It’s a really interesting framework to evaluate the suitability of the animals being used, and STRANGE is not just about reporting, but also at the design stage to assess whether the animals are actually suitable for the purpose of your experiments. ”
Along with another set of guidelines called PREPARE that are used during experimental design, the three span the continuum of scientific research – from conceptualization to data reporting. “STRANGE can be integrated directly into PREPARE, PREPARE calls for ARRIVE,” says Rutz, adding that these frameworks are already changing the way scientists deal with bias in their work. Both PREPARE and ARRIVE have been endorsed by the Association for the Study of Animal Behavior and the Animal Behavior Society.
Benjamin Farrar, a PhD student in comparative psychology at Cambridge University, who posted a comment on STRANGE in Learning and behavior, says adding a wider framework risks turning the practice of biasing manuscript submission into a “box-checking exercise”. “It seems to have a lot of redundancy with what ARRIVE is trying to achieve, and ARRIVE seems to have a lot more extensive and thoughtful about it,” says Farrar. “STRANGE is a really positive step forward, but in its current form it doesn’t quite get the strong sampling solution it wants to be.”
Farrar points out changes made by WEIRD to how bias is assessed in human research. Not only is it a bias inherent in the subjects themselves, but also a bias among researchers – about the types of tests used to examine cognition or behavior – and within the institutions themselves. Human psychologists have begun to use more robust statistical tools how to use models that include unmeasured sources of variation in their data.
While a handful of articles that have voluntarily adopted STRANGE have already been published in magazines such as Current biology and Movement ecologyIt is too early to say how effective the framework will be in combating reproducibility in animal research. “I think there will be bias at some level no matter what we do,” says Webster The scientist. “The best we can do is learn about these prejudices. We thought it was high time to really highlight this problem, bring these different problems together and propose a solution. “