Researchers often espouse “best practices” in their work, but in reality, optimal methodologies are very much project-specific and are also constantly being refined. As practicing scientists, in addition to pursuing original research, we are committed to advancing the methodologies used in our field.
Effective validation is critical for predictive models in ecology, but cross-validation can be undermined by issues of independence. Model overfit to data structure often lurks as an overlooked issue, especially in ecological data where variables often covary in space, time, or phylogeny. Blocking in model cross-validation, where validation data are selected systematically rather than randomly, offers more reliable estimates.
Roberts, D. R., V. Bahn, S. Ciuti, M. S. Boyce, J. Elith, G. Guillera-Arroita, S. Hauenstein, J. J. Lahoz-Monfort, B. Schröder, W. Thuiller, D. I. Warton, B. A. Wintle, F. Hartig, and C. F. Dormann. (2017) “Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure.” Ecography, 40(8): 913-929.
Environmental researchers often use downscaling approaches to generate climate data at biologically relevant scales from coarser resolution weather station network or climate model. Compared to a mesoscale network of observation stations in the Canadian Rocky Mountains, these data show elevational and seasonal bias. However, bias adjustment models can be developed even with a small number of observation stations, provided those stations cover a wide breadth of elevation.
Roberts, D. R., W. Wood, and S. Marshall. (2019) “Assessments of downscaled climate data with a high-resolution weather station network reveal consistent but predictable bias.” In review.