Rocky Mountain Research Station Publications
RMRS Online Publication - Journal
Articles, External Publications, and Special Reports
Effects of sample survey design on the
accuracy of classification tree models in species distribution
models
Edwards, Thomas C., Jr.; Cutler, D. Richard; Zimmermann, Niklaus E.; Geiser, Linda; Moisen, Gretchen G. 2006. Effects of sample survey design on the accuracy of classification tree models in species distribution models. Ecological Modelling. 199: 132-141.
We evaluated the effects of probabilistic (hereafter DESIGN) and non-probabilistic (PURPOSIVE) sample surveys on resultant classification tree models for predicting the presence of four lichen species in the Pacific Northwest, USA. Models derived from both survey forms were assessed using an independent data set (EVALUATION). Measures of accuracy as gauged by resubstitution rates were similar for each lichen species irrespective of the underlying sample survey form. Cross-validation estimates of prediction accuracies were lower than resubstitution accuracies for all species and both design types, and in all cases were closer to the true prediction accuracies based on the EVALUATION data set. We argue that greater emphasis should be placed on calculating and reporting cross-validation accuracy rates rather than simple resubstitution accuracy rates. Evaluation of the DESIGN and PURPOSIVE tree models on the EVALUATION data set shows significantly lower prediction accuracy for the PURPOSIVE tree models relative to the DESIGN models, indicating that non-probabilistic sample surveys may generate models with limited predictive capability. These differences were consistent across all four lichen species, with 11 of the 12 possible species and sample survey type comparisons having significantly lower accuracy rates. Some differences in accuracy were as large as 50%. The classification tree structures also differed considerably both among and within the modelled species, depending on the sample survey form. Overlap in the predictor variables selected by the DESIGN and PURPOSIVE tree models ranged from only 20% to 38%, indicating the classification trees fit the two evaluated survey forms on different sets of predictor variables. The magnitude of these differences in predictor variables throws doubt on ecological interpretation derived from prediction models based on non-probabilistic sample surveys.
Keywords: model accuracy, sample survey, study design, classification trees, lichens, accuracy assessment, probability samples, non-probability samples
About PDFs: For best results, do not open the PDF in your Web browser. Right-click on the PDF link to download the PDF file directly to your computer. Click here for more PDF help.
Download Article
http://www.fs.fed.us/rm/pubs_other/rmrs_2006_edwards_t001.pdf
PDF File Size: 415 K
Title: RMRS Other
Publications: Effects of sample survey design on the accuracy
of classification tree models in species distribution models
Electronic Publish Date: August 30, 2007
Last Update: August
30, 2007