Affordable Access

The Problem with Assessing Statistical Methods

Authors
  • Arnold, Abigail
  • Loeppky, Jason
Type
Preprint
Publication Date
Oct 05, 2015
Submission Date
Oct 05, 2015
Identifiers
arXiv ID: 1510.01417
Source
arXiv
License
Yellow
External links

Abstract

In this paper, we investigate the problem of assessing statistical methods and effectively summarizing results from simulations. Specifically, we consider problems of the type where multiple methods are compared on a reasonably large test set of problems. These simulation studies are typically used to provide advice on an effective method for analyzing future untested problems. Most of these simulation studies never apply statistical methods to find which method(s) are expected to perform best. Instead, conclusions are based on a qualitative assessment of poorly chosen graphical and numerical summaries of the results. We illustrate that the Empirical Cumulative Distribution Function when used appropriately is an extremely effective tool for assessing what matters in large scale statistical simulations.

Report this publication

Statistics

Seen <100 times