The analysis of action potentials, or "spikes," is central to systems neuroscience research. Spikes are typically identified from raw waveforms manually for off-line analysis or automatically by human-configured algorithms for on-line applications. The variability of manual spike "sorting" is studied and its implications for neural prostheses discussed. Waveforms were recorded using a micro-electrode array and were used to construct a statistically similar synthetic dataset. Results showed wide variability in the number of neurons and spikes detected in real data. Additionally, average error rates of 23% false positive and 30% false negative were found for synthetic data.