This thesis describes an exploration in achieving sparse representations of object, with special focus on spectral data. Given a database of objects one would like to know the actual aspects of each class that distinguish it from any other class in the database. We explore the hypothesis that simple abstractions (descriptions) that humans normally make, especially based on the visual phenomenology or physics on the problem, can be helpful in extracting and formulating useful sparse representations of the observed objects. In this thesis we focus on the discovery of such underlying features, employing a number of recent methods from machine learning. Firstly we find that an approach to automatic feature discovery recently proposed in the literature (Non Negative Matrix Factorization) is not as it seems. We show the limitations of this approach and demonstrate a more efficient method on a synthetic problem. Secondly we explore a more empirical approach to extracting visually attractive features of spectra from which we formulate simple re-representation of spectral data and show that the identification and discovery of certain intuitive features at various scales can be sufficient to describe a spectrum profile. Finally we explore a more traditional and principled automatic method of analyzing a spectrum at different resolutions (Wavelets). We find that certain classes of spectra can easily be discriminated between by a simple approximation of the spectrum profile while in other cases only the finer profile details are important. Throughout this thesis we employ a measure called the separability index as our measure of how easy it is to discriminate objects in a database with the proposed representations.