Abstract Sparse coding based classifier (SCC) proves to lead to the state-of-the-art result in pattern recognition. Compared with traditional generative models and discriminative models, it neither casts some assumption on the distribution of data, nor learns a hyperplane to separate samples. However, SCC is characteristic of slow prediction because an l0-norm minimization need to be solved to assign the label for each sample. In this paper, we propose a Superpixel-wise Structural Sparse Coding based Classifier (S3CC) for image segmentation. An unsupervised superpixel segmentation is first used to derive the initial labeled samples, and SCC is extended to the semi-supervised pattern where unlabeled samples are incrementally labeled and taken as the dictionary to improve the classification accuracy. Moreover, a neighborhood spatial constraint is cast on the prediction of pixel labels, to avoid the speckle-like mis-segmentation of images. Some experiments are taken on some artificial texture images, to investigate the segmentation result of our proposed S3CC. Some aspects including (1) Comparison of S3CC with SCC, (2) Comparisons of S3CC with and without spatial constraint, (3) Comparison of S3CC with semi-supervised S3CC, are tested, and the results prove the efficiency and superiority of S3CC to its counterparts.