This paper presents the design of a neuro-fuzzy visual servoing controller for robot guiding fabrics with curved edges towards sewing. Fabrics comprising real cloths consist of curved edges of arbitrary curvatures that can not be standardized. To overcome this difficulty, the idea is to train the robot sewing system and to apply this methodology in a real-time operation environment. The proposed approach for robot sewing is based on visual servoing and a learning technique that combines neural networks and fuzzy logic. A novel genetic-oriented clustering method is used to construct the initial FIS models and then, adaptive neuro-fuzzy inference systems allow tuning them so that it is possible to obtain better estimates. Extensive experiments were carried out in order to build data sets using fabrics with curved edges of various curvatures. The proposed model is validated using fabrics that had not been included in the training process and the results demonstrate that the proposed approach is efficient and effective for robot guiding fabrics with arbitrary curved edges towards sewing.