A real-time visual processing theory is used to provide a new approach to the analysis of surface perception, notably shape-from-shading. The theory elsewhere has been used to explain data about boundary detection and completion, textural segmentation, depth perception, color and brightness perception, and striate-prestriate cortical interactions. Neural network interactions within a multiple scale boundary contour (BC) system and feature contour (FC) system are used to explain these phenomena. Each spatial scale of the BC system contains a hierarchy of orientationally tuned interactions, which can be divided into two successive subsystems called the OC filter and the CC loop. The OC filter contains two successive stages of oriented receptive fields which are sensitive to different properties of image contrasts. The OC filter generates inputs to the CC loop, which contains successive stages of spatially short-range competitive interactions and spatially long-range cooperative interactions. Feedback between the competitive and cooperative stages synthesizes a coherent, multiple scale structural representation of a smoothly shaded image, called a boundary web. Such a boundary web regulates multiple-scale filling-in reactions within the FC system which generate a percept of form-and-color-in-depth. Computer simulations establish key properties of a boundary web representation: nesting of boundary web reactions across spatial scales, coherent completion and regularization of boundary webs across incomplete image data, and relative insensitivity of boundary webs to illumination level and highlights. The theory clarifies data about interactions between brightness and depth percepts, transparency, influences of highlights on perceived surface glossiness, and shape-from-texture gradients. The total network suggests a new approach to the design of computer vision systems, and promises to provide a universal set of rules for 3D perceptual grouping of scenic edges, textures, and smoothly shaded regions.