Due to the development of intelligent demand-side management with automatic control, distributed populations of large residential loads, such as air conditioners (ACs) and electrical water heaters (EWHs), have the opportunities to provide effective demand-side ancillary services for load serving entities (LSEs) to reduce the emissions and network operating costs. Most present approaches are restricted to 1) the scenarios involving with efficiently scheduling the large number of appliances in real time, 2) the issues about evaluating the contributions of individual residents towards participating demand response (DR) program, and fairly distributing the rewards, and 3) the concerns on performing cost-effective demand reduction request (DRR) for LSEs with minimal rewards costs while not affecting their living comfortableness. Therefore, this paper presents an optimal framework for residential load aggregators (RLAs) which helps solve the problems mentioned above. Under this framework, RLAs are able to realize the DRR for LSEs to generate optimal control strategies over residential appliances quickly and efficiently. To residents, the framework is designed with probabilistic model of comfortableness, which minimizes the impact of DR program to their daily life. To LSEs, the framework helps minimize the total reward costs of performing DRRs. Moreover, the framework fairly and strategically distributes the financial rewards to residents, which may stimulate the potential capability of loads optimized and controlled by RLAs in demand side management. The proposed framework has been validated on several numerical case studies.