The present work is the second article in a couple of intertwined papers. They form complementary items on the same subject. They both address the problem of joint power allocation and time slot scheduling in a wireless communication system with time varying traffic. The system is handled by a single base station transmitting over time varying channels. The operating time horizon is divided into time slots, and a fixed amount of power is available at each time slot. The mobile users share each time slot and the power available at this time slot. Since many wireless network applications have stringent delay requirements, designing high-performance resource allocation algorithms to achieve minimum possible delay is of great importance, and this is the main objective of the work presented in this paper. The delay performance of a resource allocation algorithm can be characterized by the average delay experienced by the data transmitted in the network. We propose a heavy traffic analysis for the physical system on hand, i.e., appropriate re-scaling that leads to a diffusion approximation of the system in the sense of weak convergence. The approximate diffusion is constrained or bounded in the K-dimensional positive orthant. We establish the convergence result of the heavy traffic analysis, and then a closed form solution to the resource optimization problem is provided. Here the solution relies on the ergodicity of the approximate diffusion.