Implementing the hierarchical structures of non-noble-metal-based electrocatalysts and modulating their composition can help accelerate surface reactions and fulfill the promise of renewable energy devices via water splitting. Herein, molybdenum-based compounds are constructed on activated nickel foam (act-NF) by a one-step hydrothermal growth. The product generated on the act-NF is [email protected], with a novel 3D hierarchical heterostructure, wherein the one-dimensional CoMoO4 nanorods are hierarchically integrated with the two-dimensional NixCo1-xMoO4 nanosheets ([email protected]/act-NF). The formation of [email protected] attributes to the release and diffusion of Ni2+ from act-NF. Heterogeneous [email protected] has compositional differences, and synergistic interaction between cobalt and nickel results in the modulated electronic states. Meanwhile, the hierarchical structure facilitates the exposure of active sites. Combining these two advantages, [email protected]/act-NF presents a low η10 value of 61 and 180 mV in 1.0 M KOH for the HER and OER, respectively, and it shows a low cell voltage of 1.46 V for overall water splitting with robust stability. DFT calculations reveal that Ni doping leads to the charge depletion of Co, which further optimizes the d-band center of metal sites and tunes the adsorption of adsorbates to facilitate the water splitting reaction. Thus, a promising strategy of incorporating the nanostructure design with compositional modulation is presented to develop functional materials for energy conversion.