To elucidate the inhibitory effects of different herbicides on soil nitrification, eight widely used herbicides, i.e., acetochlor, atrazine, dicamba, isoproturon, paraquat, puma, tribenuron-methyl, and 2,4-dichlorophenoxyacetic acid butyl ester (2,4-Dbe), which represent different chemical taxonomy were selected. Our results indicated that herbicide 2,4-Dbe displayed the best inhibitory effect on nitrification, followed by puma and tribenuron-methyl, whereas the remaining five herbicides exhibited less effect when 10 mg of active ingredient (A.I.) of every herbicide per kg of soil was applied in vegetable-planting soil. The inhibition appeared when 5–100 mg of A.I. 2,4-Dbe was employed, which was enhanced with an increment in its dose in both vegetable-planting and fluvo-aquic soils. However, the inhibitory effect of 10 mg of A. I. 2,4-Dbe exhibited obvious differences in these two types of soils, where the duration of inhibition was shorter as it only continued about a week in fluvo-aquic and calcic cambisols soils with strong nitrification activity but poorer effect as compared to 10 mg of dicyandiamide (DCD). In contrast, the duration of inhibition exceeded 2 months in dryland red and shajiang black soils with a weak nitrification activity which was equivalent to DCD. In addition, comparing with five nitrification inhibitors, 10 mg of 2,4-Dbe had better inhibition than the substituted pyrimidine (AM) and sulfocarbamide (SU), but was equivalent to DCD, nitrapirin, and 3,4-dimethylpyrazole phosphate (DMPP) at their recommended application rates in dryland red soil. These obtained data clearly indicated that 2,4-Dbe could play a stronger role as a nitrification inhibitor in soils.