We report on the results of systematic infrared 2.5-5 micron spectroscopy of 45 nearby ultraluminous infrared galaxies (ULIRGs) at z < 0.3 using IRC onboard the AKARI satellite. This paper investigates whether the luminosities of these ULIRGs are dominated by starburst activity, or optically elusive buried AGNs are energetically important. Our criteria include the strengths of the 3.3 micron polycyclic aromatic hydrocarbon (PAH) emission features and the optical depths of absorption features at 3.1 micron due to ice-covered dust grains and at 3.4 micron from bare carbonaceous dust grains. Because of the AKARI IRC's spectroscopic capability in the full 2.5-5 micron wavelength range, unaffected by Earth's atmosphere, we can apply this energy diagnostic method to ULIRGs at z > 0.15. We estimate the intrinsic luminosities of extended (several kpc), modestly obscured (Av < 15 mag) starburst activity based on the 3.3 micron PAH emission luminosities measured in AKARI IRC slitless spectra, and confirm that such starbursts are energetically unimportant in nearby ULIRGs. In roughly half of the observed ULIRGs classified optically as non-Seyferts, we find signatures of luminous energy sources that produce no PAH emission and/or are more centrally concentrated than the surrounding dust. We interpret these energy sources as buried AGNs. The fraction of ULIRGs with detectable buried AGN signatures increases with increasing infrared luminosity. Our overall results support the scenario that luminous buried AGNs are important in many ULIRGs at z < 0.3 classified optically as non-Seyferts, and that the optical undetectability of such buried AGNs occurs merely because of a large amount of nuclear dust, which can make the sightline of even the lowest dust column density opaque to the ionizing radiation of the AGNs.