We address the question of how the electrocaloric effect in epitaxial thin films of the prototypical ferroelectric BaTiO$_3$ is affected by the clamping to the substrate and by substrate-induced misfit strain. We use molecular dynamics simulations and a first-principles-based effective Hamiltonian to calculate the adiabatic temperature change $\Delta T$ under different epitaxial constraints. Our results demonstrate that, consistent with phenomenological theory, clamping by the substrate reduces the maximum $\Delta T$ compared to bulk BaTiO$_3$. On the other hand, compressive misfit-strain leads to a strong increase of $\Delta T$ and shifts the maximum of the electrocaloric effect to higher temperatures. A rather small compressive strain of $-0.75$% is sufficient to obtain a $\Delta T$ that is larger than the corresponding bulk value.