The covariant phase space of a Lagrangian field theory is the solution space of the associated Euler-Lagrange equations. It is, in principle, a nice environment for covariant quantization of a Lagrangian field theory. Indeed, it is manifestly covariant and possesses a canonical (functional) "presymplectic structure" w (as first noticed by Zuckerman in 1986) whose degeneracy (functional) distribution is naturally interpreted as the Lie algebra of gauge transformations. We propose a fully rigorous approach to the covariant phase space in the framework of secondary calculus. In particular we describe the degeneracy distribution of w. As a byproduct we rederive the existence of a Lie bracket among gauge invariant functions on the covariant phase space.