The fractional Poisson process (FPP) is a counting process with independent and identically distributed inter-event times following the Mittag-Leffler distribution. This process is very useful in several fields of applied and theoretical physics including models for anomalous diffusion. Contrary to the well-known Poisson process, the fractional Poisson process does not have stationary and independent increments. It is not a L\'evy process and it is not a Markov process. In this letter, we present formulae for its finite-dimensional distribution functions, fully characterizing the process. These exact analytical results are compared to Monte Carlo simulations.