Following basic ideas of information physics, probability theory features as the inner symmetries of physical laws. Consequently, we conjecture that many fundamental physical facts are already hidden in the unique logical structure of probability theory and need not be postulated. A link with statistical thermodynamics emerges via the exponential (MaxEnt) mapping between probability and entropy, whose scaling symmetry also makes a natural bridge to fractal physics and projective geometries. To facilitate links with many other symmetries and physical areas, the exponential mapping between Lie groups and Lie algebras is suggested as a generalization of the MaxEnt relationship. We point out that the natural space of probability theory is an intrinsically 6-dimensional manifold with two fundamental governing equations imposed, which gives a novel straightforward rationale for the emergence of the 4+6=10-dimensional hyperspace, particularly important in modern particle physics.