It has been shown theoretically and observationally that the Green's function for acoustic and elastic waves can be retrieved by cross-correlating fluctuations recorded at two locations. We extend the concept of the extraction of the Green's function to a wide class of scalar linear systems. For systems that are not invariant under time reversal, the fluctuations must be excited by volume sources in order to satisfy the energy balance (equipartitioning) that is needed to extract the Green's function. The general theory for retrieving the Green's function is illustrated with examples that include the diffusion equation, Schrödinger's equation, a vibrating string, the acoustic wave equation, a vibrating beam, and the advection equation. Examples are also shown of situations where the Green's function cannot be extracted from ambient fluctuations. The general theory opens up new applications for the extraction of the Green's function from field correlations that include flow in porous media, quantum mechanics, and the extraction of the response of mechanical structures such as bridges.