In physics we attempt to infer the rules governing a system given only the results of imprecise measurements. This is an ill-posed problem because certain features of the system's state cannot be resolved by the measurements. However, by ignoring the irrelevant features, an effective theory can be made for the remaining observable relevant features. We explain how these relevant and irrelevant degrees of freedom can be concretely characterised using quantum distinguishability metrics, thus solving the ill-posed inference problem. This framework then allows us to provide an information-theoretic formulation of the renormalisation group, applicable to both statistical physics and quantum field theory. Using this formulation we show that, given a natural model for an experimentalist's spatial and field-strength measurement uncertainties, the n-point correlation functions of bounded momenta emerge as relevant observables. Our methods also provide a way to extend renormalisation techniques to effective models which are not based on the usual quantum field formalism. In particular, we can explain in elementary terms, using the example of a simple classical system, some of the problems occurring in quantum field theory and their solution.