We study invariant gauge fields over the 4-dimensional non-reductive pseudo-Riemannian homogeneous spaces G/K recently classified by Fels & Renner (2006). Given H compact semi-simple, classification results are obtained for principal H-bundles over G/K admitting: (1) a G-action (by bundle automorphisms) projecting to left multiplication on the base, and (2) at least one G-invariant connection. There are two cases which admit nontrivial examples of such bundles and all G-invariant connections on these bundles are Yang-Mills. The validity of the principle of symmetric criticality (PSC) is investigated in the context of the bundle of connections and is shown to fail for all but one of the Fels-Renner cases. This failure arises from degeneracy of the scalar product on pseudo-tensorial forms restricted to the space of symmetric variations of an invariant connection. In the exceptional case where PSC is valid, there is a unique G-invariant connection which is moreover universal, i.e. it is the solution of the Euler-Lagrange equations associated to any G-invariant Lagrangian on the bundle of connections. This solution is a canonical connection associated with a weaker notion of reductivity which we introduce.