Gene and protein interaction networks have evolved to precisely specify cell fates and functions. Here, we analyse whether the architecture of these networks affects evolvability. We find evidence to suggest that in yeast these networks are mainly acyclic, and that evolutionary changes in these parts do not affect their global dynamic properties. In contrast, feedback loops strongly influence dynamic behaviour and are often evolutionarily conserved. Feedback loops are often found to reside in a clustered manner by means of coupling and nesting with each other in the molecular interaction network of yeast. In these clusters some feedback mechanisms are biologically vital for the operation of the module and some provide auxiliary functional assistance. We find that the biologically vital feedback mechanisms are highly conserved in both transcription regulation and protein interaction network of yeast. In particular, long feedback loops and oscillating modules in protein interaction networks are found to be biologically vital and hence highly conserved. These data suggest that biochemical networks evolve differentially depending on their structure with acyclic parts being permissive to evolution while cyclic parts tend to be conserved.