Instability of a binary eutectic solidification front to morphological perturbations due to rejection of a ternary impurity leads to the formation of eutectic colonies. Whereas, the instability dynamics and the resultant mi- crostructural features are reasonably well understood for isotropic systems, several experimental observations point to the existence of colonies in systems with anisotropic interfaces. In this study, we extend the un- derstanding of eutectic colonies to anisotropic systems, where only certain orientations of the solid-liquid or solid-solid interfaces are thermodynamically stable. Through phase field simulations in 2D and 3D, we have systematically probed the colony formation dynamics and the resulting microstructures, as functions of the pulling velocity and the relative orientation of the equilibrium interfaces with that of the imposed tempera- ture gradient. We find that in 2D, stabler finger spacings are selected with an increase in the magnitude of anisotropy introduced, either in the solid-liquid or in the solid-solid interface. The fingers have a well-defined orientation for the case of anisotropy in the solid-liquid interface, with no fixed orientations for the lamellae constituting the colony. For the case where anisotropy exists in the solid-solid interface, the lamellae tend to orient themselves along the direction of the imposed temperature gradient, with tilted solid-liquid interfaces from the horizontal. The 3D simulations reveal existence of eutectic spirals which might become tilted under certain orientations of the equilibrium interfaces. Our simulations are able to explain several key features observed in our experimantal studies of solidification in Ni-Al-Zr alloy.