Focused ion beam (FIB) milling has been used to fabricate magnetic nanostructures (wires, squares, discs) from single magnetic layers (Co, permalloy) and spin-valve (permalloy/Cu/Co) multilayers (thicknesses 5-50 nm) prepared by ion beam sputtering deposition. Milled surfaces of metallic thin films typically exhibit residual roughness, which is also transferred onto the edges of the milled patterns. This can lead to domain wall pinning and influence the magnetization behaviour of the nanostructures. We have investigated the milling process and the influence of the FIB parameters (incidence angle, dwell time, overlap and ion beam current) on the roughness of the milled surface. It has been found that the main reasons for increased roughness are different sputter yields for various crystallographic orientations of the grains in polycrystalline magnetic thin films. We have found that the oblique ion beam angle, long dwell time and overlap < 1 are favourable parameters for suppression of this intrinsic roughness. Finally, we have shown how to determine the ion dose necessary to mill through the whole thin film up to the silicon substrate from scanning electron microscopy (SEM) images only.