We describe a technique for fabricating micro- and nano-structures incorporating fluorescent defects in diamond with a positional accuracy in the hundreds of nanometers. Using confocal fluorescence microscopy and focused ion beam (FIB) etching we first locate a suitable defect with respect to registration marks on the diamond surface and then etch a structure using these coordinates. We demonstrate the technique here by etching an 8 micron diameter hemisphere positioned such that a single negatively charged nitrogen-vacancy defect lies at its origin. This type of structure increases the photon collection efficiency by removing refraction and aberration losses at the diamond-air interface. We make a direct comparison of the fluorescence photon count rate before and after fabrication and observe an 8-fold increase due to the presence of the hemisphere.