Sexual violence (SV) has an important impact on mental health. Childhood sexual abuse is linked to internalising disorders in later life. In older adults, SV occurs more often than previously believed. Moreover, health care workers lack the skills to address SV in later life. Studies researching the mental health impact of lifetime SV, i.e., SV during childhood, adulthood, and old age, are lacking. Between July 2019 and March 2020, 513 older adults living in Belgium participated in structured face-to-face-interviews. Selection occurred via a cluster random probability sampling with a random walk finding approach. Depression, anxiety, and posttraumatic stress syndrome (PTSD) were measured using validated scales. Participants were asked about suicide attempts and self-harm during their lifetime and in the past 12 months. SV was measured using behaviourally specific questions based on a broad SV definition. We found rates for depression, anxiety, and PTSD of 27%, 26%, and 6% respectively, while 2% had attempted suicide, and 1% reported self-harm in the past 12 months. Over 44% experienced lifetime SV and 8% in the past 12 months. Lifetime SV was linked to depression (p = 0.001), anxiety (p = 0.001), and PTSD in participants with a chronic illness/disability (p = 0.002) or no/lower education (p < 0.001). We found no link between lifetime SV and suicide attempts or self-harm in the past 12 months. In conclusion, lifetime SV is linked to mental health problems in late life. Tailored mental health care for older SV victims is necessary. Therefore, capacity building of professionals and development of clinical guidelines and care procedures are important.