We report experimental and theoretical investigations of the Hall effect in YbRh2Si2 and its reference compounds LuRh2Si2 and YbIr2Si2. Based on band-structure calculations we identify two bands dominating the Hall coefficient in all these compounds. For the case of LuRh2Si2 - the non-magnetic reference compound of YbRh2Si2 - the temperature dependence of the Hall coefficient is described quantitatively to arise from two hole-like bands. For YbIr2Si2 and YbRh2Si2, renormalized band calculations yield two bands of opposite character. In YbRh2Si2 these two bands almost compensate each other. We present strong indications that the sample dependences of the low-temperature Hall coefficient observed for YbRh2Si2 arise from slight variations of the relative scattering rates of the two bands. Minute changes of the composition appear to be the origin.