Human milk oligosaccharides (HMOs) have been shown to exhibit plenty of benefits for infants, such as prebiotic activity shaping the gut microbiota and immunomodulatory and anti-inflammatory activity. For some pathogenic bacteria, antimicrobial activity has been proved, but most studies focus on group B streptococci. In the present study, we investigated the antimicrobial and antibiofilm activities of the total and fractionated HMOs from pooled human milk against four common human pathogenic Gram-negative species (Klebsiella pneumoniae, Acinetobacter baumannii, Pseudomonas aeruginosa, and Burkholderia cenocepacia) and three Gram-positive species (Staphylococcus aureus, Enterococcus faecium, and Enterococcus faecalis). The activity of HMOs against enterococci and B. cenocepacia are addressed here for the first time. We showed that HMOs exhibit a predominant activity against the Gram-positive species, with E. faecalis being the most sensitive to the HMOs, both in planktonic bacteria and in biofilms. In further tests, we could exclude fucosyllactose as the antibacterial component. The biological significance of these findings may lie in the prevention of skin infections of the mother’s breast as a consequence of breastfeeding-induced skin laceration and/or protection of the infants’ nasopharynx and lung from respiratory pathogens such as staphylococci.