Culture of bacteria from bronchial secretions in respiratory patients has low sensitivity and does not allow for complete assessment of microbial diversity across different bronchial compartments. In addition, a significant number of clinical studies are based on sputum samples, and it is not known to what extent they describe the real diversity of the mucosa. In order to identify previously unrecognized lower airway bacteria and to investigate the complexity and distribution of microbiota in patients with chronic obstructive pulmonary disease (COPD), we performed PCR amplification and pyrosequencing of the 16S rRNA gene in patients not showing signs or symptoms of infection. Four types of respiratory samples (sputum, bronchial aspirate, bronchoalveolar lavage, and bronchial mucosa) were taken from each individual, obtaining on average >1,000 16S rRNA sequences per sample. The total number of genera per patient was >100, showing a high diversity, with Streptococcus, Prevotella, Moraxella, Haemophilus, Acinetobacter, Fusobacterium, and Neisseria being the most commonly identified. Sputum samples showed significantly lower diversity than the other three sample types. Lower-bronchial-tree samples, i.e., bronchoalveolar lavage and bronchial mucosa, showed a very similar bacterial compositions in contrast to sputum and bronchial aspirate samples. Thus, sputum and bronchial aspirate samples are upper bronchial tree samples that are not representative of the lower bronchial mucosa flora, and bronchoalveolar lavage samples showed the results closest to those for the bronchial mucosa. Our data confirm that the bronchial tree is not sterile in COPD patients and support the existence a different microbiota in the upper and lower compartments.