Plant-based extracts present a large source of natural immune modulators, many of which have been used in traditional medicine for centuries. Recent research efforts have identified plant extracts as potential modulators of Toll-like receptors (TLRs), the first responders in immunological defenses in normal and disease conditions. This review aims to provide a comprehensive discussion of the modulatory effects of plant-based extracts on TLR expression, signaling, and activation. We organized the review by extraction solvent and plant part showing how they impacted the TLRs. The phytochemical components of the extracts discovered to enable these effects are diverse and vary based on the plant part. The role of the extraction solvent and differences between the different phytochemical components, such as phenolics and polysaccharides, are discussed. Plant extracts hold promising treatments for controlling inflammation and, conversely, for stimulating the immune response. Further research is needed to identify bioactive components of the extracts, mechanisms of their action, and in vivo pharmacological effects using appropriate disease models to ultimately adapt the findings for clinical use.