The nucleoskeleton has been associated with partitioning the genome into active and inactive compartments that dictate local transcription factor (TF) activity. However, recent data indicate that the nucleoskeleton and TFs reciprocally influence each other in dynamic TF trafficking pathways through the functions of LEM proteins. While the conserved peripheral recruitment of TFs by LEM proteins has been viewed as a mechanism of repressing transcription, a diversity of release mechanisms from the lamina suggest this compartment serves as a refuge for nuclear TF accumulation for rapid mobilization and signal stability. Detailed mechanisms suggest that TFs toggle between nuclear lamina refuge and nuclear matrix lamin-LEM protein complexes at sites of active transcription. In this review we will highlight emerging LEM functions acting at the interface of chromatin and nucleoskeleton to create TF trafficking networks. Copyright © 2020 Elsevier Ltd. All rights reserved.