There are real strategic situations where nobody knows ex ante how manyplayers there will be in the game at each step. Assuming that entry and exitcould be modelized by random processes whose probability laws are commonknowledge, we use dynamic programming and piecewise deterministicMarkov decision processes to investigate such games. We study the dynamicequilibrium in games with randomly arriving players in discrete and continuoustime for both finite and infinite horizon. Existence of dynamic equilibriumin discrete time is proved and we develop explicit algorithms for bothdiscrete and continuous time linear quadratic problems. In both cases weoffer a resolution for a Cournot oligopoly with sticky prices.