In this paper by use of cohomology complex of $3$-Leibniz algebras, the definitions of Leibniz bialgebras (and Lie bialgebras) are extended for the case of $3$-Leibniz algebras. Many theorems about Leibniz bialgebras are extended and proved for the case of $3$-Leibniz bialgebras ($3$-Lie bialgebras). Moreover a new theorem on the correspondence between $3$-Leibniz bialgebra and its associated Leibniz bialgebra is proved. $3$-Lie bialgebra as particular case of the $3$-Leibniz bialgebra is investigated. Finally, some simple examples are discussed in detail.