In this thesis we apply the optimal transport (OT) theory to various disciplines of applied and computational mathematics such as scientific computing, numerical analysis, and dynamical systems. The research consists of three aspects: (1) We focus on solving OT problems from different perspectives including (a) direct approximation of the OT map in high dimensions; (b) particle evolving method for generating samples from the optimal transport plan; (c) learning high dimensional geodesics joining two given distributions. These different formulations find their own applications under distinct settings in diverse branches of data science and machine learning. We derive sample-based algorithms for each project. Our methods are supported by theoretical guarantees and numerical justifications. (2) We develop and analyze a sampling-friendly method for high dimensional Fokker-Planck equations by leveraging the generative models from deep learning. By utilizing the fact that the Fokker-Planck equation can be viewed as gradient flow on probability manifold equipped with certain OT distance, we derive an ordinary differential equation (ODE) on parameter space whose parameters are inherited from the generative models. We design a variational scheme for solving the proposed ODE. Both the convergence and error analysis results are established for our method. The performance and accuracy of the proposed algorithm are verified via several numerical examples. (3) We present a novel definition of Hamiltonian process on finite graphs by considering its corresponding density dynamics on probability manifold. We demonstrate the existence of such Hamiltonian process in many classical discrete problems, such as the OT problem, Schr\"odinger equation as well as Schr\"odinger bridge problem (SBP). The stationary and periodic properties of Hamiltonian processes are investigated in the framework of SBP. / Ph.D.