This monograph presents probabilistic representations for classical boundary value problems of mathematical physics, and is devoted to the walk on boundary algorithms. Compared to the well-known Wiener and diffusion-path integrals, the trajectories of random walks in this publication are simulated on the boundary of the domain as Markov chains generated by the kernels of the boundary integral equations equivalent to the original boundary value problem. The book opens with an introduction for solving the interior and exterior boundary value for the Laplace and heat equations, which is followed by applying this method to all main boundary value problems of the potential and elasticity theories. It should be of interest to specialists in the field of applied and computational mathematics and applied probability, as well as for postgraduates studying new numerical methods for solving PDEs.