Based on the results of over 10 years of research and development by the authors, this book presents a broad cross section of dynamic programming (DP) techniques applied to the optimization of dynamical systems. The intention of the research effort was to develop a robust path planning/trajectory optimization tool that did not require an initial guess, partially met with a combination of DP and homotopy algorithms. DP algorithms are presented here with a theoretical development, and their successful application to a variety of practical engineering problems is emphasized. The organization of the book enables the reader to use DP algorithms before thoroughly comprehending the full theoretical development. Moreover, the applications of DP algorithms are easily adapted to the reader's own interests and problems. Illustrative examples that surround linear systems applications gradually introduce DP algorithm development, while design steps applied to case studies illustrate the ideas and principles behind DP algorithms.