An introductory level text on stochastic modelling, suited for undergraduates or graduates in actuarial science, business management, computer science, engineering, operations research, public policy, statistics, and mathematics. It employs a large number of examples to show how to build stochastic models of physical systems, analyse these models to predict their performance, and use the analysis to design and control them. The book provides a self-contained review of the relevant topics in probability theory: In discrete and continuous time Markov models it covers the transient and long term behaviour, cost models, and first passage times; under generalised Markov models, it covers renewal processes, cumulative processes and semi-Markov processes. All the material is illustrated with many examples, and the book emphasises numerical answers to the problems. A software package called MAXIM, which runs on MATLAB, is available for downloading.