This is the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control. Neuro-dynamic programming uses neural network approximations to overcome the "curse of dimensionality" and the "curse of modeling" that have been the bottlenecks to the practical application of dynamic programming and stochastic control to complex problems. The methodology allows systems to learn about their behavior through simulation, and to improve their performance through iterative reinforcement. This book provides the first systematic presentation of the science and the art behind this exciting and far-reaching methodology. The book develops a comprehensive analysis of neuro-dynamic programming algorithms, and guides the reader to their successful application through case studies from complex problem areas.