This thesis explores Uncertainty Quantification for probabilistic models of physical systems. In particular, the thesis works towards a long term goal of automating numerical methods by combining traditional techniques with perspectives from Machine Learning. Uncertainty Quantification refers to the use of probabilistic models for estimating the potential variability of unknown quantities that may be present. For engineering, the types of models of interest are those that estimate the behaviour of future states of the world by simulation. In particular, as these are relevant to the types of problems arising in Civil Engineering, Partial Differential Equations with probabilistic inputs are analysed. Uncertainty Quantification can be used to assess the risk associated with proposed designs and existing structures rigorously by explicitly calculating the variability of unknown quantities. This thesis begins by detailing how Uncertainty Quantification can be incorporated into a decision making framework by reference to Game Theory and Bayesian probability. Spatially distributed data is often encountered in Civil Engineering. Probabilistic models of spatial variability, random fields, are discussed extensively. Simulation and modelling techniques for random fields are presented and analysed to facilitate later developments in the thesis. Rare event reliability analysis is discussed in detail. Civil Engineering projects frequently have a ``low probability, high consequence'' risk profile that presents unique computational challenges. Numerical methods for rare event probabilistic analysis based on Markov Chain Monte Carlo are demonstrated. The computational challenges associated with probabilistic computational mechanics are significant. If suitable self-improving numerical methods could be developed, then these computational challenges would be lessened. The thesis introduces an Artificial Neural Network surrogate model method to improve the efficiency of sampling based Uncertainty Quantification. Following this, the structure of probabilistic computational mechanics problems are analysed from a Bayesian perspective. From this interpretation of the solution of Partial Differential Equations, an adaptive Element Free Galerkin method is derived. The combination of Machine Learning, Bayesian probability and Partial Differential Equations presented indicates directions for future research in automated probabilistic numerical techniques.