Accurate prediction of structures’ durability has long been a hot topic in civil engineering. This paper proposed a new solution to this topic, i.e., integrating physical constraints into machine learning models. Machine learning methods have proven to have remarkable abilities in approximating complex functions, which provides a potential tool for finding the model that complies with all physical constraints. This paper used this method to predict the carbonation status of concrete. Partial differential equations governing the diffusion process of CO2 in concrete were used as physical constraints, which consider the effects of aggregate, load, and interface transition zone. Then, this study integrated physical constraints into deep neural network models and established CO2’s one- and two-dimensional diffusion models. Results showed that physical constraints could be well combined with deep neural networks and this new method can accurately simulate the diffusion of CO2. This method has great potential in solving multi-physics problems.