Abstract
Machine learning (ML) is emerging as a powerful tool to predict the properties of materials, including glasses. Informing ML models with knowledge of how glass composition affects short-range atomic structure has the potential to enhance the ability of composition-property models to extrapolate accurately outside of their training sets. Here, we introduce an approach wherein statistical mechanics informs a ML model that can predict the non-linear composition-structure relations in oxide glasses. This combined model offers an improved prediction compared to models relying solely on statistical physics or machine learning individually. Specifically, we show that the combined model accurately both interpolates and extrapolates the structure of Na2O–SiO2 glasses. Importantly, the model is able to extrapolate predictions outside its training set, which is evidenced by the fact that it is able to predict the structure of a glass series that was kept fully hidden from the model during its training.
Similar content being viewed by others
Introduction
Oxide glasses are a family of materials mostly known for their applications as windows due to their high transparency and durability1,2. During the last few decades, however, oxide glasses have found more and more high-tech applications, such as optical fibers for telecommunication and data transmission, bioactive glass for bone and tissue regeneration, and nuclear waste immobilization materials3,4,5,6,7. For each of these applications, the physical properties of the glass have been tuned by changing the composition to meet the requirements according to the application, such as low but controllable chemical durability in window glasses, controlled release of ions in bioactive glasses, and high chemical durability for nuclear waste immobilization.
An accelerated design of glass compositions and processing techniques is thus urgently needed8,9, especially since oxide glasses can be made from more than half of the elements in the periodic table with anywhere from two to over twenty individual elements. The number of possible compositional combinations thus exceeds by several orders of magnitudes what can be prepared and examined in any laboratory10. As such, predictive composition-property models have been highly valuable in guiding materials development11, especially those based on machine learning12,13,14,15,16. The previous machine learning models within this field have been trained based on composition input and property output data. However, while successful in property prediction within the compositional range used for training, these models largely fail to extrapolate outside the compositional region of the training set17,18. This is due to the fact that, unlike models rooted in physics, machine learning models are solely based on data and, hence, can violate the laws of physics or chemistry19,20. This paper seeks to address this gap.
It is well known that the relationship between the short-range order (SRO) structure of oxide glasses and their properties is more direct and informative than that between composition and properties21,22,23,24. This is partly due to the fact that glass properties often exhibit a non-linear dependence on composition8, which makes it challenging for models to capture such non-linearity and yield robust extrapolations. In contrast, the relationships between local structure (e.g., as captured from the glass connectivity) and properties are often fairly linear25. The linear nature of structure-property relationships makes it significantly easier for models to generalize well outside their training sets, when extrapolated toward unexplored compositional spaces19,26,27. However, a key challenge is to obtain sufficient and reliable structure data for model training due to the time-consuming nature of experimental structure characterization. The typical method to determine the local structure of the atoms in a glass is solid-state nuclear magnetic resonance (NMR) spectroscopy28,29, the accuracy of which worsens as more elements are present in the glass composition since the interpretation of the data becomes increasingly complex30.
Recently, statistical mechanical modeling has shown great promise in predicting the SRO structure of multicomponent oxide glasses based on knowledge of binary and ternary glasses31,32. The main idea of this approach is to consider the possible reactions that can occur between all the species in the glass (e.g., the formation of non-bridging oxygen atoms or the charge compensation of the network former) and determine their probabilities based on their respective enthalpies and temperature. The drawback of the statistical mechanical model is that the enthalpy interactions considered are kept to a minimum to reduce the required number of model parameters, but this can lead to some level of oversimplification and has been found to systematically over- or underestimate the fractions of some structural units33. While these errors could be corrected on a per-system basis, machine learning should be able to learn and correct for these systematic errors. Informing a machine learning model with the thermodynamics of cationic interactions (as done in the statistical mechanical model) is thus expected to offer a model that combines the extrapolation ability of statistical mechanics with the accuracy of machine learning. The aim of such combined models is to predict the compositional evolution of structural units in large compositional regions even when trained on a relatively small dataset, hence allowing for large-scale composition-structure-property modeling.
In this study, we present three different approaches to large-scale structure prediction in oxide glasses as depicted in Fig. 1. Firstly, we demonstrate the results obtained by a purely statistical mechanical model trained only on experimental data for binary glasses. Secondly, we use a multilayer perceptron neural network34 (MLP-NN) model to calculate the glass structure using only the composition as input. Thirdly, we use an MLP-NN model trained on both composition and the statistical mechanical results. We use all three models to predict structural data and make comparisons against the experimental data. Specifically, we then apply the three different modeling approaches to the Na2O–SiO2 binary glass system to illustrate the differences in predictions as a function of glass composition. Finally, all three models are used to predict the structure of a Na2O–P2O5–SiO2 glass system, which none of the models have been trained on, so as to quantify the ability of each model in offering robust extrapolations toward unknown glass compositions.
Results
Statistical mechanical structure prediction in multicomponent systems
Before comparing the three different approaches for predicting the composition-structure relation (see Fig. 1), we first address shortcomings of the statistical mechanical model. As previously mentioned, the parameters needed in the statistical mechanical modeling procedure are obtained by training the model on binary oxide glasses. The parameters are then directly used to calculate structures in more complex systems (i.e., with more than two different oxides). Figure 2 shows model predictions obtained by the statistical mechanical model described in the Methods section, which are compared to measured structure data for different glass families. In all the subfigures, the black symbols represent model prediction for the simple two-component glasses used for the model training, while the red symbols represent data for glasses with three or more components. It is thus the comparison of model prediction and experimentally measured data for the red symbols that reflects the prediction quality of the model. Specifically, in Fig. 2a, the red symbols represent data for the M2O–SiO2–B2O3 glass system, where M is either Li, Na or Cs. Here, the model prediction accuracy for the test data is within the same range as for the training data, showing the potential extrapolative power of the model. In Fig. 2b (for Na2O–Al2O3–SiO2 system) and Fig. 2c (for Na2O/CaO–P2O5–SiO2 system), however, the model predictions differ significantly from the training data to the test data. This illustrates how the direct transfer of the parameters from one system to another can be insufficient to capture all interactions occurring in multicomponent glasses. In the case of the systems in both Fig. 2b, c, the model prediction accuracy can be improved with a single parameter for the additional interaction occurring in the mixed glasses. As such, the error by the model is very systematic and it should be possible to correct it by machine learning.
Structure prediction by statistical mechanics, machine learning and combined model
The composition-structure MLP-NN model only has the glass composition as input, while the combined model has both the glass composition and the initial results from the statistical mechanical model as inputs. While this may appear to be an advantage for the combined model, the results from the statistical mechanical modeling are obtained by knowing only the compositions as well. As shown in Eq. (2) in the Methods section, a fictive temperature is required when applying the statistical mechanical model. In this study, however, to ensure a fair comparison between the physics-informed and physics-blind NN models, that value has been fixed to 700 K, which is an approximate average value for oxide glasses13. While Tf has an effect on the structure distribution, it has previously been shown to have only a minor effect within the temperature range of experimentally obtained glasses35. Additionally, excluding Tf is required for the model to be applied to predict the structural fractions (i.e., the relative numbers of bridging and non-bridging oxygens associated with the different network formers) in previously unexplored compositions as Tf must be obtained experimentally. However, we note that an additional model could be applied to estimate Tf values for the glass compositions before modeling the structure, but this would both introduce additional uncertainty from the Tf prediction and add compositional restriction from the additional model. As such, the pure composition-structure MLP-NN model and the combined model both rely on only compositions as input data. Here, the inclusion of statistical mechanics should allow the neural network to learn a baseline for the non-linear relation between composition and structure captured by the thermodynamic parameters used in statistical mechanics.
Before investigating the model results, the MLP-NN training method and results are addressed. When considering the neural network architecture, the number of hidden layers and the number of neurons per layer highly influence the outcome of the model. An overly simple architecture (too few layers and neurons) will result in underfitting, where the neural network has too few nodes to find a good path between input and output, while a too complex model will result in unnecessarily high computational costs as well as the potential of overfitting, where a perfect path is found between input and output but also capturing the data uncertainty as model features. Previous work by Hinton et al.36 has found that one hidden layer NN can approximate a continuous map** from one finite space to another, while two hidden layers NN can approximate any smooth map** to any accuracy. Additional layers can learn complex representations but is deemed excessive for the relatively small dataset used in this study.
To this end, we have used grid search cross-validation to find that two hidden layers is indeed most suitable for the data in this study. Next, to find the optimal number of neurons in each of the two hidden layers, we performed parameter optimization as described in the Methods section. Figure 3 shows a two-dimensional illustration of the optimization of the number of neurons in the hidden layers, where the optimal model complexity with the lowest root-mean-square errors (RMSE) is for 13 and 16 neurons for the first and second layer, respectively. Figure 3a illustrates the transition from underfitting with too few neurons to overfitting with too many neurons and how the optimization algorithm finds the optimal number of neurons. Furthermore, we have checked the predictive ability of the combined model based on a smaller dataset by using a partial dataset for training and testing (from 50 to 100%) with the same train/test split ratio of 0.9/0.1. As shown in Fig. 3b, the RMSE decreases initially and then reaches a plateau value when using more than 80% of the dataset. This suggests that the MLP-NN model learns from the thermodynamic information indirectly supplied by the statistical mechanical approach, without needing more training data.
Figure 4a–c show the prediction results from statistical mechanics, MLP-NN, and the combined model, respectively. When examining the RMSE, the statistical mechanics model (approach 1) performs slightly better than the composition-structure MLP-NN model (approach 2) (RMSE of 6.6% and 6.8% for statistical mechanics and MLP-NN, respectively). Additionally, we note that the statistical mechanical model is trained on a smaller dataset relative to the test set and the training sets consist of data for only binary glasses while the test sets include data for multicomponent glasses. This is unlike the machine learning models, where the test set is a random subset of the total composition-structure dataset. In summary, the statistical mechanical model predicts, at a higher accuracy, a larger test set with a smaller training set and with higher extrapolation than the MLP-NN model. When combining the statistical mechanics model with MLP-NN (approach 3), the RMSE drops significantly (35–40% reduction of the RMSE). This also supports the findings from Fig. 3b, i.e., the MLP-NN model learns from the thermodynamic information indirectly supplied by the statistical mechanical approach.
Applications of the structure prediction models
To further investigate the accuracy of the predictions generated by the three different models, Fig. 5a, b show the model predictions of the fractions of structural units in the Na2O–SiO2 glass system37. The statistical mechanical model prediction is shown with the solid lines, while the MLP-NN and the combined model are shown with dashed lines in Fig. 5a, b, respectively. Statistical mechanics predicts the most likely composition-structure relation as it captures the reaction mechanisms specific to the system and is trained on all the data for this system alone. On the other hand, the machine learning models are trained on data for all systems simultaneously and the results in the Na2O–SiO2 system reflect the predictive power of the models. The most noteworthy difference between the two different machine learning approaches is the degree of linearity in their prediction. While the pure composition-structure MLP-NN model predicts a highly linear transition from one composition-structure data point to another in Fig. 5a, the predictions of the combined model are much more non-linear and capture the structural transitions in Fig. 5b. Both machine learning models seem to have prediction accuracy in the high-modifier region when extrapolating outside the dataset used for training (symbols in Fig. 5a, b), but not in the low-modifier region which is also an extrapolation. The inaccuracy in the high-modifier region may be caused by input data from multicomponent glass systems. The inaccuracy is acceptable as it arises in a compositional area where glass formation is unlikely. Based on the generated MLP-NN model, we have also checked the prediction ability based on part of the input information by purely using chemical composition or statistical mechanical. This is achieved by varying either the chemical composition or the statistical mechanics information while kee** the other fixed to check the Q3 fraction as a function of Na2O content in the Na2O–SiO2 system. Note that the changes are made based on the input information of the Na2O–SiO2 glass that exhibits the highest Q3 fraction. As shown in Fig. 5c, the combined model can well predict the Q3 distribution as a function of Na2O composition. When solely using the statistical mechanical model, most of the non-linear features of the structural transition can be well captured, whereas the prediction based only on input from composition does not capture the non-linear composition vs. structure relation.
To further test the ability of the different models to extrapolate, we have used them to predict structures in a set of glasses from a single study38, which was not included in the training set (Fig. 5d). Specifically, we consider Na2O–P2O5–SiO2 glasses, which is a glass family for which the statistical mechanical model has been unable to make structural predictions without additional input besides composition. Interestingly, the statistical mechanics model offers the worst predictions, while the pure composition-structure MLP-NN model offers some improvement, and the combined model has a very high prediction accuracy. Remarkably, our machine learning model outperforms the physics-based statistical mechanics model for an extrapolation task. In future work, this approach will be applied to predict the structure distribution of glass series with additional oxide components, but this requires experimental work outside the scope of this study.
Discussion
Machine learning is one of the most promising tools for large-scale glass property predictions12. While current models are trained on a composition-property relation, previous work has indicated that structure-property relations are more systematic and would offer improved ability of the property models to extrapolate39,40. It could be argued that the machine learning models learn the composition-structure relations with enough data, but this study has shown that the composition-structure relation is too non-linear for pure MLP-NN to learn the complex relations with a limited dataset. We have allowed the neural network to learn the thermodynamic contribution to the structure formation by supplying the statistical mechanical structure results as an additional layer of input data. In other words, the statistical mechanics model provides a baseline so that the neural network model only has to learn how to correct the deficiencies of the former–an easier task than having to correct the structure from scratch. This is referred to as a multi-fidelity approach since our model is learning from both low-fidelity data (provided by the statistical mechanics model) and high-fidelity data (experimental composition-structure data)41.
As a result, the modeling approach in this study could improve the predictive and extrapolative power of composition-property models by offering the structural component and allowing for composition-structure-property modeling. That is, while purely physics-based models can provide insights into glass structure and offer accurate extrapolations in simple systems, they tend to oversimplify the problem in more complex systems, where multiple interactions occur between the different glass components. On the other hand, data-driven models, such as those based on machine learning, can accurately predict glass structure within the compositional space used for the model training. However, since these models typically do not embed any physics knowledge, they struggle to predict properties accurately outside of the compositional range used for training. The combined model in this work addresses this challenge by relying on experimental composition-structure data as well as a statistical mechanical model. By enforcing the physical laws of interactions between the oxide components in the glassy networks, the present model successfully integrates (noisy) data and mathematical models through a multilayer perceptron neural network. Therefore, this combined model offers an improved prediction of non-linear composition-structure relations in oxide glasses compared to models relying solely on statistical physics or machine learning individually. To this end, we note that our approach is different from the physics-informed ML approaches that rely on adding some physical constraints in the cost function of the ML model before conducting the training. Here, the physical information is provided by a frozen, independent statistical model that is not trained like a conventional ML model.
Methods
Oxide glass structure
The basic SRO structure of oxide glasses comprises network-forming cations such as Si, P, or B, forming a three-dimensional network by bridging to one another through oxygen atoms42. Electropositive cations such as Li, Na, and Ca form ionic bonds with oxygen and weaken the network by breaking the bridging oxygens (BOs) and forming non-bridging oxygen (NBOs) in the process43. The reaction of modifier ions with the glassy network is typically both entropically and enthalpically favorable as the inclusion of ionic bonds and NBOs increases the system’s entropy, and energy is released when modifier-former bonds obtain a lower potential energy than the combination of former-former and modifier-modifier bonds44,45. The SRO structures formed in oxide glasses are summarized by the coordination number of the network-forming cations and the ratios of BOs to NBOs associated with that network forming cation. Modeling examples are given in the following section.
Statistical mechanical model
Based on the knowledge of the enthalpy associated with modifier reaction with each possible structural unit and the entropy released, the probability of all possible reactions can be calculated as a function of composition and temperature using statistical mechanics46. The entropy is given by the fractions of all structural units before the reaction occurs and scales with the temperature at which the reaction occurred during glass formation. Here, the temperature is assumed to be the fictive temperature (Tf), where the glass relaxation time becomes too high as compared to the observation time for the atoms in the glass-forming liquid to rearrange. Thus, by knowing the fictive temperatures of a glass-forming system, only the enthalpy values for each modifier-structure unit pair are unknown and required to model the composition-structure relation for a given system. If the structure distributions are known for the glass system, e.g., as obtained by NMR spectroscopy, the enthalpy values can be obtained by fitting the model (see below) to the structure data47. Then, simple glass systems can be used to obtain the enthalpy values, which, in turn, can be used to compute the structure of more complex compositions. For example, enthalpy values obtained by fitting the model to experimental data of Na2O–SiO2 and Na2O–B2O3 glasses, respectively, can be used directly to predict the structures in Na2O–SiO2–B2O3 glasses31. As such, the model offers high extrapolation ability by capturing both the enthalpic and entropic contributions to the formations of SRO structural units—since the physics governing the structure of the ternary system is essentially the same as that governing the binary systems. With this approach, the number of glasses within the prediction range of the model expands exponentially as the training data expands linearly; however, the assumption of direct correlations between reactions in binary and ternary glasses has been found to be insufficient in some systems. That is, the model can show a systematic error when calculating the structures of ternary glasses from the inputs obtained from their binary counterparts25.
The statistical mechanical model used to predict the SRO structures in oxide glasses was first proposed by Mauro45 and later implemented and validated on binary oxide glasses by Bødker et al.47. Considering again Na2O–B2O3–SiO2 as an example, we can calculate the probability for sodium to interact with boron over silicon using a non-central (weighted) distribution function. We do so by assuming that the energy associated with a given network modifier-former interaction is independent of the glass composition. The statistical entropy associated with each type of interaction does, however, change depending on the glass composition, since the composition dictates the number of each element available for an interaction. Moreover, a species is not replaced after the interaction occurs. This results in a hypergeometric distribution that describes the evolution of network former species as a function of the modifier concentration. As the entropy of the system changes with composition, so does the probability for the modifier-former interaction, thereby requiring a numerical solution to predict the structural evolution of network former species.
The probability for a given modifier-structure unit interaction (pi) is calculated using a Wallenius type non-central hypergeometric distribution function48,
where
In Eq. (1), ω is the modifier concentration, wi is the weighting factor (ranging from 0 to 1) associated with structure unit i, gi is the degeneracy of species i and ni,ω is the total fraction of species i that has already interacted at modifier concentration ω. The double summation in the denominator is over all species J and each modifier concentration ω up to, but not including the current concentration ω. In Eq. (2), k is the Boltzmann constant and Hi is the energy barrier associated with species i49. As shown elsewhere, the weighting factor given by Eq. (2) is derived from the Boltzmann distribution function31. In summary, Eqs. (1) and (2) show that the probability for a modifier to interact with structure unit i is calculated by the fraction of that unit at modifier concentration ω weighted by the Boltzmann factor (scaling with Hi and Tf) relative to the sum of all species weighted by their individual Boltzmann factors. For each concentration step, the fraction ni,ω of network former species i interacting with the modifier ion can be calculated from the probability of the interaction and its fraction at the previous concentration step ω – 1. Then, that fraction is subtracted from the remaining amount of network forming species i at the next concentration, which is used to calculate the new probability, and so on. The application of Eqs. (1) and (2) to predict the structure of binary and ternary oxide glasses has been described in detail elsewhere31,47.
Next, we will explain the numerical procedure of calculating the structure distribution as a function of composition, using the M2O–B2O3–SiO2 system, where M is an alkali metal, as an example. The structural units of interest in the silicate part of the borosilicate network are the Qn units, where n is the number of BO per tetrahedron (n = 0, 1, 2, 3, or 4), while those of interest in the borate part are the Bn units, again with n = 0, 1, 2, 3, or 4. Note that Q in the silicate network is an abbreviation for quaternary, since all silicate structural units are four-fold coordinated, while in the borate network, B4 is four-fold coordinated and the remaining units are three-fold coordinated.
First, we consider the reaction mechanisms for B3 and B4 structural groups when interacting with an alkali modifier oxide M2O as previously described31,
That is, a B3 unit can either form a B4 or a B2 when reacting with a modifier ion, following the well-known boron anomaly50,51. Here, the modifier ion is charge balanced by either a negative NBO for the B2 unit or by a negatively charged boron atom for the B4 unit due to the extra covalent bond formed in the 4-fold coordinated state. Previous studies50,52 have shown that the reaction in Eq. (3) dominates at low-modifier concentration, while the reaction in Eq. (4) dominates at higher modifier concentration. To capture this behavior, a critical concentration parameter was introduced (αB4/B2), that changes from 1 to 0, allowing only one of the two reactions to occur at any point. This is a structural simplification but it has been shown to capture the structural evolution in borate containing glasses accurately31.
The following reactions occur for all silicate structural units when interacting with a modifier ion (and for the B2 and B1 structural units),
With all the possible modifier interactions established, the fractions of the structural units at concentration step ω are calculated as,
That is, the fraction of B3 at modifier concentration step ω (B3ω) equals the fraction of B3 at the previous concentration step (B3ω-1) but subtracting the fraction that reacts at ω \(\left( {p_{B^3,\omega }} \right)\). In Eq. (7), the concentration of Q3 at step ω (Q3ω) is found by the addition of Q4 reacting \(\left( {p_{Q^4,\omega }} \right)\) and subtraction of Q3 reacting \(\left( {p_{Q^3,\omega }} \right)\). The additional \(p_{B^4,\omega } \cdot p_{B^3,\omega }\) and \(p_{B^4,\omega } \cdot p_{Q^3,\omega }\) terms in Eqs. (6) and (7), respectively, arise due to the boron anomaly. Further explanation of the modeling procedure is given elsewhere31. Typically, the fitting parameters of the model would be the set of Hi values for all structure units, but the relative Hi values are first established in binary sodium borate and sodium silicate glasses. The Hi values from binary glasses are then transferred to predict the structural evolution in the ternary sodium borosilicate glasses with only one free parameter, namely the conversion factor (wSi,B). This parameter is needed, since all enthalpy values are calculated relatively within each system, i.e., the HQ3 in the sodium silicate system is relative to the HQ4 parameter, while the HB2 in the sodium borate system is relative to the HB3 parameter. That is, the model only needs to be fitted on composition-structure data for binary glasses and a few ternary glass systems to be able to calculate the structural distributions in any combinations of the elements that have been used for the training.
The enthalpy values used for this study were obtained using the StatMechGlass pip package in the object-oriented programming language, Python53. The StatMechGlass package was also used to compute the following statistical mechanics results in multicomponent glasses without any fitting.
Multilayer perceptron neural network
Machine learning models, and artificial neural networks in particular, have been successfully applied to glass systems to capture composition-property relations for properties such as glass-forming ability54 and thermal conductivity55. The composition-property relation in oxide glasses is often highly non-linear and complex, while the structure-property relation has been shown to be more directly correlated56. The aim of this study is to combine the statistical mechanics model with machine learning to calculate the fractions of SRO structures of the compositions. Such model could be used as input to property models, hence allowing for training on both composition and structure.
MLP-NN is a commonly used artificial neural network, which has been successfully used in learning the relationship between inputs and outputs within both linear and non-linear datasets57,58. MLP-NN has previously been successfully used for glass property prediction and we have therefore chosen it also for this work12,13,59. The structure of MLP-NN consists of a series of sequential layers: an input layer, an output layer, and one or more hidden layers in between. The input layer distributes the input information to the hidden layer. The hidden layer processes and transmits the information from the input layer to the output layer processed by certain sets of adjustable network parameters such as weights and biases. To achieve an accurate prediction model, the network structure of MLP-NN needs to be carefully tuned. MLP-NN with excessive neurons is likely to cause overfitting, while a network with insufficient neurons will have difficulty learning the datasets (i.e., it will tend to exhibit underfitting). The hyperparameters considered herein are the number of neurons per layer and the number of layers—that is, the non-trainable parameters that must be set before training the model. During the training of the MLP-NN, the weights and biases of the neural network will be continuously adjusted until network optimization is achieved by minimizing the error between the output value and the actual value. We used 10-fold cross validation to find the optimal architecture during hyperparameter optimization. First, however, a random 10% subset of the data was withdrawn from the dataset to be used as a test set after finishing the hyperparameter optimization. The average mean squared error (MSE) of the cross validation was used as the cost function during hyperparameter optimization. The hyperparameters were then optimized (by determining the set of hyperparameters minimizing the validation set error) using SciPy’s basin-hop** optimization algorithm60, which is well-suited for finding the global minimum in the parameter space after which the model was retrained on randomly sampled 90 % of the input data.
Data layout
The input (composition) vs. output (fraction of SRO structural units) data for the three different modeling procedures were gathered from published work in the literature on ~400 different glass compositions. To ensure a low uncertainty of the experimental composition-structure data, we only used data with measured glass compositions. All data and references are found in Supplementary Table 1. This includes data for the most commonly used network formers (SiO2, P2O5, B2O3, and Al2O3) in oxide glasses and a combination of six different network modifier oxides (Na2O, Li2O, K2O, Cs2O, CaO, and MgO). The glass compositions used as input in all three model procedures are the molar concentrations in the percentage of the different oxides shown in Supplementary Table 1. The output of the data is the measured structure distributions of the glasses. For the combined model, the results obtained by statistical mechanical modeling and the glass compositions were concatenated as input data. The statistical mechanical model is trained on structure data in binary oxide glasses only, while the MLP-NN model is trained on a random subset of all glass structures. As such, the testing results of the statistical mechanical model represent structure prediction extrapolation, while the MLP-NN interpolates.
Data availability
The data supporting the results within this paper are available at https://github.com/OxideGlassGroupAAU/StatMechML. Experimental structure data from literature are also summarized in Supplementary Table 1.
Code availability
Python scripts for this article can be found at https://github.com/OxideGlassGroupAAU/StatMechML.
References
Conradt, R. Chemical durability of oxide glasses in aqueous solutions: a review. J. Am. Ceram. Soc. 91, 728–735 (2008).
Axinte, E. Glasses as engineering materials: a review. Mater. Des. 32, 1717–1732 (2011).
Miyashita, T. & Manabe, T. Infrared optical fibers. IEEE Trans. Microw. Theory Tech. 30, 1420–1438 (1982).
Rashed, A. N. Z., Mohamed, A. E.-N. A. E. G., Hanafy, S. A. E. R. S. & Aly, M. H. A comparative study of the performance of graded index perfluorinated plastic and alumino silicate optical fibers in internal optical interconnections. Optik 20, 9259–9263 (2016).
Mouriño, V., Newby, P. & Boccaccini, A. R. Preparation and characterization of gallium releasing 3-D alginate coated 45S5 Bioglass® based scaffolds for bone tissue engineering. Adv. Eng. Mater. 12, B283–B291 (2010).
Jones, J. R. Review of bioactive glass: from Hench to hybrids. Acta Biomater. 9, 4457–4486 (2013).
Plodinec, M. J. Borosilicate glasses for nuclear waste imobilisation. Glass Technol. 41, 186–192 (2000).
Mauro, J. C., Tandia, A., Vargheese, K. D., Mauro, Y. Z. & Smedskjaer, M. M. Accelerating the design of functional glasses through modeling. Chem. Mater. 28, 4267–4277 (2016).
Mauro, J. C. Grand challenges in glass science. Front. Mater. 1, 20 (2014).
Zanotto, E. D. & Coutinho, F. A. B. How many non-crystalline solids can be made from all the elements of the periodic table? J. Non-Cryst. Solids 347, 285–288 (2004).
Mauro, J. C., Philip, C. S., Vaughn, D. J. & Pambianchi, M. S. Glass science in the United States: current status and future directions. Int. J. Appl. Glass Sci. 5, 2–15 (2014).
Liu, H., Fu, Z., Yang, K., Xu, X. & Bauchy, M. Machine learning for glass science and engineering: a review. J. Non-Cryst. Solids 557, 119419 (2021).
Cassar, D. R., de Carvalho, A. C. P. L. F. & Zanotto, E. D. Predicting glass transition temperatures using neural networks. Acta Mater. 159, 249–256 (2018).
Ravinder, R. et al. Deep learning aided rational design of oxide glasses. Mater. Horiz. 7, 1819–1827 (2020).
Hu, Y.-J. et al. Predicting densities and elastic moduli of SiO2-based glasses by machine learning. npj Comput. Mater. 6, 1–13 (2020).
Liu, H., Fu, Z., Li, Y., Sabri, N. F. A. & Bauchy, M. Parameterization of empirical forcefields for glassy silica using machine learning. MRS Commun. 9, 593–599 (2019).
Bapst, V. et al. Unveiling the predictive power of static structure in glassy systems. Nat. Phys. 16, 448–454 (2020).
Eckart, L., Eckart, S. & Enke, M. A brief comparative study of the potentialities and limitations of machine-learning algorithms and statistical techniques. E3S Web Conf. 266, 02001 (2021).
Liu, H. et al. Predicting the dissolution kinetics of silicate glasses by topology-informed machine learning. npj Mater. Degrad. 3, 1–12 (2019).
Ravinder et al. Artificial intelligence and machine learning in glass science and technology: 21 challenges for the 21st century. Int. J. Appl. Glass Sci. 12, 277–292 (2021).
Smedskjaer, M. M., Hermansen, C. & Youngman, R. E. Topological engineering of glasses using temperature-dependent constraints. MRS Bull. 42, 29–33 (2017).
Yu, Y. et al. Hardness of silicate glasses: atomic-scale origin of the mixed modifier effect. J. Non-Cryst. Solids 489, 16–21 (2018).
Wilkinson, C. J., Zheng, Q., Huang, L. & Mauro, J. C. Topological constraint model for the elasticity of glass-forming systems. J. Non-Cryst. Solids .: X 2, 100019 (2019).
Wilkinson, C. J. et al. Topological origins of the mixed alkali effect in glass. J. Phys. Chem. B 123, 7482–7489 (2019).
Zheng, Q. & Zeng, H. Progress in modeling of glass properties using topological constraint theory. Int. J. Appl. Glass Sci. 11, 432–441 (2020).
Carleo, G. et al. Machine learning and the physical sciences. Rev. Mod. Phys. 91, 045002 (2019).
Tong, Q. et al. Combining machine learning potential and structure prediction for accelerated materials design and discovery. J. Phys. Chem. Lett. 11, 8710–8720 (2020).
Youngman, R. E. & Zwanziger, J. W. Multiple boron sites in borate glass detected with dynamic angle spinning nuclear magnetic resonance. J. Non-Cryst. Solids 168, 293–297 (1994).
Youngman, R. NMR spectroscopy in glass science: a review of the elements. Materials 11, 476 (2018).
Edén, M. NMR studies of oxide-based glasses. Annu. Rep. Prog. Chem. C: Phys. Chem. 108, 177–221 (2012).
Bødker, M. S., Sørensen, S. S., Mauro, J. C. & Smedskjaer, M. M. Predicting composition-structure relations in alkali borosilicate glasses using statistical mechanics. Front. Mater. 6, 175 (2019).
Bødker, M. S., Youngman, R. E., Mauro, J. C. & Smedskjaer, M. M. Mixed alkali effect in silicate glass structure: viewpoint of 29 Si nuclear magnetic resonance and statistical mechanics. J. Phys. Chem. B 124, 10292–10299 (2020).
Bødker, M. L., Pedersen, J. B., Muñoz, F., Mauro, J. C. & Smedskjaer, M. M. Statistical mechanical model for the formation of octahedral silicon in phosphosilicate glasses. J. Am. Ceram. Soc. 105, 1031–1038 (2022).
Cheng, B. & Titterington, D. M. Neural networks: a review from a statistical perspective. Stat. Sci. 9, 2–30 (1994).
Bødker, M. S., Mauro, J. C., Youngman, R. E. & Smedskjaer, M. M. Statistical mechanical modeling of borate glass structure and topology: prediction of superstructural units and glass transition temperature. J. Phys. Chem. B 123, 1206–1213 (2019).
Hinton, G. E., Osindero, S. & Teh, Y.-W. A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006).
Maekawa, H., Maekawa, T., Kawamura, K. & Yokokawa, T. The structural groups of alkali silicate glasses determined from 29Si MAS-NMR. J. Non-Cryst. Solids 127, 53–64 (1991).
Hermansen, C. et al. Structure-topology-property correlations of sodium phosphosilicate glasses. J. Chem. Phys. 143, 064510 (2015).
Bauchy, M., Qomi, M. J. A., Bichara, C., Ulm, F.-J. & Pellenq, R. J.-M. Rigidity transition in materials: hardness is driven by weak atomic constraints. Phys. Rev. Lett. 114, 125502 (2015).
Smedskjaer, M. M. et al. Topological principles of borosilicate glass chemistry. J. Phys. Chem. B 115, 12930–12946 (2011).
Patra, A. et al. A multi-fidelity information-fusion approach to machine learn and predict polymer bandgap. Comput. Mater. Sci. 172, 109286 (2020).
Zwanziger, J. W. Structure and chemical modification in oxide glasses. Int. Rev. Phys. Chem. 17, 65–90 (1998).
Dupree, R., Holland, D. & Williams, D. S. The structure of binary alkali silicate glasses. J. Non-Cryst. Solids 81, 185–200 (1986).
Mauro, J. C. & Smedskjaer, M. M. Statistical mechanics of glass. J. Non-Cryst. Solids 396–397, 41–53 (2014).
Mauro, J. C. Statistics of modifier distributions in mixed network glasses. J. Chem. Phys. 138, 12A522 (2013).
Goyal, S. & Mauro, J. C. Statistical mechanical model of bonding in mixed modifier glasses. J. Am. Ceram. Soc. 101, 1906–1915 (2018).
Bødker, M. S., Mauro, J. C., Goyal, S., Youngman, R. E. & Smedskjaer, M. M. Predicting Q-speciation in binary phosphate glasses using statistical mechanics. J. Phys. Chem. B 122, 7609–7615 (2018).
Fog, A. Sampling methods for Wallenius’ and Fisher’s noncentral hypergeometric distributions. Commun. Stat.—Simul. Comput. 37, 241–257 (2008).
Rowlinson, J. S. The Maxwell–Boltzmann distribution. Mol. Phys. 103, 2821–2828 (2005).
Uhlmann, D. R. & Shaw, R. R. The thermal expansion of alkali borate glasses and the boric oxide anomaly. J. Non-Cryst. Solids 1, 347–359 (1969).
Sørensen, S. S., Johra, H., Mauro, J. C., Bauchy, M. & Smedskjaer, M. M. Boron anomaly in the thermal conductivity of lithium borate glasses. Phys. Rev. Mater. 3, 075601 (2019).
Yiannopoulos, Y. D., Chryssikos, G. D. & Kamitsos, E. I. Structure and properties of alkaline earth borate glasses. Phys. Chem. Glasses 42, 164–172 (2001).
Bødker, M. S., Wilkinson, C. J., Mauro, J. C. & Smedskjaer, M. M. StatMechGlass: Python based software for composition–structure prediction in oxide glasses using statistical mechanics. SoftwareX 17, 100913 (2022).
Sun, Y. T., Bai, H. Y., Li, M. Z. & Wang, W. H. Machine learning approach for prediction and understanding of glass-forming ability. J. Phys. Chem. Lett. 8, 3434–3439 (2017).
Chen, L., Tran, H., Batra, R., Kim, C. & Ramprasad, R. Machine learning models for the lattice thermal conductivity prediction of inorganic materials. Comput. Mater. Sci. 170, 109155 (2019).
Mauro, J. Topological constraint theory of glass. Bull. Am. Ceram. Soc. 90, 31–37 (2011).
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).
Bishnoi, S. et al. Predicting Young’s modulus of oxide glasses with sparse datasets using machine learning. J. Non-Cryst. Solids 524, 119643 (2019).
Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020).
Acknowledgements
This work was supported by the Independent Research Fund Denmark (grant no. 7017-00019) and the Elite Research Travel Grant awarded to M.L.B. by the Danish Ministry of Higher Education and Science (grant no. 9095-00019A). M.B. acknowledges funding from the National Science Foundation under the grants DMR- 1944510, DMR-1928538, and CMMI-1826420.
Author information
Authors and Affiliations
Contributions
M.M.S. and M.L.B. conceived and planned the study. M.L.B. and T.D. performed the statistical mechanics and machine learning calculations with inputs from M.B. and M.M.S. M.L.B. and M.M.S. wrote the manuscript with revisions from M.B., J.C.M., and T.D. All authors participated in discussing the data.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bødker, M.L., Bauchy, M., Du, T. et al. Predicting glass structure by physics-informed machine learning. npj Comput Mater 8, 192 (2022). https://doi.org/10.1038/s41524-022-00882-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41524-022-00882-9
- Springer Nature Limited