Machine learning (ML), especially using artificial neural networks (ANNs), enables machines to learn from data just as we learn from experience. Their intelligence increases and they can perform tasks quicker and better without us having to explain everything. It’s like having a clever assistant that keeps improving through practice. 

ANNs work in a similar way to the biological neural networks in our brains. What makes ANNs special is that they are models that can, theoretically, learn any connections between input and output data. An example would be deciding whether a picture shows an elephant or a mouse. The model learns to recognize specific characteristics (“It’s gray”, “It’s got four legs”, “It’s got a trunk”, for example) from the input data and can use that knowledge to make a decision.

Seeing Inside the Box

Often, we don’t know which features ANNs have learned and what the ultimate decision is based on. It tends to feel like they’re a black box we can’t see into. So numerous researchers are working on explaining the processes inside ANNs to “open up” the black box and allow us to look inside – as in our exhibit.

Like humans, ML models learn from experience. For ML, “experience” is the information and data supplied. Data quality and quantity are among the most important factors in ensuring high-quality ML models. In fact, ML models are said to only be as good as the data they are given. 

Machine Learning, ANNs and Simulations

ML models like ANNs are used extensively in simulation science. Simulations are a key element in many areas of research, allowing us to visualize specific real-world phenomena, such as various physical processes. We can change the characteristics of simulations as required to explore how the real world would behave if the same characteristics occurred there.  

One way in which simulations are used is to ascertain the potential effects of climate change on weather patterns. Simulations often require considerable computing power because they calculate complex physical processes and generate large amounts of data.

ML models help us analyze the sort of large data volumes that simulations generate. But we can also use the data from simulations to teach ML models to behave like simulations, i.e. to become simulations of simulations. Known as “surrogate models”, these models replace some of the compute-intensive steps in the original simulation and can hugely accelerate them. That in turn enables significantly more simulations to be carried out and even more data to be generated. That data is then analyzed and used to develop more surrogate models–a crucial cycle of simulation and data, helping to drive research forward.

In our research, we use machine learning intensively for simulations to investigate a wide variety of physical processes. In doing so, the models are often not only optimized for the data, they also have to obey physical laws, which can be directly integrated into the models. One class of such models is called "Physics-Informed Neural Networks".

In the following, we present three applications in which we have used machine learning for simulations.

Material development with machine learning: stability of novel quantum bits.

To run quantum computers, qubits are needed that can permanently store information. This can be done by molecular magnets: single molecules that act like bar magnets. The quantum information is coded by the direction of the magnetic field. A group of SimTech physicists, chemists and mathematicians has succeeded in simulating how stable this quantum information can be stored.

Machine learning methods had to be adapted in such a way that they fulfill physical principles. As this kind of learning is high in effort and low in accuracy, the group developed neural networks that fulfill these known physical properties a priori. The researchers use them to simulate how the magnetization of molecular magnets changes when the molecule moves at high temperature and whether information can be lost in the process.

Real-Time Storm and Rain Forecasting

Forecasting of weather events like rain and storms is based on complex mathematical models that describe the physical properties of the environment and air currents. A typical example is forecasting of hurricanes based on current wind, temperature, pressure and radar measurements. These models require lengthy calculations on expensive and energy-hungry supercomputers.

SimTech researchers are working on machine learning methods in order to develop more efficient forecasting models. Neural networks that have been trained with historical data can generate faster forecasts in real time. Known as “surrogate models”, these models can replace complex calculation methods. To make them even more accurate and stable, SimTech researchers are working on integrating physical laws into such surrogate models.

Can Machine Learning Do Physics Too? Predicting Wave Phenomena Using Physics-Informed Neural Networks

Flows play a key role in a number of fields, from how pollutants spread in groundwater to predicting tsunamis. Conventional numerical simulations are precise but slow and require a great deal of computing power. In crisis situations where time is of the essence, the tsunami has long since arrived before the prediction has been calculated. This is where machine learning techniques come in handy. Although the training takes a lot of time, they can then deliver extremely swift predictions. By “teaching” the ML model physics, results that incorporate physical laws can be produced, even in complex scenarios. 

In a simple example, we used an ML model to simulate a wave spreading across a sandbank. But we can also deliver a result within a few milliseconds for more complicated scenarios where conventional simulations would take many seconds or minutes. To achieve this, we supplemented an artificial neural network with knowledge about physically correct behavior. This knowledge delivers forecasts in line with physics laws, even far into the future.

In this way, we have shown how neural networks can provide fast and accurate predictions without the need for costly, extensive training data from simulations. This significantly speeds up the prediction of wave phenomena and is especially important in time-critical situations.