In an article recently put out by Physical Review Research, it is demonstrated how deep learning can assist in identifying solutions to basic equations of quantum mechanics for real-world systems. This is not just a critical and basic scientific question, but it could also have the outcome of practical use cases in the future, facilitating researchers and scientists to go about prototyping new materials and chemical syntheses in silico prior to attempting to produce them in the laboratory. DeepMind has produced a new neural network architecture, the Fermionic Neural Network or FermiNet, which is apt to modelling the quantum state of big collections of electrons, the basic building blocks of chemical bonds. The FermiNet was the preliminary demonstration of deep learning for computation of the energy of molecules and atoms from first principles that was precise enough to prove useful, and it stays the most precise neural network method to this date. The hope is that the utilities, tools, concepts, and ideas generated over the course of Artificial Intelligence research at DeepMind can assist in finding solutions to basic issues within the natural sciences, and the FermiNet is a step in this direction joining their work on protein folding, glassy dynamics, lattice quantum chromodynamics and several other projects which endeavour to bring their vision alive.

A short history of quantum mechanics

Bring up the term “quantum mechanics” and you are more probable to stir up confusion rather than anything else. The term conjures up imagery of Schrodinger’s cat, which is in a paradoxical state of life and death, and basic particles that are also, in some fashion, waves. Within quantum systems, a particle like an electron doesn’t have a precise location, as it would in a conventional description. Rather, its position is detailed by a probability cloud, it’s smeared out in all locations its permitted to be. This counterintuitive state of affairs caused Richard Feynman to declare: “If you think you comprehend quantum mechanics, you don’t comprehend quantum mechanics.” Regardless of this mystical weirdness, the crux of the theory can be whittled down to only a few straightforward equations. The most popular of these, the Schrodinger equation, details the behaviour of particles at the quantum scale in the same fashion that Newton’s laws detail the behaviour of objects at the more familiar human scale. While the interpretation of this equation can be the reason behind limitless confusion, the mathematics is much simpler to work with, bringing up the typical exhortation from professors to “shut up and calculate” when bombarded with thorny philosophical enquiries by students.

These equations are adequate to detail the behaviour of all the familiar matter we observe around us at the level of atoms and nuclei. Their counterintuitive nature causes all types of unique phenomena: superconductors, superfluids, lasers and semiconductors are only feasible due to quantum effects. However, even the humble covalent bond – the fundamental building block of chemistry – is a result of the quantum interactions of electrons. After these rules were decided upon in the 1920s, researchers came to the realization that, for the first time, they possessed a comprehensive theory with regards to how chemistry operates. In principle, they could merely setup these equations for differing molecules, identify solutions for the energy of the system, and find out which molecules were stable and which reactions would occur spontaneously. However, when they had a sitting down to calculate the solutions to these equations, they discovered that they do it precisely for the simplest atom (hydrogen) and literally nothing else. Everything else was too complex.

The heady optimism of this era was summarized by Paul Dirac:

“The underlying physical laws required for the mathematical theory of a large part of physics and the entirety of chemistry are therefore totally known, and the difficulty is only that the precise application of these laws has the outcome of equations much too complex to be soluble. It thus becomes desirable that approximate practical methods regarding application of quantum mechanics should be produced.”

Several took up Dirac’s charge, and very soon scientists constructed mathematical strategies that could approximate the qualitative behaviour of molecular bonds and other chemical phenomena. These strategies began from an approximate description of how electrons act that might be familiar from introductory chemistry. In this description, every electron is allocated to a specific orbital, which provides the probability of a singular electron being discovered at any point close to an atomic nucleus. The shape of every orbital is then dependent on the average shape of all other orbitals. As this “mean field” description regards every electron as being allocated to only a singular orbital, it is a very incomplete picture of how electrons behave in reality. Nonetheless, it is adequate to estimate the cumulative energy of a molecule with just about 0.5% error.

Atomic Orbitals. The surface signifies the region of increased odds of discovering an electron. In the blue area the wavefunction is positive, whereas in the purple region, it happens to be negative. 

Unluckily, 0.5% error still isn’t adequate to be good to the working chemist. The energy in molecular bonding is only a small fraction of the cumulative energy of a system, or approximately 0.2% of the remainder “correlation” energy. For example, while the cumulative energy of the electrons in a butadiene molecule is approximately 100,000 kilocalories per mole, the variation in energy amongst differing potential shapes of the molecule is only 1 kilocalorie per mole. That implies that if you wish to accurately forecast butadiene’s natural shape, then the same degree of accuracy is required as measuring the width of a football pitch down to the millimeter. 

With the proliferation of digital computing following the end of World War II, researchers produced an entire menagerie of computing methods that surpassed this mean field description of electrons. While these strategies come in a staggering alphabet soup of abbreviations, they all typically fall someplace on an axis that trades off precision with efficiency. At one end of the spectrum, there are techniques that are basically exact, but scale worse than exponentially with the number of electrons, making them impractical for all but the tiniest of molecules. At the other end of the spectrum are strategies that scale linearly, but are not very precise. These computing strategies have had a massive impact on the practice of chemistry – the 1998 Nobel Prize in Chemistry was awarded to the originators of several of these algorithms. 

Fermionic Neural Networks 

Regardless of the breadth of current computational quantum mechanics utilities, the prevailing sentiment was that a new strategy was required to tackle the issue of efficient representation. There’s a reason that the biggest quantum chemical calculations only go into the tens of thousands of electrons for even the mot approximate strategies, while classical chemical calculation strategies such as molecular dynamics can manage millions of atoms. The state of a classical system can be detailed easily – we only have to track the position and momentum of every particle. Representing the state of the quantum system is a lot more challenging. A probability has to be allocated to each potential configuration of electron positions. This is encoded within the wavefunction, which allocates a positive or negative numeral to each configuration of electrons, and the wavefunction squared provides the probability of identifying the system in that configuration. The space of all potential configurations is massive, if we attempted to represent it as a grid with 100 points along every dimension, then the number of potential electron configurations for the silicon atom would be bigger than the number of atoms in the entire universe! Whew! 

This is precisely where the train of thought tilted towards deep neural networks being helpful. Over the course of the last several years, there have ben massive progressions in representing complicated, high-dimensional probability distributions with neural networks. We now are aware how to go about training these networks efficiently and scalably. Provided these networks have already proven their worth at fitting high-dimensional functions in artificial intelligence issues, perhaps they could be leveraged to indicate quantum wavefunctions as well. This is not the first time this has been thought of, scientists like Giuseppe Carleo and Matthias Troyer and others have demonstrated how advanced deep learning could be leveraged for identifying solutions to idealised quantum problems. The wish was to leverage deep neural networks to handle more realistic issues within chemistry and condensed matter physics, and that implied integrating electrons within the calculations. 

There is only one wrinkle when managing electrons. Electrons must adhere to the Pauli exclusion principle, which implies that they cannot be in the same space at the same time. This is due to the fact that electrons are a variant of particle referred to as fermions, which consists of the building blocks of most matter out there – neutrons, protons, quarks, neutrinos, etc. Their wavefunction must be antisymmetric – if the you switch the position of two electrons, the wavefunction gets multiplied by -1. That implies that if two electrons are atop one another, the wavefunction (and the probability of the configuration) will be nil. 

This implied that a new variant of neural network had to be developed that was antisymmetric with regards to its inputs, which has been christened the Fermionic Neural Network, or FermiNet. In a majority of quantum chemistry methodologies, antisymmetry is put forth leveraging a function referred to as the determinant. The determinant of a matrix has the attribute that if you switch two rows, the output is multiplied by -1, much like a wavefunction for fermions. Therefore you can take up a grouping of single-electron functions, assess them for each electron in the system, and pack all of the outcomes into a singular matrix. The determinant of that matrix is subsequently a properly antisymmetric wavefunction. The dominant restriction of this strategy is that the outcome function – referred to as a Slater determinant – is not really general. Wavefunctions of actual systems are typically a lot more complex. The usual way to build on this is to take a big linear combo of Slater determinants – at times millions or even more – and include some simplistic corrections on the basis of pairings of electrons. Even then, this may not be adequate to precisely compute energies. 

Deep Neural Networks can usually be far more efficient at representing complicated functions that linear combos of basis functions. In the FermiNet, this is accomplished by making every function going into the determinant a function of all electrons (1). This surpasses strategies that just leverage one – and two electron functions. The FermiNet has an individual stream of data for every electron. With no interactions amongst these streams, the network would be no more expressive than a typical Slater determinant. To surpass this, we average together data from across all streams at every layer of the network, and pass this data to every stream at the subsequent layer. In this way, these streams have the correct symmetry attributes to develop an asymmetric function. This is like how graph neural networks aggregate data at every layer. Unlike the Slater determinants, FermiNets are universal function approximations, at least in the limit where the neural network layers become broad enough. This implies that, if we can go about training these networks in the right way, they should be capable to fit the nearly-exact solution to the Schrodinger issue. 

The FermiNet is fitted by reducing the energy of the system. To perform that precisely, we would require to assess the wavefunction at all potential configurations of electrons, so we have to perform it approximately instead. We choose an arbitrary selection of electron configurations, assess the energy locally at every arrangement of electrons, add up the contributions from every arrangement and reduce this rather than the true energy. This is referred to as the Monte Carlo method, as it’s a little like a gambler throwing dice repeatedly. While it is approximate, if we require to make it more precise we can always do the dice roll again. As the wavefunction squared provides the probability of observing an arrangement of particles in any location, it is most convenient to produce samples from the wavefunction itself, basically, simulation of the act of observing the particles. Whereas a majority of neural networks receive training from some external information, in this scenario the inputs leveraged to train the neural network are produced by the neural network itself. It’s a little like pulling yourself up by your own bootstraps, and it implies that we don’t require any training information other than the positions of the atomic nuclei that the electrons are dancing around. The fundamental idea, referred to as variational quantum Monte Carlo (or VMC for short), has been around ever since the 1960s, and it is typically viewed as a cheap but not very precise method of computing the energy of a system. By substituting the simple wavefunctions on the basis of Slater determinants with the FermiNet, what has occurred is a dramatic increase in the precision of this strategy on each system that has been looked at. 

To ensure that the FermiNet really does signify progress in the bleeding edge, researchers began by looking into simple, well-researched systems, such as atoms in the first row of the periodic table (hydrogen through neon). These are small systems – 10 electrons or lesser – and simplistic enough that they can be treated by the most precise (but exponential scaling) strategies. The FermiNet outpaces comparable VMC calculations by a huge margin – usually cutting the error comparative to the exponentially-scaling calculations by 50% or more. On larger systems, the exponentially scaling strategies become intractable, so the researchers instead leverage the “coupled cluster” method as a baseline. This strategy functions well on molecules in their stable config, but suffers when bonds get stretched or broken, which is crucial for comprehending chemical reactions. While scales a lot better than exponentially, the specific coupled cluster strategy that has been leveraged still scales as the number of electrons raised to the seventh power, so it can just be leveraged for mid-sized molecules. The FermiNet was applied to progressively bigger molecules, beginning with lithium hydride and working their way up to bicyclobutane, the biggest system they looked at, possessing 30 electrons. On the smallest molecules, the FermiNet captured an amazing 99.8% of the difference between the coupled cluster energy and the energy you obtain from a singular Slater determinant. On bicyclobutane, the FerrmiNet still managed to capture 97% or more of this correlation energy – a massive achievement for an apparently “cheap but imprecise” approach. 

While coupled cluster strategies function well for molecules that are stable, the actual frontier in computational chemistry is in comprehending how molecules stretch, twist, and break. There, coupled cluster methods typically struggle, so they have to contrast against as many baselines as doable to ensure they obtain a consistent answer. They looked at dual benchmark stretched systems, the nitrogen molecule (N2) and the hydrogen chain with 10 atoms, (H10). Nitrogen is a particularly challenging molecular bond, as every nitrogen atom contributes 3 electrons. The hydrogen chain, meanwhile, is of interest for comprehending how electrons act in materials, for example, forecasting whether or not a material will conduct electricity. On both systems, coupled cluster performed well at equilibrium, however faced problems as the bonds were stretched. Traditional VMC calculations did badly across the board. However, the FermiNet was amongst the best methods investigated, regardless of the bond length. 

Conclusion 

The FermiNet is the beginning of great things to follow for the fusion of deep learning and computational quantum chemistry. A majority of the systems that have been looked at so far are well-researched and well-comprehended. But just as the first good outcomes with deep learning in other domains led to an explosion of follow-up work and rapid progression, the hope is that FermiNet will inspire lots of research on scaling up and several ideas for new, even improved network architectures. Since the research was first put out on arXiv previous year, other groups and units have shared their approaches to application of deep learning to first-principles calculations on the many-electron problem. The surface of computational quantum physics has just been scratched, and researchers look ahead to applying the FermiNet to difficult issues within material sciences and condensed matter physics as well. Mostly, the hope is that by putting out the source code leveraged in the experiments, the researchers can serve as an inspiration to others to add on top of their work and attempt new applications that previously hadn’t been thought of.