Sophisticated artificial intelligence frameworks tackle activities such as identifying objects in imagery and forecasting the 3D structure of proteins as a meticulous student would study for an examination. Through training on various instances of problems, they reduce their errors over the course of the time until they accomplish success. However, this is a solo endeavour and merely one of the available formats of learning. Learning also occurs through interaction and play with other people. It’s uncommon that a singular individual can find solutions to extremely complicated issues by themselves. By facilitating problem solving to adopt these traits inspired by gaming, previous DeepMind initiatives have undertaken training of AI agents to play Capture the Flag games and attain Grandmaster level at StarCraft. This has compelled us to think if such a viewpoint modelled on game theory could assist in finding solutions to other basic machine learning issues. 

At the ICLR 2021, (the International Conference on Learning Representations) “EigenGame: PCA as a Nash Equilibrium” was presented, and it went on to receive an outstanding paper prize. The research delved into a new perspective to an old issue: principal component analysis (PCA) was reformulated, a variant of eigenvalue problem, as a competitive multi-agent game referred to as EigenGame. PCA is usually formulated in the form of an optimisation issue (or single-agent issue), but it was discovered that the multi-agent viewpoint enabled the development of unique insights and algorithms which leverage the latest computing assets. This facilitated the scaling of humongous data sets that prior would have been too taxing from a computational perspective, and provides an alternative approach for subsequent exploration. 

PCA as a Nash equilibrium 

Initially detailed in the early 1900s, PCA is a time-tested strategy for interpreting the structure of high-dimensional information. This strategy is now commonplace as a preliminary step in the data-processing pipeline and makes it simple to cluster  and go about visualizing information. It can also be an efficient tool for understanding low-dimensional representations with regards to classification and regression. Over a century on, there are still valid reasons to undertake the study of PCA. 

To start with, data was initially gathered manually in paper notebooks, and currently it is recorded in data centres as big as mini factories. As an outcome, this commonplace analysis has become a bottleneck, from a computing standpoint. Scientists have looked into randomized algorithms and other perspectives to enhance the scaling of PCA, but have discovered that these strategies find it tough to scale to humongous datasets as they cannot completely leverage the latest deep-learning centric progressions in computation – particularly access to several parallel GPUs or TPUs.  

Next, PCA shares a mutual solution with several critical Machine Learning and engineering problems, particularly the singular value decomposition (SVD). By tackling the PCA problem in the correct fashion, the insights and algorithms find application more widely throughout the branches of the machine learning tree. 

Just like any board game, to go about reinventing PCA as a game, we require a grouping of rules and objectives for players to adhere to. There are several potential methods to design a game like this, but, critical concepts are from the PCA itself; the best solution is made up of eigenvectors which record the critical variance in the data and are orthogonal to one another. 

In EigenGame, every user controls an eigenvector. Players improve their scores by detailing variance within the information but are levied penalties if they’re in too close alignment with the other players. We also setup a hierarchy: Player 1 is only bothered about the maximization of variance, while other players also have to be concerned with regards to minimization of their alignment with players who supersede them in the hierarchy. This combo of rewards and punishments goes about defining every player’s utility. 

With appropriately developed Var and Align terms, we can demonstrate that: 

• If all users play in an optimal fashion, in conjunction, they accomplish the Nash equilibrium of the game, which in turn, is the PCA solution. 
• This can be accomplished if every player goes about maximizing their utility independently and simultaneously leveraging gradient ascent. 

This independence attribute of simultaneous ascent is especially critical as it facilitates for the computation to be distributed throughout several Google Cloud TPUs, facilitating both data – and model-parallelism. This renders it doable for the algorithm to go about adapting to really large-scale information. EigenGame identifies the principal components in a question of hours for 100TB datasets consisting of millions of features or billions of rows. 

Updates, utilities, and everything else 

By contemplating with regards to PCA from a multi-agent viewpoint, they were able to go about proposing scalable algorithms and novel analyses. What was also unveiled was a shocking connection to Hebbian Learning – or, how neurons go about adapting when learning. In EigenGame, every player maximizes their utilities, giving rise to updated equations that are like updated rules obtained from Hebbian models of synaptic plasticity in the brain. Hebbian updates are renowned to converge to the PCA solution but are not obtained as the gradient of any utility functionality. Game theory provides us a fresh perspective to look at Hebbian Learning, and also indicates a continuum of strategies to machine learning issues. 

On one point of the machine learning continuum is the well-developed pathway of proposing an objective function that can undergo optimization. Leveraging the concept of convex and non-convex optimisation, scientists can reason with regards to the global attributes of the solution. On the other side of the continuum, pure connectionist strategies and update rules which draw inspiration from neuroscience are mentioned directly, however, analysis of the complete system can be more tough, typically invoking the research of complex dynamical systems. 

Game Theoretic strategies such as EigenGame rest somewhere in the middle. Player updates are not restricted to be the gradient of a function, only the ideal response to the present techniques of the other players. End-users have the freedom to develop utilities and updates with desired attributes, for instance, detailing updates which are unbiased or accelerated – while making sure that the Nash attribute still enables us to undertake analysis of the cumulative system. 

EigenGame is representative of a concrete instance of developing the solution to a machine learning problem as the output of a massive multi-agent framework. In a general sense, developing machine learning problems in the form of multi-agent games is a challenging mechanism design issue, but researchers have already leveraged the class of two-player, zero-sum games to find solutions to machine learning issues. Most noteworthy, the proliferation of generative adversarial networks as a technique to generative modelling has spearheaded interest in the relationship amongst machine learning and game theory.  

EigenGame goes beyond this to the more complicated several-player, general sum setting. This facilitates more overt parallelism for improved scale and speed. It also puts forth a quantitative benchmark for the community to evaluate novel multi-agent algorithms in conjunction with richer domains, like Soccer and Diplomacy. 

The hope is that this blueprint for developing utilities and updates will compel people to look into this direction for developing new algorithms, agents, and frameworks. It’s going to be interesting to discover what other issues can be made as games and if the insights we obtain will further enhance our comprehension of the multi-agent nature of intelligence.