From a human being’s perspective, the planet goes a lot further beyond the mere images reflected within our corneas. For instance, when we observe a construction and admire the complexity of its design, we can feel like appreciating the craftsmanship and finesse finishing a task like that needs. This capability to interpret objects via the utilities that developed them provides us a richer comprehension of the planet and is a critical facet of our intelligence. 

Our ultimate wish is for our systems to develop similarly rich representations of the world. For instance, when looking at an image or a painting we would wish them to comprehend the brush strokes leveraged to develop it and not merely the pixels that indicate it on a screen. 

In a latest research work, artificial agents were equipped with identical tools that are leveraged to produce imagery and illustrate that they can go about reasoning about how digits, characters, and portraits are developed. Critically, the go about to learning to do this autonomously and without the requirement for human labelled-datasets. This stands in comparison with latest research which has till now been reliant on learning from human demonstrations, which can be time-consuming procedure. 

A deep reinforcement learning agent was developed that has interactions with a computer paint program, effecting strokes on an electronic canvas, and modifying the brush size, pressure and colour. The untrained agent begins by placing arbitrary strokes with no obvious intention or structure. To surpass this, a method had to be developed to reward the agent that compels it to generate meaningful drawings. 

With this  end in mind, a second neural network received training, referred to as the discriminator, whose entire purpose is to forecast if a specific drawing was generated by the agent, or if it was sampled from a dataset of actual photographs. The painting agent receives rewards by how much it handles to “trick” the discriminator into thinking that its drawings are the real deal. To put it in different words, the agent’s reward signal is itself learned. While this is like the strategy leveraged in Generative Adversarial Networks (GANs), it is different as the generator in GAN setups is usually a neural network that directly puts out pixels. By comparison, the agent deployed generates imagery by authoring graphics programs to have interactions with a paint environment. 

In the first grouping of experiments, the agent received training to produce imagery resembling MNIST digits: it was illustrated what the digits appear like, but not how they are drawn. By making an attempt to produce imagery that tricks the discriminator, the agent goes about learning to control the brush and to manipulate it to be fitting to the style of differing digits, a strategy referred to as visual program synthesis. 

It was also trained to reproduce particular imagery. In this scenario, the discriminator’s intention is to decide if the reproduced image is a copy of the targeted image, or if it has been generated by the agent. The more tough this distinction is for the discriminator, the more the agent is rewarded. 

Critically, this framework is also decipherable as it generates a sequence of motions that manage a simulated brush. This implies that the model can apply what it has gone about learning on the simulated paint program to re-create personalities in other resemblant environments, for example on a simulated or actual robotic arm. There is also the possibility to scale this framework to actual datasets. Upon receiving training to paint celeb faces, the agent has the potential to capture the main characteristics of the face, like shape, tone, and hair style, just like a graffiti or street artist would when doing a portrait with a minimal number of brush strokes. 

Recovering structured representations from raw sensations is a capability that human beings readily have and consistently leverage. Here, it was illustrated that it is doable to guide artificial agents to generate identical representations by providing them access to the same utilities that we leverage to recreate the planet around us. By doing so they go about learning to generate visual programs that succinctly express the causal relationships that give rise to their observations. 

This is only a minor step towards the cause of flexible program synthesis, it is anticipated that similar strategies may be required to facilitate artificial agents with human-like cognitive, generalization and communicative capabilities.