New Perspectives of Generative Adversarial Networks (GANs) Application

The concept of Generative Adversarial Networks (GANs) was first coined by Goodfellow et al. in their now-famous 2014 paper. The researchers proposed an unusual training setup where two networks, a generator and discriminator, were pitched against each other in a competition.

Here’s how they accomplished this.

The generator had to generate fake images given random noise as inputs, while the discriminator had to discern between the fake and real images from the target domain they wanted the model to learn (e.g. facial images).

So, what were the results? Over time, both networks progressively improved in their tasks, and following this technique anyone can now obtain a trained generator model that replicates images from the target domain pretty well.

While GANs include both a generative and a discriminative model, generative and discriminative models can exist separately and be used for different tasks. However, only generative models are able to create novel data samples from the target distribution.

Understanding the Basics of GANs: Options & More

When it comes to powerful generative models for image synthesis, the most commonly mentioned are StyleGAN—and its updated version StyleGAN2.

These models are able to solve image generation tasks and produce remarkably high-fidelity images of non-existent people, animals, landscapes, and other objects given an appropriate training dataset.

StyleGAN, just like the other GAN architectures, features two sub-networks: discriminator and generator. During training, the generator is tasked with producing synthetic images while the discriminator is trained to differentiate between the fakes created by the generator and the real images.

The first iteration of StyleGAN originally appeared in 2019. It was applied to produce fake faces with high detailization and natural appearance with resolutions as high as 1024×1024—something not previously achieved by other similar models.

However, some AI-generated images had artifacts, so it was decided to improve the model and presented StyleGAN2. One of the main issues of the original StyleGAN were the blob-like artifacts that looked like drops of water.

StyleGAN vs. StyleGAN2: What Changed?

The development of StyleGAN2 was related not only to addressing the issue with AdaIN—there were other key improvements over its predecessor StyleGAN. These included:

1. Network redesign: Small changes were made in the order and type of operations within the style blocks of the network leading to small positive improvements in the network’s performance.

2. Lazy regularization: Researchers discovered that it wasn’t necessary to compute regularization terms in addition to the main loss function at every training iteration. Instead, it could be computed periodically, reducing the computational costs while having little to no effect on the quality of the results.

3. Path length regularization: It was observed that uniformity of latent space W (this space is sampled from to obtain styles for image generation) had a positive effect on image quality. Uniformity was achieved by ensuring that a fixed-size step was used when sampling from W space, which led to a fixed-size change in the generated image regardless of the direction of the step.

4. Progressive growth was replaced: The original architecture used progressive growing to train the model for higher resolutions (at first the model was trained to generate 8×8 images, followed by 16×16, and so on). However, this method introduced certain artefacts in the produced images. This led the authors to skip connections in the generator and discriminator, which allowed them to avoid this problem.

Applying GANs to Image Generation

There are many practical use cases of GAN technology, few of them are:

• New data samples generation, including images, text and audio;
• Image manipulation, including inpaining, upscaling, and blending;
• Text-to-image and image-to-image translation.

Let’s take a look at the research performed by MobiDev’s AI engineers. They applied the StyleGAN2 architecture to image generation tasks. They used all the recent developments and improvements introduced by researchers, things like adaptive discriminator augmentation, mixed-precision training, and self-attention layers to receive GAN generated logotypes. This process used nearly 49,000 images as a training dataset.

Text-based images were removed because generating textual logotypes requires at least three models. For instance, a language model like BERT or GPT-2 to produce synthetic logotype text, a GlyphGAN model that creates characters with unique fonts to visualize the synthetic text, and a third model for generating the logotype itself.

The training dataset was aggregated into 10 clusters. From there, this information helped the model in generating images from various logotype groups. As a result, the model was able to generate logotypes with quality ranging from good to medium and poor. The results were carefully analyzed to understand the reason for the model’s outputs and to find new ways to improve its performance in the future.

The research showed that AI has huge potential in the area of image creation. But still, the question remains: what else could be achieved with the help of GANs?

This paper published in Nature suggests that machine learning could significantly contribute to material science as GAN architecture was found to be able to generate chemically-valid hypothetical inorganic compounds.

Even though the system is still missing some key components (prediction of hypothetical materials’ crystal structure, addition of strict chemical rule filters), the published results show that machine learning may be responsible for inventing new materials such as ultra-light durable alloys, solid state electrolytes for Li-Ion batteries, and other technologies in the not so distant future.

AI-driven technologies are only getting started. And as innovative research teams continue to push the boundaries of what’s possible, it’s clear that there will be no shortage of transformative technologies coming down the pipeline.