Autoencoders

Autoencoders are a type of neural network used to learn efficient representations of data without labels. The network is trained to produce a recreation of the input. But the neural architecture is designed such that the input data must be mapped to a lower dimensional representation before producing the output recreation.

How Autoencoders Work

1. Encoder: The first part of the network compresses the input into a latent-space representation. It encodes the input as an internal fixed-size representation in reduced dimensionality.

2. Latent Space: This is the reduced-dimensionality space where the compressed features reside. It's a bottleneck through which the network learns the most important features of the input data.

3. Decoder: The second part of the network reconstructs the input data from the internal representation. It maps the encoded data back to the original data space.

The network is trained to minimize the difference between the input and the reconstructed output. This type of network can be useful in unsupervised learning scenarios where you're trying to discover an informative representation of the input data without using any labels.

Convolutional Autoencoders

These are autoencoders that use convolutional layers in the encoder and decoder parts. They are particularly useful for reconstructing images.

Variational Autoencoders

Variational autoencoders (VAEs) are a type of autoencoder that produces a probabilistic latent space. Unlike a regular autoencoder, which only learns a point in the latent space, VAEs learn a distribution. This means that new interpolations are possible. We can artificially change the latent space and see resulting output changes in a smooth fashion.