Computational Statistics in Data Science. Группа авторов

Чтение книги онлайн.

Читать онлайн книгу Computational Statistics in Data Science - Группа авторов страница 42

Computational Statistics in Data Science - Группа авторов

Скачать книгу

alt="bold-italic upper D colon double-struck upper R Superscript p Baseline right-arrow double-struck upper R Superscript n"/> (decoder), that minimize the following loss function w.r.t. the input bold-italic x:

StartLayout 1st Row 1st Column double-vertical-bar bold-italic x minus bold-italic upper D left-parenthesis bold-italic upper E left-parenthesis bold-italic x right-parenthesis right-parenthesis double-vertical-bar squared 2nd Column Blank EndLayout

      The encoder (bold-italic upper E) and decoder (bold-italic upper D) can be any mappings with the required input and output dimensions, but for image analysis, they are usually CNNs. The norm of the distance can be different, and regularization can be incorporated. Therefore, a more general form of the loss function is

      (3)StartLayout 1st Row 1st Column upper L left-parenthesis bold-italic x comma ModifyingAbove bold-italic x With Ì‚ right-parenthesis plus regularizer 2nd Column Blank EndLayout

stat08316fgz005

      Source: Krizhevsky [14]

      The output of the encoder part is known as the embedding, which is the compressed representation of input learned by an autoencoder. Autoencoders are useful for dimension reduction, since the dimension of an embedding vector can be set to be much smaller than the dimension of input. The embedding space is called the latent space, the space where the autoencoder manipulates the distances of data. An advantage of the autoencoder is that it can perform unsupervised learning tasks that do not require any label from the input. Therefore, autoencoder is sometimes used in pretraining stage to get a good initial point for downstream tasks.

      5.3 Variational Autoencoder

      Many different variants of the autoencoder have been developed in the past years, but the variational autoencoder (VAE) is the one that achieved a major improvement in this field. VAE is one of the frameworks, which attempts to describe an observation in latent space in a probabilistic manner. Instead of using a single value to describe each dimension of the latent space, the encoder part of VAE uses a probability distribution to describe each latent dimension [17].

      Given an observed dataset left-brace bold-italic x Subscript i Baseline right-brace Subscript i equals 1 Superscript n, the marginal log‐likelihood is composed of a sum over the marginal log‐likelihoods of all individual data points: log p Subscript theta Baseline left-parenthesis bold-italic x 1 comma bold-italic x 2 comma period period period comma bold-italic x Subscript n Baseline right-parenthesis equals sigma-summation Underscript i equals 1 Overscript n Endscripts log p Subscript theta Baseline left-parenthesis bold-italic x Subscript i Baseline right-parenthesis, where each marginal log‐likelihood can be written as

      (4)StartLayout 1st Row 1st Column log p Subscript theta Baseline left-parenthesis bold-italic x Subscript i Baseline right-parenthesis equals KL left-parenthesis q Subscript phi Baseline left-parenthesis bold-italic z vertical-bar bold-italic x Subscript i Baseline right-parenthesis StartAbsoluteValue EndAbsoluteValue p Subscript theta Baseline left-parenthesis bold-italic z vertical-bar bold-italic x Subscript i Baseline right-parenthesis right-parenthesis plus script l left-parenthesis theta comma phi semicolon bold-italic x Subscript i Baseline right-parenthesis 2nd Column Blank EndLayout

      where the first term is the KL divergence [18] between the approximate and the true posterior, and the second term is called the variational lower bound. Since KL divergence is nonnegative, the variational lower bound is defined as

      (5)StartLayout 1st Row 1st Column log p Subscript theta Baseline left-parenthesis bold-italic x Subscript i Baseline right-parenthesis greater-than-or-equal-to script l left-parenthesis theta comma phi semicolon bold-italic x Subscript i Baseline right-parenthesis 2nd Column equals double-struck upper E Subscript q Sub Subscript phi Subscript left-parenthesis bold-italic z vertical-bar bold-italic x Sub Subscript i Subscript right-parenthesis Baseline left-bracket minus log q Subscript phi Baseline left-parenthesis bold-italic z vertical-bar bold-italic x right-parenthesis plus log p Subscript theta Baseline left-parenthesis bold-italic x comma bold-italic z right-parenthesis right-bracket 2nd Row 1st Column Blank 2nd Column equals double-struck upper E Subscript q Sub Subscript phi Subscript left-parenthesis bold-italic z vertical-bar bold-italic x Sub Subscript i Subscript right-parenthesis 
						<noindex><p style= Скачать книгу