Computational Statistics in Data Science. Группа авторов
Чтение книги онлайн.
Читать онлайн книгу Computational Statistics in Data Science - Группа авторов страница 43
![Computational Statistics in Data Science - Группа авторов Computational Statistics in Data Science - Группа авторов](/cover_pre1083664.jpg)
Figure 6 Architecture of variational autoencoder (VAE).
Therefore, the loss function of training a VAE can be simplified as
where the first term captures the reconstruction loss, and the second term is regularization on the embedding. To optimize the loss function (6), a reparameterization trick is used. For a chosen approximate posterior
(7)
where
(8)
where
6 Recurrent Neural Networks
6.1 Introduction
The previously introduced models have the same assumptions on the data, that is, the independence among the samples and the fixed input size. However, these assumptions may not be true in some cases, thus limiting the application of these models. For example, videos can have different lengths, and frames of the same video are not independent, and sentences of an chapter can have different lengths and are not independent.
A RNN is another modified DNN that is used primarily to handle sequential and time series data. In a RNN, the hidden layer of each input is a function of not just the input layer but also the previous hidden layers of the inputs before it. Therefore, it addresses the issues of dependence among samples and does not have any restriction on the input size. RNNs are used primarily in natural language processing applications, such as document modeling and speech recognition.
6.2 Architecture
As illustrated in Figure 7, a general neural network
RNN can work with sequence data, which has input as sequence or target as sequence or both. An input sequence data can be denoted as