Computational Statistics in Data Science. Группа авторов

Чтение книги онлайн.

Читать онлайн книгу Computational Statistics in Data Science - Группа авторов страница 43

Computational Statistics in Data Science - Группа авторов

Скачать книгу

left-bracket log p Subscript theta Baseline left-parenthesis bold-italic x Subscript i Baseline vertical-bar bold-italic z right-parenthesis right-bracket minus KL left-parenthesis q Subscript phi Baseline left-parenthesis bold-italic z vertical-bar bold-italic x Subscript i Baseline right-parenthesis StartAbsoluteValue EndAbsoluteValue p Subscript theta Baseline left-parenthesis bold-italic z right-parenthesis right-parenthesis EndLayout"/>

stat08316fgz006

      (7)StartLayout 1st Row 1st Column bold-italic z overTilde equals g Subscript phi Baseline left-parenthesis bold-italic epsilon comma bold-italic x right-parenthesis comma bold-italic epsilon tilde p left-parenthesis bold-italic epsilon right-parenthesis 2nd Column Blank EndLayout

      where bold-italic epsilon is an auxiliary variable with independent marginal p left-parenthesis bold-italic epsilon right-parenthesis, and g Subscript phi Baseline left-parenthesis dot right-parenthesis is some vector‐valued function parameterized by phi. With this reparameterization trick, the variational lower bound can be estimated by sampling a batch of bold-italic epsilon from p left-parenthesis bold-italic epsilon right-parenthesis:

      (8)StartLayout 1st Row 1st Column script l left-parenthesis theta comma phi semicolon bold-italic x Subscript i Baseline right-parenthesis almost-equals sigma-summation Underscript j equals 1 Overscript upper B Endscripts left-bracket minus log q Subscript phi Baseline left-parenthesis bold-italic z Subscript left-parenthesis i comma j right-parenthesis Baseline vertical-bar bold-italic x Subscript i Baseline right-parenthesis plus log p Subscript theta Baseline left-parenthesis bold-italic x Subscript i Baseline comma bold-italic z Subscript left-parenthesis i comma j right-parenthesis Baseline right-parenthesis right-bracket 2nd Column Blank EndLayout

      where bold-italic z Subscript left-parenthesis i comma j right-parenthesis Baseline equals g Subscript phi Baseline left-parenthesis bold-italic epsilon Subscript left-parenthesis i comma j right-parenthesis Baseline comma bold-italic x Subscript i Baseline right-parenthesis and bold-italic epsilon Subscript left-parenthesis i comma j right-parenthesis Baseline tilde p left-parenthesis bold-italic epsilon right-parenthesis. The selections of p left-parenthesis epsilon right-parenthesis and g Subscript phi Baseline left-parenthesis dot right-parenthesis are discussed in detail in Kingma and Welling [17].

      6.1 Introduction

      The previously introduced models have the same assumptions on the data, that is, the independence among the samples and the fixed input size. However, these assumptions may not be true in some cases, thus limiting the application of these models. For example, videos can have different lengths, and frames of the same video are not independent, and sentences of an chapter can have different lengths and are not independent.

      A RNN is another modified DNN that is used primarily to handle sequential and time series data. In a RNN, the hidden layer of each input is a function of not just the input layer but also the previous hidden layers of the inputs before it. Therefore, it addresses the issues of dependence among samples and does not have any restriction on the input size. RNNs are used primarily in natural language processing applications, such as document modeling and speech recognition.

      6.2 Architecture

Скачать книгу