Doob martingale

A Doob martingale (named after Joseph L. Doob,[1] also known as a Levy martingale) is a mathematical construction of a stochastic process which approximates a given random variable and has the martingale property with respect to the given filtration. It may be thought of as the evolving sequence of best approximations to the random variable based on information accumulated up to a certain time.

When analyzing sums, random walks, or other additive functions of independent random variables, one can often apply the central limit theorem, law of large numbers, Chernoff's inequality, Chebyshev's inequality or similar tools. When analyzing similar objects where the differences are not independent, the main tools are martingales and Azuma's inequality.

Definition

Let ${\displaystyle Y}$ be any random variable with ${\displaystyle \mathbb {E} [|Y|]<\infty }$. Suppose ${\displaystyle \left\{{\mathcal {F}}_{0},{\mathcal {F}}_{1},\dots \right\}}$ is a filtration, i.e. ${\displaystyle {\mathcal {F}}_{s}\subset {\mathcal {F}}_{t}}$ when ${\displaystyle s<t}$. Define

${\displaystyle Z_{t}=\mathbb {E} [Y\mid {\mathcal {F}}_{t}],}$

then ${\displaystyle \left\{Z_{0},Z_{1},\dots \right\}}$ is a martingale,[2] namely Doob martingale, with respect to filtration ${\displaystyle \left\{{\mathcal {F}}_{0},{\mathcal {F}}_{1},\dots \right\}}$.

To see this, note that

• ${\displaystyle \mathbb {E} [|Z_{t}|]=\mathbb {E} [|\mathbb {E} [Y\mid {\mathcal {F}}_{t}]|]\leq \mathbb {E} [\mathbb {E} [|Y|\mid {\mathcal {F}}_{t}]]=\mathbb {E} [|Y|]<\infty }$;
• ${\displaystyle \mathbb {E} [Z_{t}\mid {\mathcal {F}}_{t-1}]=\mathbb {E} [\mathbb {E} [Y\mid {\mathcal {F}}_{t}]\mid {\mathcal {F}}_{t-1}]=\mathbb {E} [Y\mid {\mathcal {F}}_{t-1}]=Z_{t-1}}$ as ${\displaystyle {\mathcal {F}}_{t-1}\subset {\mathcal {F}}_{t}}$.

In particular, for any sequence of random variables ${\displaystyle \left\{X_{1},X_{2},\dots ,X_{n}\right\}}$ on probability space ${\displaystyle (\Omega ,{\mathcal {F}},{\text{P}})}$ and function ${\displaystyle f}$ such that ${\displaystyle \mathbb {E} [|f(X_{1},X_{2},\dots ,X_{n})|]<\infty }$, one could choose

${\displaystyle Y:=f(X_{1},X_{2},\dots ,X_{n})}$

and filtration ${\displaystyle \left\{{\mathcal {F}}_{0},{\mathcal {F}}_{1},\dots \right\}}$ such that

${\displaystyle {\begin{aligned}{\mathcal {F}}_{0}&:=\left\{\phi ,\Omega \right\},\\{\mathcal {F}}_{t}&:=\sigma (X_{1},X_{2},\dots ,X_{t}),\forall t\geq 1,\end{aligned}}}$

i.e. ${\displaystyle \sigma }$-algebra generated by ${\displaystyle X_{1},X_{2},\dots ,X_{t}}$. Then, by definition of Doob martingale, process ${\displaystyle \left\{Z_{0},Z_{1},\dots \right\}}$ where

${\displaystyle {\begin{aligned}Z_{0}&:=\mathbb {E} [f(X_{1},X_{2},\dots ,X_{n})\mid {\mathcal {F}}_{0}]=\mathbb {E} [f(X_{1},X_{2},\dots ,X_{n})],\\Z_{t}&:=\mathbb {E} [f(X_{1},X_{2},\dots ,X_{n})\mid {\mathcal {F}}_{t}]=\mathbb {E} [f(X_{1},X_{2},\dots ,X_{n})\mid X_{1},X_{2},\dots ,X_{t}],\forall t\geq 1\end{aligned}}}$

forms a Doob martingale. Note that ${\displaystyle Z_{n}=f(X_{1},X_{2},\dots ,X_{n})}$. This martingale can be used to prove McDiarmid's inequality.

McDiarmid's inequality

Statement[1]

Consider independent random variables ${\displaystyle X_{1},X_{2},\dots X_{n}}$ on probability space ${\displaystyle (\Omega ,{\mathcal {F}},{\text{P}})}$ where ${\displaystyle X_{i}\in {\mathcal {X}}_{i}}$ for all ${\displaystyle i}$ and a mapping ${\displaystyle f:{\mathcal {X}}_{1}\times {\mathcal {X}}_{2}\times \cdots \times {\mathcal {X}}_{n}\rightarrow \mathbb {R} }$. Assume there exist constant ${\displaystyle c_{1},c_{2},\dots ,c_{n}}$ such that for all ${\displaystyle i}$,

${\displaystyle {\underset {x_{1},\cdots ,x_{i-1},x_{i},x_{i}',x_{i+1},\cdots ,x_{n}}{\sup }}|f(x_{1},\dots ,x_{i-1},x_{i},x_{i+1},\cdots ,x_{n})-f(x_{1},\dots ,x_{i-1},x_{i}',x_{i+1},\cdots ,x_{n})|\leq c_{i}.}$

(In other words, changing the value of the ${\displaystyle i}$th coordinate ${\displaystyle x_{i}}$ changes the value of ${\displaystyle f}$ by at most ${\displaystyle c_{i}}$.) Then, for any ${\displaystyle \epsilon >0}$,

${\displaystyle {\text{P}}(f(X_{1},X_{2},\cdots ,X_{n})-\mathbb {E} [f(X_{1},X_{2},\cdots ,X_{n})]\geq \epsilon )\leq \exp \left(-{\frac {2\epsilon ^{2}}{\sum _{i=1}^{n}c_{i}^{2}}}\right),}$

${\displaystyle {\text{P}}(f(X_{1},X_{2},\cdots ,X_{n})-\mathbb {E} [f(X_{1},X_{2},\cdots ,X_{n})]\leq -\epsilon )\leq \exp \left(-{\frac {2\epsilon ^{2}}{\sum _{i=1}^{n}c_{i}^{2}}}\right),}$

and

${\displaystyle {\text{P}}(|f(X_{1},X_{2},\cdots ,X_{n})-\mathbb {E} [f(X_{1},X_{2},\cdots ,X_{n})]|\geq \epsilon )\leq 2\exp \left(-{\frac {2\epsilon ^{2}}{\sum _{i=1}^{n}c_{i}^{2}}}\right).}$

Proof

Pick any ${\displaystyle x_{1}',x_{2}',\cdots ,x_{n}'}$ such that the value of ${\displaystyle f(x_{1}',x_{2}',\cdots ,x_{n}')}$ is bounded, then, for any ${\displaystyle x_{1},x_{2},\cdots ,x_{n}}$, by triangle inequality,

${\displaystyle {\begin{aligned}&|f(x_{1},x_{2},\cdots ,x_{n})-f(x_{1}',x_{2}',\cdots ,x_{n}')|\\\leq &|f(x_{1},x_{2},\cdots ,x_{n})-f(x_{1}',x_{2},\cdots ,x_{n})|\\&+\sum _{i=1}^{n-1}|f(x_{1}',\cdots ,x_{i}',x_{i+1},\cdots ,x_{n})-f(x_{1}',x_{2}',\cdots ,x_{i}',x_{i+1}',x_{i+2},\cdots ,x_{n})|\\\leq &\sum _{i=1}^{n}c_{i},\end{aligned}}}$

thus ${\displaystyle f}$ is bounded.

Define ${\displaystyle Z_{i}:=\mathbb {E} [f(X_{1},X_{2},\cdots ,X_{n})\mid X_{1},X_{2},\cdots ,X_{i}]}$ for all ${\displaystyle i\geq 1}$ and ${\displaystyle Z_{0}:=\mathbb {E} [f(X_{1},X_{2},\cdots ,X_{n})]}$. Note that ${\displaystyle Z_{n}=f(X_{1},X_{2},\cdots ,X_{n})}$. Since ${\displaystyle f}$ is bounded, by the definition of Doob martingale, ${\displaystyle \left\{Z_{i}\right\}}$ forms a martingale. Now define ${\displaystyle {\begin{aligned}U_{i}&={\underset {x\in {\mathcal {X}}_{i}}{\sup }}\mathbb {E} [f(X_{1},\cdots ,X_{n})\mid X_{1},\cdots ,X_{i-1},x]-\mathbb {E} [f(X_{1},\cdots ,X_{n})\mid X_{1},\cdots ,X_{i-1}],\\L_{i}&={\underset {x\in {\mathcal {X}}_{i}}{\inf }}\mathbb {E} [f(X_{1},\cdots ,X_{n})\mid X_{1},\cdots ,X_{i-1},x]-\mathbb {E} [f(X_{1},\cdots ,X_{n})\mid X_{1},\cdots ,X_{i-1}].\end{aligned}}}$

Note that ${\displaystyle L_{i}\leq Z_{i}-Z_{i-1}\leq U_{i}}$ and ${\displaystyle U_{i},L_{i}}$ are both ${\displaystyle {\mathcal {F}}_{i-1}}$-measurable. In addition,

${\displaystyle {\begin{aligned}U_{i}-L_{i}&={\underset {x_{u}\in {\mathcal {X}}_{i},x_{l}\in {\mathcal {X}}_{i}}{\sup }}\mathbb {E} [f(X_{1},\cdots ,X_{n})\mid X_{1},\cdots ,X_{i-1},x_{u}]-\mathbb {E} [f(X_{1},\cdots ,X_{n})\mid X_{1},\cdots ,X_{i-1},x_{l}]\\&={\underset {x_{u}\in {\mathcal {X}}_{i},x_{l}\in {\mathcal {X}}_{i}}{\sup }}\int _{{\mathcal {X}}_{i+1}\times \cdots \times {\mathcal {X}}_{n}}f(X_{1},\cdots ,X_{i-1},x_{u},x_{i+1},\cdots ,x_{n}){\text{d}}{\text{P}}_{X_{i+1},\cdots ,X_{n}\mid X_{1},\cdots ,X_{t-1},x_{u}}(x_{i+1},\cdots ,x_{n})\\&\quad -\int _{{\mathcal {X}}_{i+1}\times \cdots \times {\mathcal {X}}_{n}}f(X_{1},\cdots ,X_{i-1},x_{l},x_{i+1},\cdots ,x_{n}){\text{d}}{\text{P}}_{X_{i+1},\cdots ,X_{n}\mid X_{1},\cdots ,X_{t-1},x_{l}}(x_{i+1},\cdots ,x_{n})\\&={\underset {x_{u}\in {\mathcal {X}}_{i},x_{l}\in {\mathcal {X}}_{i}}{\sup }}\int _{{\mathcal {X}}_{i+1}\times \cdots \times {\mathcal {X}}_{n}}f(X_{1},\cdots ,X_{i-1},x_{u},x_{i+1},\cdots ,x_{n}){\text{d}}{\text{P}}_{X_{i+1},\cdots ,X_{n}}(x_{i+1},\cdots ,x_{n})\\&\quad -\int _{{\mathcal {X}}_{i+1}\times \cdots \times {\mathcal {X}}_{n}}f(X_{1},\cdots ,X_{i-1},x_{l},x_{i+1},\cdots ,x_{n}){\text{d}}{\text{P}}_{X_{i+1},\cdots ,X_{n}}(x_{i+1},\cdots ,x_{n})\\&={\underset {x_{u}\in {\mathcal {X}}_{i},x_{l}\in {\mathcal {X}}_{i}}{\sup }}\int _{{\mathcal {X}}_{i+1}\times \cdots \times {\mathcal {X}}_{n}}f(X_{1},\cdots ,X_{i-1},x_{u},x_{i+1},\cdots ,x_{n})\\&\quad -f(X_{1},\cdots ,X_{i-1},x_{l},x_{i+1},\cdots ,x_{n})\ {\text{d}}{\text{P}}_{X_{i+1},\cdots ,X_{n}}(x_{i+1},\cdots ,x_{n})\\&\leq {\underset {x_{u}\in {\mathcal {X}}_{i},x_{l}\in {\mathcal {X}}_{i}}{\sup }}\int _{{\mathcal {X}}_{i+1}\times \cdots \times {\mathcal {X}}_{n}}c_{i}\ {\text{d}}{\text{P}}_{X_{i+1},\cdots ,X_{n}}(x_{i+1},\cdots ,x_{n})\\&\leq c_{i}\end{aligned}}}$

where the third equality holds due to the independence of ${\displaystyle X_{1},X_{2},\cdots ,X_{n}}$. Then, applying the general form of Azuma's inequality to ${\displaystyle \left\{Z_{i}\right\}}$, we have

${\displaystyle {\text{P}}(f(X_{1},\cdots ,X_{n})-\mathbb {E} [f(X_{1},\cdots ,X_{n})]\geq \epsilon )={\text{P}}(Z_{n}-Z_{0}\geq \epsilon )\leq \exp \left(-{\frac {2\epsilon ^{2}}{\sum _{i=1}^{n}c_{i}^{2}}}\right).}$

One-sided bound from the other direction is obtained by applying Azuma's inequality to ${\displaystyle \left\{-Z_{i}\right\}}$ and two-sided bound follows from union bound. ${\displaystyle \square }$

See also

• Concentration inequality - a summary of McDiarmid's and several similar inequalities.

References

1. Doob, J. L. (1940). "Regularity properties of certain families of chance variables" (PDF). Transactions of the American Mathematical Society. 47 (3): 455–486. doi:10.2307/1989964. JSTOR 1989964.
2. Doob, J. L. (1953). Stochastic processes. 101. New York: Wiley. p. 293.