Existence of a Progressive Measurable process

This blog is devoted to the proof (given in P.A. Meyer, Probability and Potentials) of the following theorem (The idea is to "untersify" the proof given there and make it more explicit and also as a reference point for me later on)

Let $X$ be an $\mathcal{F}_t$-adapted stochastic process. Then there exists a modification of $X$ which is progressively measurable.

First, a few supporting results are given.

If $X$ is an $E$ valued measurable function, where $E$ is a separable metric space then there exist a sequence of elementary (i.e. with finite or countable range) $E$ valued measurable functions, $X_n$ which converge uniformly to $X$.

Let $(x_n)_{(n \in \mathbb{N})}$ be a countable dense subset of $E$. Let $B_{n,m}$ be the open ball with center $x_n$ and radius $\epsilon_m = \frac{1}{2^m}$. Let $C_{n,m} = B_{n,m} \backslash \left(\cup_{p < n} B_{p,m}\right)$. Then $C_{n,m}$ are disjoint and cover $E$. Now define, $X_m(\omega) = x_n$ if $X(\omega) \in C_{n,m}$. Then clearly, $\|X_m(\omega)-X(\omega)\| < \frac{1}{2^m}$ and $X_m^{-1}(x_n) = X^{-1}(C_{n,m})$.

Let $\mathcal{M}$ be the collection of equivalence classes of real-valued random variables defined on a probability space $(\Omega,\mathcal{F},P)$. Define the function $\pi : \mathcal{M} \times \mathcal{M} \to \mathbb{R}$ by $\pi(D_1,D_2) = E(|f-g|\wedge 1)$ where $f\in D_1, g\in D_2$ are arbitrary. Then $\pi$ is a metric on $\mathcal{M}$ i.e., ($\mathcal{M},\pi$) is a metric space. $\mathcal{M}$ is also a vector space w.r.t. pointwise addition.

(Note: two random variables belong to the same class, if they are a.s. equal) The following theorem appears in P.A. Meyer on page 11 (Theorem T20)

Let $\mathcal{H}$ be a vector space of bounded real-valued functions defined on $\Omega$, which contains the constant $1$, is closed under uniform convergence, and is such that for every increasing uniformly bounded sequence of non-negative functions $f_n \in \mathcal{H}$, the function $f=\lim_f f_n$ belongs to $\mathcal{H}$. Let $C$ be a subset of $\mathcal{H}$ closed under (pointwise) multiplication. Then the space $\mathcal{H}$ contains all the bounded functions measurable w.r.t. the $\sigma$-field generated by $C$

Let $C^\prime$ be the algebra generated by the constant function $1$ and the elements of of $\mathcal{C}$. That is $C^\prime = \{\sum_{i=1}^N\Pi_{j=1}^{M_i} a_{ij}f_{ij}\}$ where $a_{ij} \in \mathbb{R}$ and $f_{ij} \in C \cup \{1\}$. Clearly, $C^\prime \subset \mathcal{H}$. Consider the set of all algebras $A$ for which $C^\prime \subset A \subset \mathcal{H}$. Then by Zorn's lemma, there exists a maximal algebra $A_0$ in this set.

The algebra $A_0$ is closed under uniform convergence. This is because if $g \in \bar{A_0}$, then as $\mathcal{H}$ is closed under uniform convergence, we have $g \in \mathcal{H}$. Then the algebra generated by $A_0$ and $g$ is strictly "greater" than $A_0$ and is contained in $\mathcal{H}$ (if $f\in A_0, \|fg_n-fg\| \leq \|f\|\|g_n-g\| \to 0$ uniformly and hence $fg\in \mathcal{H}$) which contradicts the maximality of $A_0$. It can also be easily seen that the algebra $A_0$ contains all the constants as it contains $1$. So, we have $A_0 \subset \mathcal{H}$ satisfying all the properties $\mathcal{H}$ satisfies.

Now, it is shown that if $f \in A_0$ then $|f| \in A_0$. The map $x \to |x|$ can be approximated uniformly by polynomials on each compact interval of $\mathbb{R}$ using Stone-Weierstrass theorem (i.e., there exist a sequence of polynomials $p_n$ such that for every compact subset $K$ of $\mathbb{R}$ and $\epsilon > 0$ there exists an $N(\epsilon,K)$, so that $\sup_{x\in K}|p_n(x) -|x|| < \epsilon$ for all $n \geq N$). Therfore, the composition $|\cdot| \circ f : \Omega \to \mathbb{R}$ can be uniformly approximated by $p_n\circ f : \Omega \to \mathbb{R}$ where $p_n$ are some polynomials. To show this, let $|f(\omega)| \leq M$ for all $\omega\in \Omega$. Then there exists an $n$ such that for all $n \geq N(\epsilon)$, $\sup_{x\in [-M,M]}|p_n(x) -|x|| < \epsilon$. Hence, we also have $\sup_{\omega \in \Omega} ||f(\omega)|-p_n(f(\omega))| < \epsilon$. This proves the assertion. Now, $p_n \circ f$ is a polynomial in $f$ and hence belongs to $A_0$. As $f \wedge g = ((f+g) + |f-g|)/2, f \vee g = -((-f) \wedge (-g))$, we have that $f\wedge g , f \vee g$ are both contained in $A_0$ if $f,g \in A_0$.

Similarly, let $g$ be the limit of an increasing uniformly bounded sequence of positive elements of $A_0$. Then $g \in \mathcal{H}$. The algebra generated by $A_0$ and $g$ is contained in $\mathcal{H}$ (if $f\in A_0$, write $f=f^+-f^-$ and then there exists a sequence of uniformly bounded non-negative functions $f_n^+ \uparrow f$ pointwise. Then $0 \leq f_n^+g_n \uparrow f^+g$ pointwise. Hence, $f^+g \in \mathcal{H}$ etc.) and therefore, it is equal to $A_0$.

Now, let $\mathcal{G}= \{ E \subset \Omega | 1_E \in A_0\}$. Then, $\mathcal{G}$ is a field. Let $E_i \in \mathcal{G}$ so that $E_i \uparrow$. Then, as $A_0$ is closed w.r.t. limit of non-negative uniformly bounded increasing functions, we have that $\cup_i E_i \in \mathcal{G}$. By De-Movire's formula, $\cap_i E_i \in \mathcal{G}$ if $E_i \downarrow$. Therefore, $\mathcal{G}$ is a $\sigma$-algebra. If $f : \Omega \to \mathbb{R}$ is a bounded $\mathcal{G}$-measurable function, then there exist a sequence of elementary $\mathcal{G}$-measurable functions $f_n$ (see the previous proposition) which uniformly converge to $f$. As $f$ is bounded $f_n \in A_0$ and hence $f \in A_0$.

So, it remains to prove that $C^\prime \subset \mathcal{G}$. This can be shown if every set $B=\{\omega: f(\omega) \geq 1\}$ belongs to $\mathcal{G}$ for $f \in C^\prime$. Now, $g=(f \wedge 1)^+$ belongs to $A_0$ and $1_B = \lim_{n \to \infty} g^n$ (If $\omega \in B$ then $g^n(\omega) = 1$ for every $n$ and if $\omega \in B^c$ then $g^n(\omega) \downarrow 0$ as $n \to \infty$).

Now, the main theorem is proved.

Let $X$ be an $\mathcal{F}_t$-adapted stochastic process. Then there exists a modification of $X$ which is progressively measurable.

Let $\mathcal{M}$ be the collection of equivalence classes of real-valued random variables defined on $(\Omega,\Sigma,P)$ obtained by the a.s. equivalence relation. Every stochastic process $Y_t$ gives rise to a mapping of $\mathbb{R}_+ \to \mathcal{M}$ given by $t \to M_t$, where $\dot{Y}_t$ is the equivalence class of $Y_t$. Consider the collection $\mathcal{C}$ of processes $(Y_t)$ such that:

1. The map $t \to \dot{Y}_t$ takes values in a seprable subspace of $\mathcal{M}$
2. The inverse image of every open ball in $\mathcal{M}$ under this map is a Borel subset of $\mathbb{R}_+$
3. The map $t \to \dot{Y}_t$ is the uniform limit of a sequence of measurable elementary functions with values in $\mathcal{M}$

Now, the first two properties imply the third property and vice versa. These are shown next. Firstly, if the map $t \to \dot{Y}_t$ satisfies the first two properties, then by proposition above, there exist $\mathcal{M}$ valued elementary measurable functions which converge uniformly to the given map. For the converse, let the map $g: \mathbb{R}_+ \to \mathcal{M}$ be the uniform limit of the maps $f_n : \mathbb{R}_+ \to \mathcal{M}$. Let $A_n = \text{range}(f_n)$ and $S=\cup_n A_n$. Clearly, $S$ is countable. As $g$ is the uniform limit of $f_n$, it is also the pointwise limit and so $g(t) = \lim_{n \to \infty} f_n(t)$. As each $f_n(t) \in S$ and $S$ is countable, we have that $g$ takes values in a separable subspace of $\mathcal{M}$ (namely the closure of $S$ in $\mathcal{M}$). As each $f_n$ is measurable, we immediatley have that $g$ is measurable.

Every process $(X_t)$ is a map $(t,\omega) \to X_t(\omega)$. Addition of two processes is defined as pointwise addition and scalar multiplication is defined in the obvious manner. Then, $\mathcal{C}$ is a vector space. This is because, if $f_n \to (t \to \dot{Y}_t)$ and $g_n \to (t \to \dot{Z}_t)$ uniformly where $f_n,g_n$ are elementary, then $f_n + g_n \to (t \to \dot{Y}_t + \dot{Z}_t ) =(t \to \dot{(Y+Z)}_t)$ uniformly. Hence, $\mathcal{C}$ is closed w.r.t. addition. It will be shown next that $\mathcal{C}$ is also closed w.r.t. sequential pointwise convergence. The processes $Y^n$ converge to $Y$ pointwise if $Y^n_t \to Y_t$ for each $t$ almost surely.

Let $Y^n \in \mathcal{C}$ for each $n \in \mathbb{N}$. Then, $g_n(t) = \dot{Y}^n_t$ takes values in a separable subspace $E_n$ of $\mathcal{M}$ and $S_n$ be a countable dense subset of $E_n$. As before, take $S=\cup_n S_n, E=\cup_n E_n$. Then, clearly $E$ is separable and $S$ is a countable dense subset of $E$. Assume now that $Y^n_t \to Y_t$ for each $t$ almost surely. Then, $\dot{Y}_t^n \to \dot{Y}_t$ for each $t$ as $E(|Y^n_t-Y_t|\wedge 1) \to 0$. Let $g(t) = \lim_n g_n(t) = \lim_n \dot{Y}^n_t = \dot{Y}_t$. Then, as $g_n(t) \in E$ for every $n,t$, $g$ takes values in the separable subspace $\bar{E}$. Measurability of $g$ follows from the measurability of $g_n$.

Now, note that $\mathcal{C}$ contains all processes of the form $Y_t(\omega) = I(t)Y(\omega)$, (where $I$ is the indicator function of an interval of $\mathbb{R}_+$ and $Y$ is a bounded r.v.) as the range of the map $t \to \dot{Y}_t$ is $\{0,\dot{Y}\}$. Properties 1) and 2) above are then obviously satisfied by such processes. Now, as $\mathcal{C}$ is closed under sequential pointwise convergence, it is closed under uniform convergence. Let $\mathcal{H} \subset \mathcal{C}$ such that $\mathcal{H}$ contains all the bounded processes in $\mathcal{C}$. Then, clearly, $\mathcal{H}$ is closed under pointwise convergence. By applying the previous theorem, we get that $\mathcal{C}$ contains all bounded measurable processes. Any measurable process is a pointwise limit bounded measurable processes. Hence $\mathcal{C}$ contains all measurable processes.

Consider now a measurable process $(X_t)$ adapted to $\mathcal{F}_t$. From what has just been proven, the map $t \to \dot{X}_t$ is the uniform limit of a sequence of measurable elementary functions $f_n : \mathbb{R}_+ \to \mathcal{M}$. This sequence of functions satisfy the following:

1. As the $f_n$ are elementary, there exists a partition of $\mathbb{R}_+$ into a sequence of Borel sets $A_k^n$ and a sequence of elements of $\mathcal{M}$ denoted by $M_k^n$ such that $f_n = M_k^n$ on $A_k^n$.
2. As $f_n$ converges uniformly, $\pi(\dot{X}_t, f_n(t)) \leq \frac{1}{2^{n+1}}$ for every $t$.

Fix $X^n_t \in f_n(t)$ and $M_k^n \ni H_k^n = X_t^n$ for $t \in A_k^n$. Then clearly, $\pi(X_t - X_t^n) \leq \frac{1}{2^{n+1}}\forall t$ (where $\pi(X) = E(|X| \wedge 1)$). Construct random variables $G_k^n$ in the following way: Let $s_k^n$ be the infimum of the set $A_k^n$. If $s_k^n \in A_k^n$ , put $G_k^n = X_{s_k^n}$. It is clear that $\pi(G_k^n-H_k^n) \leq \frac{1}{2^{n+1}}$ in this case. If $s_k^n \notin A_k^n$ denote by $G_k^n$ any random variable, measurable w.r.t. $\mathcal{F}_{s^n_k+}$ such that $\pi(G_k^n - H_k^n) \leq \frac{1}{2^{n+1}}$. For example, we can take $G_k^n = \liminf X_{t_p}$ where $A_k^n \ni t_p \downarrow s_k^n$. Define, $Y_t^n = G_k^n$ for $t \in A_k^n$. Then $Y_t^n$ is clearly progressively measurable. Also, $\pi(X_t-Y_t^n) \leq \pi(X_t-X_t^n + X_t^n - Y_t^n) = \pi(X_t - X_t^n) + \pi (X_t^n - Y_t^n) \leq 2\frac{1}{2^{n+1}}$. Finally, set $$Y_t(\omega) = \begin{cases} \lim_n Y_t^n(\omega) & \text{ if the limit exists}\\ 0 & \text{ otherwise } \end{cases} $$ Then, $Y_t$ is progressively measurable and as $\sum_n \pi(Y_t^n-Y_t^{n+1}) < \infty$, the sequence converges in probability and almost surely. Hence, the set where the limit does not exist is P-null. Similarly, we get that $Y_t = X_t$ a.s.

If $X_t$ is a bounded process, it can be seen that $Y_t$ is also a bounded process.

Ito Integral

Let $(\Omega,\mathcal{F},P)$ be a probability space. We first define a stochastic process.

A stochastic process is a measurable map from $[0,\infty) \times \Omega \to \mathbb{R}$.

Therefore, if $X(t,\omega)$ is a stochastic process, $X^{-1}(B) \in \mathcal{B}[0,\infty)\times \mathcal{F}$ for every Borel set $B \in \mathcal{B}(\mathbb{R})$. Hence, whenever we say "stochastic process" we automatically assume measurability.

We also have a notion of adaptedness w.r.t. increasing $\sigma$-fields (also known as a filtration). Filtrations are useful in giving mathematical meaning to phrases such as "less information"/"more information" etc.

A stochastic process $X$ is said to be adapted to a increasing family of $\sigma$-fields $\mathcal{F}_t$, if $X(t,\cdot)$ is $\mathcal{F}_t$-measurable

A more stronger notion than the joint measurability required in the definition of a stochastic process is that of progressive measurability.

A stochastic process $X(t,\omega) : [0,\infty) \times \Omega \to \mathbb{R}$ is said to be progressively measurable if the restricted map $X|_t : [0,t] \times \Omega \to \mathbb{R}$ of $X$ is measurable w.r.t $\mathcal{B}[0,t] \times \mathcal{F}_t$ for every $t \in \mathbb{R}$

A process is continuous if the sample paths, i.e. the (random) functions $X(\cdot,\omega)$ are continuous almost surely (in $\omega$). Left/Right-continuous processes are defined similarly.

Progressive measurability implies i) measurability and ii) adaptedness.

As $X$ is progressively measurable, we have, for every Borel set $B\in \mathcal{B}(\mathbb{R})$, $(X|_t)^{-1} (B) \in \mathcal{B}[0,t] \times \mathcal{F}_t$. We also have $X^{-1}(B) = \cup_{t=1}^\infty (X|_t)^{-1} (B)$. To prove measurability, it suffices to note that $\mathcal{B}[0,t] \times \mathcal{F}_t \subset \mathcal{B}[0,\infty) \times \mathcal{F}$ for every $t$. As $((X)^{-1}(B))_t = ((X|_t)^{-1}(B))_t \in \mathcal{F}_t $ (where the second subscript denotes the cross-section of the set) we get adaptedness.

Adaptedness and right-continuity imply progressive measurability. The proof can be found in Kallianpur and is not too difficult. This is stated as a lemma next.

Let $X$ be a stochastic process, which is $\mathcal{F}_t$ adapted and right-continuous. Then $X$ is progressively measurable.

More important is the following theorem which generalizes the above proposition (when the assumption of right continuity is dropped). Before, this theorem is given, the notion of modification of a process is defined.

Modification: $Y$ is a modification of $X$ if $P(Y(t,\cdot) = X(t,\cdot))$ for each $t$.

Let $X$ be an $\mathcal{F}_t$-adapted stochastic process. Then there exists a modification of $X$ which is progressively measurable.

The proof of this can be found in P.A. Meyer and is given here.

Now, basic ideas about sections of sets are given. Let $(\Omega_1,\Sigma_1,\mu_1), (\Omega_2,\Sigma_2,\mu_2)$ be two finite measure spaces. Let the product space be denoted by $(\Omega,\Sigma)$ where $\Omega =\Omega_1 \times \Omega_2$ and $\Sigma = \Sigma_1 \times \Sigma_2$, the product $\sigma$-albegra.

Section of a set : Let $E\in \Sigma$. Then the section of $E$ at $\omega_1$ (denoted by $(E)_{\omega_1})$ is defined as $(E)_{\omega_1} = \{\omega_2 | (\omega_1,\omega_2) \in E\}$

Suppose we define a new function $\Omega_1 \to \mathbb{R}$ by $\omega_1 \to \mu_2((E)_{\omega_1})$ for some fixed $E \in \Sigma$. Is this function measurable? It is not directly obvious but the proof given next shows that this is the case.

Let $\mathcal{D} = \{E \subset \Omega| \omega_1 \to \mu_2((E)_{\omega_1}) \text{ is measurable }\}$. The following can now be observed

• If $E = E_1\times E_2$ (called a rectangle) where $E_1 \in \Sigma_1, E_2 \in \Sigma_2$ then $\mu_2((E)_{\omega_1}) = \mu_2(E_2)I_{E_1}(\omega_1)$, which is measurable.
• Let $E^1, E^2$ be two rectangles then it can be seen that $E^1 \cap E^2$ belongs to $\mathcal{D}$ and that $(E^1)^c$ is also in $\mathcal{D}$
• If $E^i \in \mathcal{D}$ and $E^i \uparrow$ then using monotonicity of the measure it is clear that $\cup E^i \in \mathcal{D}$. Similalry, $\cap E^i \in \mathcal{D}$ if $E^i \downarrow$

Above, we have shown that $\mathcal{D}$ is a monotone class and that it contains a field which generates the product $\sigma$-field $\Sigma$. Now, we use the fact that the smallest monotone class and the smallest $\sigma$-field containing a given field are the same. Therefore, $\mathcal{D}$ contains all measurable sets.

We now proceed with the construction of the Ito integral. This integral is developed in much the same way as the Lebesgue integral, for example, i.e. defining the integral for "simple" functions first and then extending the definition for the general case.

Before, we start it is important that we already have constructed a Brownian Motion, B, on some complete probability space $(\Omega,\mathcal{F},P)$. Some of the simplest constructions are given in 1) Brownian Motion by Peres and Morters and 2) Stochastic Filtering Theory by G. Kallianpur. Let $\mathcal{F}_t$ be the smallest $\sigma$-field containing $\sigma(B_s, 0\leq s\leq t)$ and all the $P$-null sets of $\mathcal{F}$. (Note: $\mathcal{F}_t = \{A \cup B | A \in \sigma(B_s, 0\leq s\leq t), B \text{ a P-null set of } \mathcal{F}\}$). Adding the $P$-null sets of $\mathcal{F}$ does not disturb the properties of the Brownian Motion, i.e., $B_s - B_t$ is still independent of $\mathcal{F}_t$.

This following is based on the texts by Bernt Oksendal, Kartazas Shreve and attempts to give any missing details

Let $\mathcal{V} = \mathcal{V}(S,T)$ be the class of functions $f(t,\omega) : [0,\infty) \times \Omega \to \mathbb{R}$ which satisfy

• $(t,\omega) \to f(t,\omega)$ is $\mathcal{B}[0,\infty) \times \mathcal{F}$ measurable.
• $f(t,\omega)$ is $\mathcal{F}_t$-adapted
• $E(\int_S^T f(t,\omega)^2 dt) < \infty$

The integral is defined in steps:

Step 1 : Bounded Step Processes :- First, the integral is defined for step processes. Let $S = t_0 < t_2 < \cdots < t_n = T$ and $e_j : \Omega \to \mathbb{R}$ be $\mathcal{F}_{t_j}$-measurable and bounded. Now, define $\phi(t,\omega) = \Sigma_{j=0}^{n-1} e_j(\omega)\chi_{[t_j,t_{j+1})}(t)$. It needs to be checked if the process $\phi(t,\omega)$ satisfies the requirements of the definition above

• $\phi^{-1}(B)= \cup_{j=0}^{n-1} \left ([t_j, t_{j+1})\times e_j^{-1}(B) \right) \in \mathcal{B}[0,\infty) \times \mathcal{F}$ for every Borel set $B \in \mathcal{B}(\mathbb{R})$.
• If $t\in [t_j,t_{j+1})$ then $\phi(t,\omega) = e_j(\omega)$. $e_j$ is $\mathcal{F}_{t_j}$-measurable and hence $\mathcal{F}_t$-measurable.
• As $\phi(t,\omega)$ is bounded, the expectation $E(\int_S^T \phi(t,\omega)^2 dt)$ is finite.

The stochastic integral for these "step processes" $\phi(t,\omega)$ is defined as \[\int_S^T\phi(t,\omega)dB_t(\omega) = \sum_{j=0}^ne_j(\omega) (B_{t_{j+1}}(\omega) - B_{t_j}(\omega))\] The first thing to note here is that the stochastic integral is a random variable.

Ito Isometry for bounded step processes : \[E\left[\left(\int_S^T\phi(t,\omega)dB_t(\omega)\right)^2\right] = E\left(\int_S^T \phi(t,\omega)^2dt\right)\]

The proof is straight-forward and uses independence of increments of Brownian motion and the fact that $e_j$ is independent of $B_{t_{j+1}} - B_{t_j}$. This Lemma is used in later steps to show uniqueness of the integral.

The following proposition is easy to see.

The integral for bounded step-processes is Linear, i.e., if $\phi(t,\omega), \eta(t,\omega)$ are two bounded step processes then $\int_S^T(\phi(t,\omega)+\eta(t,\omega))dB_t(\omega) = \int_S^T\phi(t,\omega)dB_t(\omega) +\int_S^T\eta(t,\omega)dB_t(\omega)$

Step 2: Bounded Continuous Processes : Now, the integral is defined for bounded, continuous processes.

Let $g\in \mathcal{V}$ be bounded and $g(\cdot,\omega)$ continuous for each $\omega$. Then, there exist bounded step processes $\phi_n \in \mathcal{V}$ such that \[E\left(\int_S^T (g-\phi_n)^2dt\right) \to 0~\text{as } n \to \infty\]

Let $\phi_n(t,\omega) = \sum_{j=0}^{\lfloor (T-S)(2^n) \rfloor -1}g(S+\frac{j}{2^n},\omega)\chi_{[S+\frac{j}{2^n}, S+\frac{j+1}{2^n})}(t)$. As the dyadic rationals are dense in $\mathbb{R}$ and $g$ is continuous we get that $\phi_n(t,\omega) \to g(t,\omega)$ as $n \to \infty$ for every $t \in [S,T)$ and $\omega \in \Omega$. Therefore, $(\phi_n(t,\omega) - g(t,\omega))^2 \to 0$ as $n \to \infty$. As $g$ and $\phi_n$ are bounded processes, using dominated convergence theorem, we get \[\int_S^T (g(t,\omega)-\phi_n(t,\omega))^2 dt \to 0 \text{ as } n \to \infty\] Now,$\int_S^T (g(t,\omega)-\phi_n(t,\omega))^2 dt$ is uniformly bounded for each $n$ and therefore using dominated convergence once again, we get that \[E\left(\int_S^T (g(t,\omega)-\phi_n(t,\omega))^2 dt\right) \to 0 \text{ as } n \to \infty\]

$\{\int_S^T \phi_n(t,\omega)dB_t(\omega)\}$ form a Cauchy sequence in $\mathcal{L}^2(\Omega,\mathcal{F},P)$

\begin{array}\\ E\left(\int_S^T \phi_n(t,\omega)dB_t(\omega) - \int_S^T \phi_m(t,\omega)dB_t(\omega)\right)^2 &= E\left(\int_S^T (\phi_n(t,\omega)- \phi_m(t,\omega))dB_t(\omega)\right)^2 \\ &= E\left(\int_S^T (\phi_n(t,\omega)- \phi_m(t,\omega))^2dt\right) \end{array} The first equality is due to linearity of the integral for bounded step processes. The second equality is because of Ito isometry proved above. By adding and subtracting $g(t,\omega)$ in the integral, it can be seen that the left hand quantity above can be made arbitrarily small by choosing large enough $n,m$.

As the sequence $\{\int_S^T \phi_n(t,\omega)dB_t(\omega)\}$ is cauchy, there exists a subsequence which converges almost surely (Note: this is the main part in the proof that $\mathcal{L}^2$ is complete. See, Probability with Martingales by David Williams, Page 65). This subsequence can infact be constructed. Let the limit of this subsequence be $I_g(\omega)$. Then we have $E(I_g(\omega) - \int_S^T \phi_n(t,\omega)dB_t(\omega))^2 \to 0$ as $n \to \infty$.

The integral for bounded, continuous processes is then defined as \[\int_S^Tg(t,\omega)dB_t(\omega) = I_g(\omega) = \lim_{n \to \infty}\int_S^T \phi_n(t,\omega)dB_t(\omega)\text{ (limit in } \mathcal{L}^2(\Omega,\mathcal{F},P))\]

Step 3: Bounded Progressively Measurable Processes

Suppose we have a bounded progressively measurable process, $f : [0,\infty) \times \Omega \to \mathbb{R}$ and define $F(t,\omega) = \int_0^t f(s,\omega)ds, 0\leq t$. Then, $F(t,\omega)$ is adapted and continuous (and hence progressively measurable)

Let \[f_m(s,\omega) = \sum_{j=-m2^m+1}^{m2^m} \left(\frac{j-1}{2^m}\right) \chi_{[\frac{j-1}{2^m},\frac{j}{2^m})}(f(s,\omega))\] We have $f_m(s,\omega) \to f(s,\omega)$ as $m \to \infty$. Hence, it suffices to show that $\int_0^t f_m(s,\omega)ds$ is progressively measurable for each $m$ (as $\int_0^t f_m(s,\omega)ds \to \int_0^t f(s,\omega)ds$ due to boundedness). Then, \[\int_0^t f_m(s,\omega)ds = \sum_{j=-m2^m+1}^{m2^m} \left(\frac{j-1}{2^m}\right) \lambda\left(\left(f^{-1}\left[\frac{j-1}{2^m}, \frac{j}{2^m}\right)\right)_\omega\cap [0,t]\right)\] In the above, $\left(\left(f^{-1}\left[\frac{j-1}{2^m}, \frac{j}{2^m}\right)\right)_\omega\cap [0,t]\right)$ equals $\left((f|_t)^{-1}\left[\frac{j-1}{2^m}, \frac{j}{2^m}\right)\right)_\omega \in \mathcal{B}[0,t] \times \mathcal{F}_t$ due to the progressive measurability of $f$. The map $\Omega \to \mathbb{R}$, given by $\omega \to \lambda(E_\omega)$ is $\mathcal{F}_t$ measurable, if $ E \in \mathcal{B}[0,t] \times \mathcal{F}_t$. This can be shown similarly to the method described before. Hence, $\int_0^t f_m(s,\omega)ds$ and therefore $\int_0^t f(s,\omega)ds$ is adapted. Continuity is stratight forward due to boundedness.

Now, let $f(t,\omega)$ be a bounded progressively measurable process and define $F(t,\omega) = \int_0^t f(s,\omega)ds$ as before. Also let $f_m(t,\omega) =(F(t,\omega) - F(\max((t-\frac{1}{m}),0),\omega))/(1/m)$. Then, $f_m \in \mathcal{V}$ and continuous. Therefore, by the previous step we have a sequence of bounded step processes $f_{m,n}$ such that \[E\int_S^T |f_m(t,\omega)-f_{m,n}(t,\omega)|^2dt \to 0\text{ as } n \to \infty\]

By the fundamental theorem of calculus $F^\prime(t,\omega) = f(t,\omega)$ a.e. w.r.t. the Lebesgue measure on $[0,\infty)$. However, by definition $F^\prime(t,\omega) = \lim_{m \to \infty} f_m(t,\omega)$. This gives \[E\int_S^T |f(t,\omega)-f_{m,n}(t,\omega)|^2dt \to 0\text{ as } m,n \to \infty\]

Step 4 : Bounded, Measurable and Adapted process

In this case, the process $F(t,\omega)$ may not be adapted (This is because $\left(\left(f^{-1}\left[\frac{j-1}{2^m}, \frac{j}{2^m}\right)\right)_\omega\cap [0,t]\right)$ might not have a nice representation as we had in the progressively measurable case. However, we know that there exists a bounded progressively measurable modification $g(t,\omega)$ of $f(t,\omega)$. (The proof of this statement can be found in Probabilty and Potentials by P.A. Meyer. A post on this can be found here. This is also proposition 1.1.4. in Gopinath Kallianpur's text). From the previous step, we then have $g_n(t,\omega)$ such that $E\int_S^T |g(t,\omega)-g_{n}(t,\omega)|^2dt \to 0\text{ as } n \to \infty$. As $g,f$ are modifications of each other, we get $E\int_S^T |f(t,\omega)-g_{n}(t,\omega)|^2dt \to 0\text{ as } n \to \infty$.

Step 5 : $f\in \mathcal{V}$ . Let $h_n(t,\omega) = (-n)1_{f(t,\omega) < -n} + f(t,\omega)1_{-n < f(t,\omega) \leq n} + (n)1_{f(t,\omega) > n}$. We have $h_n(t,\omega) \to f(t,\omega)$ and therefore $E\int_S^T (f-h_n)^2 \to 0$ as $n \to \infty$ from dominated convergence theorem. As each $h_n$ is a bounded, measurable and adapted process, from the previous step it can be approximated by a sequence of bounded step processes. Therefore, the same holds true for $f$.

Holder Spaces

Let $\Omega \subset \mathbb{R}^n$ be an open set with $k \geq 0$ being an integer and $0 < \alpha \leq 1$. The space of Holder continuous functions $C^{0,\alpha}(\Omega)$ is defined as consisting of all those continuous functions $f \in C(\Omega)$ so that \[[f]_{C^{0,\alpha}(K)} := \sup_{x,y \in K, x\neq y} \left\{\frac{|u(x)-u(y)|}{|x-y|^\alpha}\right\} < \infty\] for every compact $K \subset \Omega$. Similarly, the space $C^{0,\alpha}(\bar{\Omega})$ is the set $f\in C(\bar{\Omega})$ so that $[f]_{C^{0,\alpha}(\bar{\Omega})} < \infty$. This space is equipped with the norm $\|f\|_{C^{0,\alpha}(\bar{\Omega})} := \|f\|_{C^0(\bar{\Omega})} + [f]_{C^{0,\alpha}(\bar{\Omega})}$.

$(C^{0,\alpha}(\bar{\Omega}), \|\cdot\|_{C^{0,\alpha}(\bar{\Omega})})$ is a Banach space

Let $f_n$ be a cauchy sequence in $C^{0,\alpha}(\bar{\Omega})$. Therefore, for every $\epsilon > 0$ there exits $M$ such that for all $n,m > M$, we have $\|f_n-f_m\|_{C^{0,\alpha}(\bar{\Omega})} < \epsilon$. This gives that $[f_n-f_m]_{C^{0,\alpha}(\bar{\Omega})} < \epsilon$ and $\|f_n -f_m\|_{C^0(\bar{\Omega})} < \epsilon$. From the latter we get that $f_n$ converges (in the sup norm) to a bounded continuous function $f$ (on $\bar{\Omega}$). From the former we get that $[f_n-f]_{C^{0,\alpha}(\bar{\Omega})} < \epsilon$ by letting $m \to \infty$. Therefore $\|f_n-f\|_{C^{0,\alpha}(\bar{\Omega})} \to 0$ as $n \to \infty$. It remains to show that $f \in {C^{0,\alpha}(\bar{\Omega})}$ but this follows easily by \[[f]_{C^{0,\alpha}(\bar{\Omega})} = [f-f_n+f_n]_{C^{0,\alpha}(\bar{\Omega})} \leq [f-f_n]_{C^{0,\alpha}(\bar{\Omega})} + [f_n]_{C^{0,\alpha}(\bar{\Omega})} < \infty\]

Note that this result does not depend on $\alpha$.

The space $C(\bar{\Omega})$

This post tries to establish that the above mentioned space is Banach. The proof is straight-forward but it is here for reference sake! Let $\Omega \subset \mathbb{R}^d$ be an open set. Denote by $C(\bar{\Omega})$ the set of all bounded continuous functions $u: \Omega \to \mathbb{R}$ which can be continuously extended to $\bar{\Omega}$ in a bounded manner. (These are exactly the same functions which are bounded and continuous on $\bar{\Omega}$) Now, we put the norm (called the sup norm) $\|f\| = \sup_{x \in \bar{\Omega}} f(x)$ on this space. Clearly, $C(\bar{\Omega})$ is a vector space and $\|f\| = \sup_{x \in \bar{\Omega}} |f(x)|$ defines a norm on $C(\bar{\Omega})$. The only non-trivial thing needed to prove this is that $\sup_x (|f(x)| + |g(x)|) \leq \sup_x |f(x)| + \sup_x |g(x)|$ (note that these two values need not be equal).

$C(\bar{\Omega})$ with the norm defined above is a Banach space

Let $f_n$ be a cauchy sequence in $C(\bar{\Omega})$, i.e. for every $\epsilon > 0$, there exists $N$ such that for all $n,m > N$, we have $\|f_n-f_m\| < \epsilon$, i.e. $\sup_x |f_n(x) - f_m(x)| < \epsilon$. This gives that $f_n(x)$ is a cauchy sequence for each $x\in \bar{\Omega}$. As $\mathbb{R}$ is complete, $f_n(x)$ converges for each $x$. Let the limit be $f(x)$ (i.e. $f(x) = \lim_{n \to \infty} f_n(x)$)

Note that we have convergence $f_n$ to $f$ in sup norm. (Just take the limit $m \to \infty$ in the cauchy criterion). It remains to show that $f$ is a bounded continuous function on $\bar{\Omega}$. \[\sup_x|f(x)| \leq \sup_x|f(x) - f_n(x)| + \sup_x|f_n(x)|\] For large enough $n$ the first term on the r.h.s can be made small and second term is finite as $f_n$ is bounded. Therefore $f$ is bounded.

As for continuity, we have \[|f(x)-f(y)| \leq |f(x)-f_n(x)| + |f_n(x)-f_m(x)| + |f_m(x)-f_m(y)| + |f_m(y)-f(y)|\] The first, second and the fourth terms can be made small by choosing large enough $n,m$ independent of $x,y$. The third term can be made arbitrarily small by choosing $y$ sufficiently close to $x$ due to the continuity of $f_m$.

Taylor's Theorem

The Taylor's theorem lets one to write down a function in terms of its derivatives and gives a direct way to compute limits in a few cases. For example, it is useful in computing limits of integrals as will be shown later in the post with the help of an example. Some of the basic theorems are stated/ proved first before giving a statement of the Taylor's theorem. The basic ideas are from Apostol. (All the integrals are Riemann).

Mean Value Theorem (Differentiation) : If $f:[a,b]\to \mathbb{R}$ is continuous on $[a,b]$ and differentiable on $(a,b)$ then $f(b)-f(a)=f^\prime(c)(b-a)$ for some $c\in (a,b)$.

Mean Value Theorem (Integration): Assume that $\int_a^b f(x)dx$ exists. Let $M = \sup_{x \in [a,b]} f(x), m = \inf_{x \in [a,b]} f(x)$. Then there exists a real number $c$ such that $m \leq c \leq M$ and \[\int_a^b f(x)dx = c(b-a)\] If $f$ is continuous on $[a,b]$ then, $c=f(x_0)$ for some $x_0\in [a,b]$

See Apostol for a Proof.

First Fundamental Theorem of Calculus: Assume that $\int_a^b f(x)dx$ exists. Let $F(x) = \int_a^x f(x)dx$. Then $F^\prime(x)$ exists at each point $x \in [a,b]$ where $f$ is continuous.

For $y\in (a,b)$ we have $F(y+h) - F(y) = \int_y^{y+h} f(x)dx = c(y,h)h$ where $\inf_{x \in [y,y+h]} f(x) \leq c(y,h) \leq \sup_{x \in [y,y+h]} f(x)$. Therefore, $\frac{F(y+h) - F(y)}{h} = c(y,h)$. If $f$ is continuous at $y$, then both $\inf_{x \in [y,y+h]} f(x)$ and $\sup_{x \in [y,y+h]} f(x)$ converge to $f(y)$ as $h \to 0$. The same reasoning will give that $\frac{F(y)-F(y-h)}{h}$ also converges to $f(y)$. Hence $F^\prime(y) = f(y)$. For the cases $y=a,y=b$, if we define one side limits of $F$, then clearly the above reasoning applies and we get $F^\prime(a)=f(a)$ and $F^\prime(b)=f(b)$ if $f$ is continuous at $a$ and $b$ respectively. (Ofcourse, continuity is also one-sided here).

Second Fundamental Theorem of Calculus: Assume that $\int_a^b f(x)dx$ exists. Let $g : [a,b] \to \mathbb{R}$ so that $g^\prime(x) = f(x)$ for all $(a,b)$ and $g(a) - g(a+) = g(b) - g(b-)$ (note that $g(a+),g(b-)$ should exist). Then \[\int_a^b f(x)dx = \int_a^b g^\prime(x)dx = g(b) - g(a)\]

Taylor's Theorm : Let $f:[a,b] \to \mathbb{R}$ be such that $f^{(n)}$ is finite on $(a,b)$ and $f^{(n-1)}$ is continuous on $[a,b]$. Assume that $c\in [a,b]$. Then for every $x \in [a,b], x\neq c$ there exits $x_1$ (which depends on $x$) interior to the interval joining $x$ and $c$ such that \[f(x) = f(c) + \sum_{k=1}^{n-1}\frac{f^{(k)}(c)}{k!}(x-c)^k + \frac{f^{(n)}(x_1)}{n!}(x-c)^n\]

If it is known that $f\in C^n[a,b]$ then Taylor's theorem gives a simple bounds as $f^{(n)}$ is bounded on $[a,b]$. A simple application of Taylor's theorem is given below. Suppose $f\in C^2_b(\mathbb{R})$ and we want to estimate the limit \[\lim_{t \to 0} \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \frac{f(x+y\sqrt{t})-f(x)}{t}\exp{(-\frac{1}{2}y^2)}dy\] Using Taylor's theorem, we can write $f(x+y\sqrt{t}) = f(x) + f^{(1)}(x)y\sqrt{t} +\frac{f^{(2)}(x+\theta(y)\sqrt{t})}{2}ty^2$. Therefore, the limit becomes \[\lim_{t \to 0} \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \frac{f^{(1)}(x)y\sqrt{t} +\frac{f^{(2)}(x+\theta(y)\sqrt{t})}{2}ty^2}{t}\exp{(-\frac{1}{2}y^2)}dy\] \[\lim_{t \to 0} \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \frac{f^{(2)}(x+\theta(y)\sqrt{t})}{2}y^2\exp{(-\frac{1}{2}y^2)}dy\] Now, using dominated convergence (based on the fact that $f^{(2)}(x)$ is bounded and continuous), the limit can be pushed inside the integral. Finally we get that the original expression equals \[\frac{1}{2}f^{(2)}(x)\]

$C^k$ Space

Let $\Omega \subset \mathbb{R}^d$ be an open set and $C^k(\bar{\Omega})$ be the set of all bounded functions $u : \Omega \to \mathbb{R}$ whose partial derivatives $D^a u$ for $0 \leq |a| \leq k$ are continuous on $\Omega$ and can be continuously extended to $\bar{\Omega}$ in a bounded way. (Here $D^a u$ represents partial derivative where $a=(a_1,a_2,\cdots,a_d)$ is a multi-index). Define a norm on $C^k(\bar{\Omega})$ by $\|u\|_{C^k} = \underset{0 \leq |a| \leq k}{\max} \underset{x \in \bar{\Omega}} \sup|D^au(x)|$. Then $(C^k(\bar{\Omega}),\|\cdot\|_{C^k})$ is a Banach Space.

Note that $\Omega$ can be unbounded here. The norm is still meaningful as the functions themselves are bounded. The following Mean-Value theorem in higher dimensions is used in the proof. Let $f : \Omega \to \mathbb{R}$ be differentiable (e.g. all partial derivatives of $f$ are continuous). Take two points $x,y\in \Omega$ and assume that the line joining the two points is also contained in $\Omega$. Define $g : [0,1] \to \mathbb{R}$ by $g(t) = f((1-t)x+ty)$. Now, clearly $g$ is differentiable and hence by the Mean Value Theorem, $g(1) - g(0) = g^\prime(c)$ where $c\in (0,1)$. Hence, $f(y) - f(x) = \langle \nabla f((1-c)x+cy),(y-x)\rangle$.

Assume now that $\{u_n\}$ is a cauchy-sequence and let $0 \leq |a| < k$. Also, let $\lim_{n\to \infty} u_n = v^0$. This limit exists and belongs to $C(\bar{\Omega})$. Similarly, let $\lim_{n\to \infty} D^au_n = v^a$. We need to show that $D^av^0 = v^a$.

As $\{u_n\}$ is cauchy, for a given $\epsilon > 0$, there exists $N$ such that $\underset{x \in \bar{\Omega}} \sup |\frac{\partial}{\partial x_i} D^a u_n (x) - \frac{\partial}{\partial x_i} D^a u_m (x)| < \epsilon$ for all $m,n \geq N$. Choose $f = D^au_n - D^au_m : \Omega \to \mathbb{R}$ in the previous paragraph. Then \begin{array} a\frac{\|(D^au_n(y) - D^au_m(y)) - (D^au_n(x) - D^au_m(x))\|}{\|y - x\|} &\leq \|\nabla f((1-c)x+cy)\| \\ &\leq \epsilon \sqrt{d} \end{array}

Now let $x\in \Omega$ and $\phi_n^i : \Omega_i \to \mathbb{R}$ given by $\phi_n^i(y) = \frac{\|D^au_n(x_1,x_2,\cdots,x_{i-1},y,x_{i+1},\cdots,x_d)- D^au_n(x_1,x_2,\cdots,x_{i-1},x_i,x_{i+1},\cdots,x_d)\|} {|y - x_i|}$. Here $\Omega_i \subset \mathbb{R}$ is an open ball around $x_i$ minus the point $x_i$ From the above we get that $\phi_n^i$ is uniformly-cauchy and hence converges uniformly in $\Omega_i$. Let $\lim_{n\to\infty} \phi_n^i(y)= \phi^i(y)$ in $\Omega_i$. Then, $\lim_{y \to x_i}\lim_{n \to \infty} \phi_n^i(y)= \lim_{n \to \infty} \lim_{y \to x_i} \phi_n^i(y)$ (i.e. the limits can be interchanged because of uniform convergence, see Rudin's PrinMathAnalysis Theorem 7.11) if $\lim_{y \to x_i} \phi_n^i(y)$ exists. In this case, this is true. Let $b=(a_1,a_2,\cdots,a_i+1,\cdots,a_n)$, then we get using recursion, \[D^bu(x)= \lim_{n\to \infty} D^{b}u_n(x)\]

Ascoli-Arzela Theorem

The content of this blog is almost a copy of the Wikipedia page on this topic. The proof given in Wikipedia is for real intervals. Here, a general compact set is considered. The sketch of the proof for general compact sets is already given in the Wikipedia page. This is merely a completion for the sake of reference.

Let $X$ be a compact space and $C(X;\mathbb{R}^n)$ be the set of continuous functions from $X$ to $\mathbb{R}^n$. Let $F \subset C(X;\mathbb{R}^n)$ be such that it satisfies the following two properties

• Equicontinuity : For each $\epsilon > 0$, for all $x\in X$, there exists a neighbourhood $U_x$ such that $\|f(y) - f(x)\| < \epsilon$ for all $y \in U_x$ and all $f \in F$
• Pointwise Boundedness : For each $x\in X$, $\sup\{\|f(x)\| : f \in F\} < \infty$

Then $\bar{F}$ is compact. (Here $C(X;\mathbb{R}^n)$ is given the metric $\|f\| = \sup\{\|f(x)\| : x \in X\}$

Let $\epsilon_n = \frac{1}{2^n}$. For each $x$, choose $U_x$ (from the equicontinuity of $\mathcal{F}$) such that the oscillation of any function in $\mathcal{F}$ is less than $\epsilon_1$. As $U_x$ form an open cover of $X$ and since $X$ is compact there exists a finite subcover which covers $X$. Denote this cover by $U_{x_{11}},U_{x_{12}},\cdots,U_{x_{1N_1}}$. With this process, we obtain for every $n$ a finite open cover $U_{x_{n1}},U_{x_{n2}},\cdots,U_{x_{nN_n}}$.

Rename the points $x_{11},x_{12},\cdots,x_{1N_1},x_{21}\cdots,x_{2N_2}\cdots$ as $x_1,x_2,\cdots$

Let $\{f_n\}$ be a sequence in $\mathcal{F}$. We want to show that there exists a sub-sequence which converges uniformly. As the space $\bar{\mathcal{F}}$ is a metric space (with the metric mentioned above) we get that it is compact.

Step 1 : As $\{\|f_n(x_1)\|\}$ is bounded, there exists a sub-sequence $\{f_{n_1}\}$ such that $f_{n_1}(x_1)$ converges. Now, we can choose a sub-sequence $f_{n_2}$ of $f_{n_1}$ such that $f_{n_2}(x_2)$ converges. This process is repeated ad-infinitum. Now, the "diagonal" sequence whose $m$th term is $m$th term in the $m$th subsequence $f_{n_m}$ is chosen and denoted by $f_m$. By construction, $f_m(x_i)$ converges for all $i$. This seems to be the central idea. For the next steps fix $l$.

Step 2: From the above, we know that for each $x_k$, there exists an integer $N(\epsilon,x_k)$ such that $\|f_n(x_k) - f_m(x_k)| < \epsilon_l$ for all $n,m > N(\epsilon_l,x_k)$.

Step 3 : Clearly, for $K=\sum_{i=1}^l N_i$ each open set $U_{lj}$, $1 \leq j \leq N_l$, contains at-least one point $x_k$ with $1 \leq k \leq K$

Step 4: For any $x \in X$, there exist $j,k$ such that $x \in U_{lj}, x_k \in U_{lj}$ where $1 \leq j \leq N_l$. For this $k$, \[\|f_n(x) - f_m(x)\| \leq \|f_n(x)-f_n(x_k)\| + \|f_n(x_k) - f_m(x_k)\| + \|f_m(x_k) - f_m(x)\| < 3\epsilon_l\] for all $n,m > \max(N(\epsilon,x_1),N(\epsilon,x_2),\cdots,N(\epsilon,x_K))$. Therefore, the sequence is uniformly-cauchy and hence converges to a continuous function $g \in C(X;\mathbb{R}^n)$. It is obvious that $g \in \bar{\mathcal{F}}$