Learning random variables as a high-schooler is quite easy, even in my first course of Probability theory, a random variable $X$ was just a map from $\Omega \to \RR$. Everything was nice and easy until the time $X$ was taking countably many values, but then there was a need of $\sigma-$algebras. There was no looking back; we had reached a point where everything required a proof. So today, I’ll save everyone from the seemingly unexpected terror of random variables.

Just to be more precise going forward, we are going to prove measurability of events(subsets of $\Omega$) and functions from $\Omega \to \ol\RR$ where $\ol\RR$ is the set of extended reals.

Why the Jargon?

This deserves a blog post on its own, but my professor at ISI Kolkata(Dr. Arnab Chakraborty) who taught us Probability I did a wonderful job giving us the idea as to why you need to consider the $\sigma-$algebras. I don’t think I have much to add to that explanation, so I would recommend everyone to read our class webpage where he has discussed the case when $\Omega$ is uncountable and the construction by Vitali.

Prerequisites

• Definition of a Measure and Measurable Space
• Sigma Algebras
• Construction of Vitali’s set(mostly to get a feel for the need of sigma algebras)

The Updated Definitions

Anyone learning axiomatic probability-theory, first learns about the probability space only being about the sample space $\Omega$ and the probability function $\P\colon \mathcal{P}(\Omega) \to [0,1]$. But the moment you tackle uncountable probability spaces, then one must resort to the use of the sigma-field $\mathcal{F}$, to filter out the “good sets” and work with them.

With the above new definition of the probability space, what could possibly go wrong with random variables? Well, what if you want to compute $\P(X \in B)$ where $B \subseteq \RR$? Recall that

If we really want to compute the probability of this event, then we need to make sure that this event $[X \in B]$(we will also denote this by $X^{-1}(B)$) lies in $\mathcal{F}$. If you saw the construction by Vitalli from the web-page which I shared above, then you basically saw that you cannot really assign a uniform probability over $[0,1]$. Thus, rather than computing the probability of events $[X \in B]$ where $B \subseteq \RR$, we restrict $B$ to lie in $\mathcal{B}(\RR)$. We are now ready to define the notion of an ordinary random variable over the measurable space $(\Omega, \mathcal{F})$.

We have only defined an ordinary random variable where $X$ can’t take values like $\pm \infty$. Often times, in cases of markov chains and other stochastic processes, we sometimes denote the return time of a particle by a random variable $X$. If the particle never returns then we set $X(\omega) = \infty$ in that case, so we will now define what is an extended random variable. From here, we shall now denote $\overline{\RR} = \mathbb{R} \cup \{\pm \infty \}$.

Its a perfectly valid question to ask how does the borel sigma field over $\overline\RR$ looks like… For some beginners, they might be unfamiliar with the notion of borel-sigma algebra. Let’s talk about it briefly before moving onto the actual results to be proven for random variables.

Borel Sigma Algebras

Due to the construction by Vitalli, we know that not all subsets of $\RR$ can be assigned a probability with the function $\P$. Thus we resort to what we call the Borel $\sigma-$algebra over $\RR$. Consider the usual metric topology¹ on $\RR$ where we let $\tau$ denote the set of all open sets² of $\RR$. Then we define

where $\sigma(\tau)$ means the smallest $\sigma-$algebra containing all the elements of $\tau$. Although, dealing with all open intervals is quite cumbersome when you have to prove results. Luckily, $\RR$ is quite forgiving and we have a wonderful result, which is good enough that we state it as a theorem.

With this, we have the luxury to define the borel-sigma algebra way more freely. We will now state a really really important lemma. If you really want to prove this by yourself, then you can find an important lemma here in the appendix.

Outline of Proof: This one requires some practice with sigma-algebras, which is not what we are doing in this blog. Note that 2nd point is true is by definition. By the theorem stated above, 1st point becomes true as well. For the other points, you need to take some countable union of open sets to generate an arbitrary closed interval. For example, if you want to make the closed interval $[a,b]$ from open sets, then consider

So, you can construct any closed interval from an open intervals. The other parts can be done similarly. $\square$

If we want to talk about $\mathcal B (\ol\RR)$, then we need to characterize the open sets of $\ol \RR$, which is more or less a question of topology. If someone is interested in characterizing the open sets of $\ol\RR$, then you can refer the appendix one possible approach. For this blog post, we will work a simpler definition of $\mathcal B (\ol\RR)$, which is the following.

Even with all this, I will give an alternate definition to the borel sigma algebra over $\ol\RR$ in the appendix over extended reals. Anyways, we are now ready to head to the more important results for random variables.

Important Results

We will start by proving the following theorem.

Before starting the proof, we recall the collection $B = \{(-\infty, x]: x \in \RR \}$ generates the entire $\mathcal{B}(\RR)$ thus $\mathcal{B}(\RR) = \sigma(B)$. Thus to show that any map $f\colon \Omega \to \RR$ is a random variable, it’s enough to show that $f^{-1}\left((-\infty, x] \right)\in \mathcal{F}$. If you want a rigorous proof as to why only checking it for generators suffices, then you will find it in the appendix.

Proof to part 1: We shall first show that $aX_1$ is a random variable. If $a = 0$, then

If $x < 0$, then then $(aX_1)^{-1}$ is precisely $\emptyset \in \mathcal F$. If $x > 0$, then $(aX_1)^{-1}$ is precisely $\Omega \in \mathcal{F}$. This shows that $aX_1^{-1}(-\infty, x] \in \mathcal F$ for all $x \in \RR$. This case is done.

For the next case, assume $a > 0$. Once again, consider $(aX_1)^{-1}(-\infty, x]$ which is precisely

Since $X_1$ is a random variable, $X_1^{-1}(-\infty, x] \in \mathcal F$ completing the proof for this case as well.

The last case is slightly tricky. This time dividing by $a$ would change the sign of equality. So, we will work with another class of generators, that is $\{[x, \infty): x \in \RR \}$. Consider

since $X$ is a random variable and $B = \{(-\infty, x]: x \in \RR \}$ is also a generator. The proof is now complete. $\square$

Proof to part 2: We will stick with class of generators $B = \{(-\infty, x]: x \in \RR \}$. Consider

If $x < 0$, then $|X_1|^{-1} = \emptyset \in \mathcal F$. If $x \ge 0$, then

Even in this case, if you consider the class of generators only consisting of closed sets, then one would realize $|X_1|^{-1} \in \mathcal F$ in this case as well. The proof is complete now. $\square$

Counter Example to part 3: Consider $\Omega = \{ 0, 1\}$ and $\mathcal F = \left\{\{\emptyset \}, \Omega \right\}$ and consider a map $Y\colon \Omega \to \RR$ such that $Y(0) = 1$ and $Y(1) = -1$. In this case, $|Y|$ is indeed a random variable, but $Y$ is not. To see this, note that $\{ 1\} \in \mathcal B(\RR)$ and $Y^{-1}(\{ 1\}) = \{0\} \not\in \mathcal F.$

Proof to part 4: We will first prove $\min(X_1, X_2)$ is a random variable. Consider

But for any two reals $a,b$, $\min(a,b) \le x \iff a \le x$ or $b \le x$. This gives

Since $X_1, X_2$ are random variables, both the events in the union above lie in $\mathcal F$, thus their intersection would lie there as well. We omit the prove for $\max(X_1, X_2)$, but the idea to prove that is very very similar.

Let $Y = X_1 + X_2$. We shall now prove that $Y$ is a random variable. This one truly tricky and uses a typical analysis trick which exploits that $\RR$ is separable³. For this proof, we will use a different class of generators, namely $\{ (x, \infty) : x \in \RR \}$. Now, consider

Since $\QQ$ is countable, let $\{q_i\}_{i = 1}^\infty$ be an enumeration of $\QQ$. We have the following claim.

The proof to the claim is extremely simple. If $\omega \in Y^{-1}(x, \infty)$, then we know

Since these two quantities are different, by density of $\QQ$ over $\RR$, we know there must exist some index $j$ such that

which immediately gives $\omega \in \bigcup_{i = 1}^\infty \{\omega: X_1(\omega) > q_i\} \cap \{\omega: X_2(\omega) >x - q_i \}.$ The other direction is rather trivial, so we shall skip it.

We can now finish the proof, since $X_1$ and $X_2$ are random variables and $\{ (x, \infty) : x \in \RR \}$ is a class of generators, we get that both events $\{ \omega: X_1(\omega) > q_i \}$ and $\{\omega: X_2(\omega) > x - q_i \}$ both lie in $\mathcal F$ for all $i \in \NN$. The intersection would lie in $\mathcal F$ and so would a countable intersection! The proof is now complete. $\square$

Proof to part 5: Define $Y\colon \Omega \to \ol\RR$ such that $Y(\omega) = \sup_n X_n(\omega)$. We shall use the generator of $\ol\RR$, which is the collection $\{ [-\infty, x]: x \in \RR\}$. Thus, consider

Now, recall that $\sup_n b_n \le x \iff b_n \le x$ for all $n \in \NN$ where $b_n$ is a sequence of reals. With this, we get the following

Since $X_n$ are random variables(which means they extended random variables as well), we know that $\{\omega: X_n(\omega) \le x\} \in \mathcal F$ for $n \in \NN$. So does their countable union! This shows that $Y$ is an extended random variable as desired.

We’ll give a small sketch for the infimum part, this time consider the collection $\{ [x, \infty]: x \in \RR\}$ which generates $\mathcal B (\RR)$. This time use the fact that $\inf b_n \ge x \iff b_n \ge x$ for all $x \in \RR$ where $b_n$ is a real sequence. This concludes the proof. $\square$

Proof to part 6: We shall only prove it for $\limsup X_n$. Once again define $Y\colon \Omega \to \ol\RR$ given by $\displaystyle Y(\omega) = \limsup_{n \to \infty} X_n(\omega)$. Define another sequence of functions $Z_n\colon \Omega \to \ol\RR$ given $Z_n(\omega) = \sup_{k \ge n} X_k(\omega)$ for all $n \in \NN$. By definition of $\limsup$, we know

We shall prove that $Z_n$ is an extended random variable, then by the result proven above, we must have that $Y = \inf Z_n$ is also a random variable. One needs to be careful; in the previous proof we assumed that $X_i$‘s are real valued random variables, then $\inf X_n$ is an extended random variable. In our case, we have a sequence of extended random variables $Z_n$ and then we are concluding $\inf Z_n$ is also a random variable. The proof of this completely the same as above and can be worked out quite easily, so we’ll skip it here.

This completes the proof to part 6.

Proof to part 7: We’ll use our favourite class of generators, namely $\{ (-\infty, x]: x \in \RR \}$. Define $Y\colon \Omega \to \RR$ such that $Y(\omega) = X_1\bm{1}_A(\omega) + X_2\bm{1}_{A^c}(\omega)$ for all $\omega \in \Omega$. Once again, we want to show $Y$ is a random variable.

The idea is simple, just check cases whether $\omega \in A$ or $\omega \not\in A$. This yields

The final trick here is to notice that

Clearly, $Y^{-1}$ is a union of two measurable sets. This concludes the proof. $\square$

This concludes a series of very important results, which are pretty much used everytime while you are dealing with random variables. Here are a couple of more important results which are must to know. As a matter of fact, this is quite an important theorem which I previously used in my blog post regarding the Ky-Fan Metric. The details and the reference of this theorem is mentioned in the blog post now!

We will now begin with the proofs.

Proof to 2nd part: This one is easy, just recall the cauchy criteria for convergence over $\RR$. We say a real sequence $a_n$ is cauchy iff for every $k \in \NN$ there exists some $N \in \NN$ such that $|a_m - a_n| < \frac 1 k$ for $m,n > N$. We know that $\lim a_n$ exists finitely iff $a_n$ is cauchy. With this criteria, we may write

Well, since $|X_m - X_n|$ is a random variable for all $m,n \in \NN$, thus the set $\left\{\omega : |X_m(\omega) - X_n(\omega)| < \frac 1 k \right\}$ lies in $\mathcal{F}$. Under countable union and intersection, the final set remains in $\mathcal{F}$ completing the proof. $\square$

Proof to part 1: We will take the help of part 2 this time. We have already resolved the case when the limit lies in $\RR$, so we need to handle the case when limit is $\pm \infty$. We may write

The first set of the union lies in $\mathcal F$. For the second part, we may partition the set into cases where limit is $\infty$ or $-\infty$. We will only give a proof for limit being infinity here. Its easy, just observe

Now, $\{\omega \in \Omega: X_n(\omega) > M \} \in \mathcal F$ since $X_n$ is a random variable for all $n \in \NN$. After taking countable union and intersection, we still get a set a in $\mathcal{F}$ completing the proof. $\square$

Sketch to proof of part 3: Consider $X^{-1}$ which is

The above can be re-written as

It is easy to see that $\{\omega: 0 \le x \} \cap A^c \in \mathcal{F}$. It suffices to show that $\{\omega \in A: \lim_{n \to \infty} X_n(\omega) > x \} \in \mathcal F$. Now, $\lim_{n \to \infty} X_n(\omega) > x \iff X_n(\omega) > x$ for all $n > N$ where $N \in \NN$. This means,

This concludes the proof since $\{\omega: X_n(\omega) > x \,\,\text{for }n> N\}$ for all $n \in \NN$ since $X_n$ is a random variable. $\square$

We will skip the proof to part 4, since the blog became too repetitive with the same idea. The reader is encouraged to workout the details if they are feeling uncomfortable. To end the blog, I’ll present the final a very significant result, which answers questions like, if $X$ is a random variable then should $\sin(X)$ should be a random variable? What about simpler thing like $X^2 + 4X$?

Indeed, a natural generalization follows. Instead of saying $f$ continuous, if you imposed the condition that $f$ was a measurable function from $\RR \to \RR$, i.e., for any Borel set $B$, $f^{-1}(B) \in \mathcal{B}(\RR)$, even then the conclusion holds true.

Product of Two Random Variables

If $X,Y$ are two random variables, then $X+Y$ is also a random variable. By the above theorem, $(X+Y)^2 = X^2 + 2XY + Y^2$ is also a random variable. Also, $X^2$ and $Y^2$ are random variables. Thus,

which concludes that $XY$ must be a random variable as well.

Appendix

Two Essential Lemmas for Sigma Fields

The following lemma is quite extensively used(honestly quite easy to prove as well) to prove if two collections $\mathcal{C}_1$ and $\mathcal C_2$ generate the same $\sigma-$algebra or not. This result is exclusively used in proving this lemma where we have discussed about several generators of $\mathcal B(\RR)$.

The following theorem allows us to prove that a certain map from $\Omega \to \RR$ is a random variable. The special thing is, we can choose whatever generator we want, and it would still suffice. A version of the theorem for general measure spaces can be found here on page 24.

Conventions on Extended Reals

This section is rather short. But in case of $\ol\RR$, we define

for our probability purposes. The reason for this, is to define expected value. If an extended random variable $X$ takes values $\pm\infty$ with probability $0$, then we essentially want to treat it as an orindary random variable over $\RR$. For this to truly hold, you need to define the multiplication of infinity with $0$ to be $0$.

Borel Sigma Algebra over Extended Reals

I’ll brief the idea as to how someone can topologically describe open sets of $\ol\RR$. If someone wants a shortcut, then you can simply define $\mathcal{B}(\ol\RR) \coloneqq \sigma(D)$ where $D = \{[-\infty, a]: a \in \RR \}$. Life will be fine with definition.

We will construct a homeomorphism in a topological sense(this approach is Exercise 33, in Chapter 1 of A Probability Path by Sidney Resnick) from $[-1,1] \to \ol\RR$ given by the map

Thus in principle, what one can do it take all open sets in $\RR-$subspace topology of $[-1,1]$ and map it to a new open set in $\ol\RR$. This would generate all your open sets in $\ol\RR$. An interested reader can go and look for the details online(here’s a recommendation although, this one goes into measure theory), but we shall skip it since it requires a good knowledge of metric spaces and topologies induced by it.

Footnotes