Observation in a random sample falling outside of the distribution?

Question

I am wondering if an observation in a random sample is allowed to fall outside of the distribution.

What prompted this question is the example linked in this answer. The example is on page 13 in the pdf referenced in that answer. I will summarize my question/the example here, but see the reference if I have not explained well.

Say we have the uniform distribution on $[0,\theta],\ U[0,\theta]$ on the interval $[0,\theta]$. In the example, the likelihood function is written $$ \prod_{i=1}^n f(X_i \vert \theta) = \frac{1}{\theta^n}I(X_1,\dots,X_n \in [0,\theta]) = \frac{1}{\theta^n}I(\max (X_1,\dots,X_n) \leq \theta) $$

Where $I$ is the indicator function. The example notes that they write the likelihood function using an indicator function because:

"What the indicator above means is that the likelihood will be equal to $0$ if at least one of the factors is $0$ and this will happen if at least one observation $X_i$ will fall outside of the ’allowed’ interval $[0, \theta]$."

I don't see how an observation can fall outside the allowed interval.

I realize that the uniform distribution is technically defined everywhere, but has probability zero outside the given interval. Since all points outside the interval have probability zero, I don't see how a draw can be from outside the interval.

For easier access, I am linking the pdf with the example here as well, see the bottom of page 13

Answer 1

I think that sentence is to define the indicator function (it's a function by itself), which is independent to the distribution.

Without any knowledge about the distribution, the indicator function is still valid i.e. it returns value $0$ or $1$ base on the value of $X_i$ for a defined interval.

Now that you know about the uniform distribution, such that $X_i$ has $0$ probability to be outside the interval, we can use the indicator function as a way to rewrite the likelihood function and introduce $Max(X_1,...,X_n)$ which is useful for the next part.

The fact that the we use the indicator function does not interfere with the hypothesis that the distribution is uniform, in which case we know for sure that all values of $X_i$ will be inside the interval. But keep in mind that we are doing statistics, in the real world, the distribution is not "given". Your question is more philosophical than technical. It is like saying, there is a probability for A to happen, that probability is equal to $0$. The sentence is technically not self-contradicting, even then in common language people will find it a little bit confusing.

Answer 2

So I guess I was thinking about the problem in the wrong way, and thus asked a question that perhaps doesn't make much sense. To answer what I asked, an observation cannot fall outside the allowed interval, as stated in a comment above.

My misconception (which I post in case anybody else has the same misconception), is that I was thinking of a uniform distribution, $U\sim [0,\theta]$ from which we are taking draws $x_1,x_2,\dots ,x_n$, in which case all of the $x_i$'s would need to be in $[0,\theta]$. I was then thinking that the likelihood was talking about the likelihood of taking draws of a $U\sim [0,\theta]$ distribution, but the likelihood is really talking about taking draws from a $U\sim [0,\theta']$ distribution, $\theta'$ and estimate of $\theta$.

To be long winded, what is happening in the problem is kind of like this: The distribution is $U\sim[0,\Theta]$ (perhaps even say $U\sim [\Theta_1,\Theta_2]$ if you want more generality). Therefore, all the $x_i$'s are in $[0,\Theta]$. We do not know $\Theta$, though. What the likelihood function is doing is saying "what is the likelihood" that $\theta = \Theta$. Well, since $x_i \ in [0,\Theta]$, if there exists and $x_j,\ j\leq n$ such that $x_j \not\in [0,\theta_1]$ then the likelihood that $\theta_1 = \Theta $ is $0$ because $x_j \in [0,\Theta]$. That is, the observations cannot fall outside of the actual distribution, so if they are falling outside of an estimate of the distribution, that estimate must have a likelihood of $0$.

In short: when the problem mentions an observation $x_i$ falling outside of the "allowed interval", it is not talking about the actual distribution (the actual interval), but the estimated distribution/interval $[0,\theta]$.

A little add on: The indicator function here allows us to represent/deal with the fact that which outcomes are possible depends on the parameter we are estimating (whereas, say for the normal distribution if we estimate the variance or mean, different means or variances do not make observing something like $10000$ impossible -- they simply make it more or less likely).