How to calculate the pdf (and the Expectation) of $f(X, Y)$ when X and Y are continuous random variables and Y depends on X?
For example:$$ X∼\operatorname{Uniform}(0,1) \\ Y∼\operatorname{Uniform}(X, 1) \\ f(X, Y) = (Y-X) ^ 2$$
How to calculate the pdf (and the Expectation) of $f(X, Y)$ when X and Y are continuous random variables and Y depends on X?
For example:$$ X∼\operatorname{Uniform}(0,1) \\ Y∼\operatorname{Uniform}(X, 1) \\ f(X, Y) = (Y-X) ^ 2$$
What you define with $Y$ actually is $Y|X$ thus you joint density is
$$f_{XY}(x,y)=\frac{1}{1-x}\cdot\mathbb{1}_{0<x<y<1}$$
now you can use standard technics to derive what you want...
for example, if you want to calculate the law of $Z=(Y-X)^2$ you can use the definition of CDF and calculating
$$\mathbb{P}[Z>z]=\int_0^{1-\sqrt{z}}\frac{1}{1-x}\left[\int_{x+\sqrt{z}}^1dy \right]dx=\dots=1-\sqrt{z}+\sqrt{z}\log\sqrt{z}$$
Thus
$$F_Z(z)=\sqrt{z}(1-\log\sqrt{z})$$
and derivating you get
$$f_Z(z)=-\frac{\log\sqrt{z}}{2\sqrt{z}}\cdot\mathbb{1}_{(0;1]}(z)$$
$E(Y-X)^{2}=E(E(Y-X)^{2}|X)=E(\frac 1 {1-X}\int_x^{1}(y-X)^{2}dy=\int_0^{1} \frac 1 {1-x}\int_x^{1}(y-x)^{2}dydx$. Similar idea for any function $f(X,Y)$.
Can you explain about limits of the integrals?
It is easy to use the definition of $P(Z>z)$.
$$\mathbb{P}[Z>z]=\mathbb{P}[(Y-X)^2>z]=\mathbb{P}[|Y-X|>\sqrt{z}]$$
that is the integral of $f(x,y)$ over the purple region below