From the text I'm using:
Given $E[X|Y]=Y, E[Y|X]=X$ for $X,Y \in L^1(\mathcal A)$, show $X=Y$ a.s. by utilizing a bounded function $h(x)$. Hint: compute $E[(X-Y)(h(X)-h(Y))]$.
(Assuming probability space $(\Omega, \mathcal A, P)$.)
It's easy to show that $E[X]=E[Y]$ using iterated expectation/tower property, but I haven't a clue as to how $h(x)$ can help take the next step to show $X=Y$. Any ideas?
If we knew that $ X, Y $ were $ L^2 $, we could prove this by taking $ h $ to be the identity and noting that
$$ \mathbb E[(X-Y)^2] = \mathbb E[X^2] + \mathbb E[Y^2] - 2 \mathbb E[XY] $$
We have that $ \mathbb E[XY] = \mathbb E[Y \cdot \mathbb E[X | Y]] = \mathbb E[Y^2] $, and by symmetry we also have that $ \mathbb E[XY] = \mathbb E[X^2] $. All these equalities are enough to infer that $ \mathbb E[(X-Y)^2] = 0 $ and therefore $ X - Y $ vanishes almost surely, i.e. $ X = Y $ almost surely.
We can deduce the claim for the $ L^1 $ case from this case by using standard approximation methods: we have that $ X \textbf 1_{|X| < a} $ and $ Y \textbf 1_{|Y| < a} $ are $ L^2 $ for $ a \in \mathbb R^{> 0} $. This implies that $ X, Y $ must be equal almost surely whenever both of them are finite. Since they are also almost surely finite by virtue of being $ L^1 $, this argument proves the claim for the $ L^1 $ case.
If we want to use the given hint, we can argue that in general we will have
$$ \mathbb E[(X-Y)(h(X) - h(Y)] = \mathbb E[X h(X)] + \mathbb E[Y h(Y)] - \mathbb E[X h(Y)] - \mathbb E[Y h(X)] $$
for any bounded $ h $ and by a similar argument to the above case we know that this sum vanishes for all such $ h $. We then just choose $ h $ to be any strictly increasing bounded function, say $ h(X) = \tanh(X) $. If $ h $ is strictly increasing then $ (X-Y)(h(X) - h(Y)) \geq 0 $ with equality holding if and only if $ X = Y $, so we just conclude in a similar way to the above $ L^2 $ case.
