The solution has been discussed so I'll just try to address your other concerns.
The expression to be simplified contains certain suggestions to the substitution.
The $x$ and $\sqrt{1-x^2}$ have a sum of squares of 1. This is reminiscent of either the $\sin x$ function or $ \cos x$ functions.
So we try to substitute say, $x=\sin\alpha$.
With inverse trigonometric expressions however this gets a little tricky.
You have to ensure that your substitution satisfies the domain. For example, here plugging $x=\sin\alpha$ would be invalid if $x$ could take all real values because the range of $\sin x$ is $[-1,1]$. We can do that here because $\sqrt{1-x^2}$ requires $\mid x\mid <1$ .
$\mathrm{sin}\alpha$ is a many-one function. In fact for any $\alpha$ you take $n\pi +(-1)^n\alpha$ gives the same value. So if your simplified expression contains $\alpha$ you would get an infinite number of values for the resultant function . But the $\mathrm{arccos}$ function is single valued. So we have to restrict the domain of $\alpha$ to ensure that $\sin\alpha$ takes all values of $x$ and that each $x$ corresponds to a single $\alpha$.(Establish a bijection between $x$ and $\alpha$ so to speak)
The easiest way to do this usually is to assume $\alpha =\sin^{-1}(x)$ this forces $\alpha\in [-\tfrac{\pi}{2},\tfrac{\pi}{2}]$.
Another note other people missed on is that $\sqrt{1-sin^2\alpha}$ is $\mid\cos\alpha\mid$. It is only when you consider our restriction on $\alpha$ that you can justify that $ \cos\alpha$ is positive in $\in [-\tfrac{\pi}{2},\tfrac{\pi}{2}]$.
Finally, for inverse trig functions, while:
$$\mathrm{trig}(\mathrm{trig}^{-1}x)=x$$
Is true but:
$$\mathrm{trig}^{-1}(\mathrm{trig}(x))=x$$
This is true only when $x$ lies in the Principal value branch of $\mathrm{trig}$(Denotes any of the six functions.
So, once you have simplified the function to:
$\cos^{-1}(\cos(\alpha-\tfrac{\pi}{4}))$
You have to look at what your value of $\alpha$ is before you cancel cos inverse and cos.
Look up the graph for $\cos^{-1}(\cos x)$ and you'll notice that:
$$\cos^{-1}(\cos x)= x ;0\leq x\leq \pi$$
$$\cos^{-1}(\cos x)= -x ;-\pi\leq x\leq 0$$
So,
$$\cos^{-1}(\cos (\alpha-\tfrac{\pi}{4}))= \alpha-\tfrac{\pi}{4};\tfrac{\pi}{4}\leq \alpha\leq \tfrac{5\pi}{4}$$
$$\cos^{-1}(\cos (\alpha-\tfrac{\pi}{4}))= \tfrac{\pi}{4}-\alpha ; -\tfrac{3\pi}{4}\pi\leq \alpha\leq \tfrac{\pi}{4}$$
Checking the appropriate ranges on $x$ corresponding to $\alpha$ gives $x\geq \tfrac{1}{\sqrt{2}}$ and $x\leq \tfrac{1}{\sqrt{2}}$ for the first and second case respectively.
Finally a note on choosing substitutions.
You should try to find the substitution that has an identity most closely resembling the given expressions.
Examples:(Try to simplify the expressions and identify the corresponding identity.)
- $x, \sqrt{x^2-a^2}$ , use $x=a\sec\alpha$ or $\csc\alpha$.
- $x, \sqrt{a^2+x^2}$, use $x=a\tan\alpha$
- $\sqrt{1-x},\sqrt{1+x}$, use $x=a\cos2\alpha$
- $\sqrt{a-x}{x-b}, \sqrt{\dfrac{a-x}{x-b}}$ use $x=a\sin^2\theta +b\cos^2 \theta$. (Slightly esoteric, rare use)