Arrange the following functions in increasing order of growth rate: $n^2$,$(5/4)^n$, $logn*logn$, $log(n!)$
I think this is the order but I have problem with proving it:
$logn*logn<log(n!)<(5/4)^n<n^2$
I would like to see if my arrangement is correct or not and how to prove it
Your arrangement is not correct. Also, using inequalities are not the best way to represent growth - you could do it with big-O-notation, but I'll rather write it out.
$\log n\cdot \log n$ is slower than $\log(n!)$, slower than $n^2$ slower than $(5/4)^n$.
To prove it, it is sufficient to prove inequalities on their differential quotients. Thus, we think of all the functions as functions on $\mathbb{R}_{\geq 1}$, so let's replace $n$ by $x$, and we want to prove
$$\frac{d}{dx}\log(x)^2 \leq \frac{d}{dx}\log(\Gamma(x+1)) \leq \frac{d}{dx}x^2 \leq \frac{d}{dx}\left(\frac54\right)^x ,$$
holding for all $x\geq N$ for some $N\in\mathbb{N}$.
Notice that I replaced $x!$ by $\Gamma(x+1)$, the Gamma function on $x+1$. This is necessary to extend the factorial function to the reals - it holds that $n!=\Gamma(n+1)$ for all $n\in\mathbb{N}$, so it's just a continuation.
Using usual differentiation rules, we get
$$\frac{d}{dx}\log(x)^2 = 2\frac{\log(x)}{x} ,\qquad \frac{d}{dx}\log(\Gamma(x+1))=\frac{\Gamma'(x+1)}{\Gamma(x+1)} ,\\ \frac{d}{dx}x^2 = 2x, \qquad \frac{d}{dx}\left(\frac54\right)^x =\log(5/4)\left(\frac54\right)^x.$$
Now, $\frac{\Gamma'(x+1)}{\Gamma(x+1)} = \psi(x+1)$ is the digamma function, and it is known that $\psi(x+1)\sim \log(x)-\frac{1}{2x}$. Also, it is clear that $2\frac{\log(x)}{x}$ goes to zero for $x\to\infty$. Hence the first inequality should be clear, and I hope you are familiar with $\log(x)\leq x$ for $x\geq 1$, giving you the second inequality. For the final one, note that $\log(5/4)\left(\frac54\right)^{30}\approx 180$, while $2\cdot 30=60$, and the exponential functions grows with a factor of $5/4$ every time you increase $x$ by $1$, while the linear functions only grows by adding $2$.
This is of course not a completely formal proof of the final inequality, but the fact that exponential functions grows faster than polynomial functions is a fundamental fact, see How to prove that exponential grows faster than polynomial? for a rigourous proof.
