Find $$\lim\limits_{n \to \infty} \left( \frac{n^{1/3}}{2} \text{arccos} \left(\dfrac1{\sqrt{1+\frac{4}{(k(n)-1)^2}}\sqrt{1+\frac{8}{(k(n)-1)^2}}} \right) \right).$$
where $k(n) = \dfrac{1}{12} (108n+12 \sqrt{768+81n^2})^{1/3}-\dfrac{4}{ (108n+12 \sqrt{768+81n^2})^{1/3}} + 1$.
Help me, please.
The idea is to use l'Hospital rule in a judicious way. Before applying l'Hospital rule, let us do some algebraic manipulations and get it into a reasonably manageable form.
We need $$\lim\limits_{n \to \infty} \left( \frac{n^{1/3}}{2} \text{arcsec} \left(\sqrt{1+\frac{4}{k(n)^2}}\sqrt{1+\frac{8}{k(n)^2}} \right) \right).$$
where $k(n) = \dfrac{1}{12} (108n+12 \sqrt{768+81n^2})^{1/3}-\dfrac{4}{ (108n+12 \sqrt{768+81n^2})^{1/3}}$.
Note that my $k(n)$ is the OP's $k(n) - 1$. Also, note that we have converted the problem from $\arccos$ to $\text{arcsec}$ since $$\arccos \left( \dfrac1x \right) = \text{arcsec}(x)$$ Let $$f(n) = \text{arcsec} \left(\sqrt{1+\frac{4}{k(n)^2}}\sqrt{1+\frac{8}{k(n)^2}} \right)$$ and $$g(n) = \dfrac{2}{n^{1/3}}$$ We want $$\lim_{n \rightarrow \infty} \dfrac{f(n)}{g(n)}$$ We shall use L'Hospital rule. But before we jump into using L'Hospital rule, we will massage the expression so that it can be handled with relative easy.
$$\frac{df}{dn} = \dfrac{df}{dk} \dfrac{dk}{dn}$$
Recall that $$\dfrac{d \text{arcsec}(x)}{dx} = \dfrac1{x\sqrt{x^2-1}}.$$ Hence, $$\dfrac{df}{dk} = - \dfrac{k(6k^2+32)}{\sqrt{k^2+4}\sqrt{k^2+8}(k^2+4)(k^2+8)} \dfrac{\sqrt{k^4+12k^2+32}}{\sqrt{3k^2+8}}$$
$$\dfrac{dk}{dn} = \dfrac{3^{1/3} \left(3 \sqrt{3} n + \sqrt{256 + 27n^2} \right)\left(8 \times 3^{1/3} + 2^{1/3} \left( 9n + \sqrt{768+81n^2} \right)^{2/3} \right)}{2 \times 2^{2/3} \sqrt{256+27n^2} \left( 9n + \sqrt{768+81n^2} \right)^{4/3}}$$
$$\dfrac{dg}{dn} = - \dfrac{2}{3n^{4/3}}$$
Now note that as $n \rightarrow \infty$, $k(n) \rightarrow \infty$. As $k \rightarrow \infty$, $$\dfrac{df}{dk} \sim - 2\sqrt{3} \dfrac1{k^2}$$ As $n \rightarrow \infty$, $$\dfrac{dk}{dn} \sim \dfrac16 \frac1{n^{2/3}}$$
Hence, as $n \rightarrow \infty$, $$\dfrac{df}{dn} \sim - \dfrac1{\sqrt{3}} \frac1{k^2n^{2/3}}$$
When we write $h(n) \sim l(n)$, we mean that $\displaystyle \lim_{n \rightarrow \infty} \dfrac{h(n)}{l(n)} = 1$ i.e. $h(n) = l(n) + E(n)$, where $\dfrac{E(n)}{l(n)} \rightarrow 0$.
Hence, to summarize $$\lim\limits_{n \to \infty} \left( \frac{n^{1/3}}{2} \text{arcsec} \left(\sqrt{1+\frac{4}{k(n)^2}}\sqrt{1+\frac{8}{k(n)^2}} \right) \right) = \lim_{n \rightarrow \infty} \dfrac{f(n)}{g(n)} = \lim_{n \rightarrow \infty} \dfrac{df/dn}{dg/dn}\\ = \dfrac{- \dfrac1{\sqrt{3}} \frac1{k^2n^{2/3}}}{- \dfrac{2}{3n^{4/3}}} = \lim_{n \rightarrow \infty} \dfrac{\sqrt{3}}{2} \dfrac{n^{2/3}}{k^2}$$
Now as $n \rightarrow \infty$, $k(n) \sim \dfrac1{12}(108n+108n)^{1/3} = \dfrac1{12} \times 6n^{1/3} = \dfrac{n^{1/3}}{2}$. Hence, $$k^2(n) \sim \dfrac{n^{2/3}}{4}$$
Hence, we finally get that $$\lim\limits_{n \to \infty} \left( \frac{n^{1/3}}{2} \text{arcsec} \left(\sqrt{1+\frac{4}{k(n)^2}}\sqrt{1+\frac{8}{k(n)^2}} \right) \right) = \lim_{n \rightarrow \infty} \dfrac{\sqrt{3}}{2} \dfrac{n^{2/3}}{k^2} = \dfrac{\sqrt{3}}{2} \times 4 = 2 \sqrt{3}$$
Hence, the limit is $2 \sqrt{3}$. The approximate value is $3.464$ which is what you have mentioned in your comments (though the answer should be $2\sqrt{3}$ and not $3 \sqrt{2}$)
EDIT As Peter Tamaroff rightly points out in the comment, if we are looking at the limit of the sequence, then we need to apply the Stolz Cesaro rule (the equivalent of l'Hospital's rule for sequences). However, if the function converges to a limit, then so will the sequence, which is the case here. However, note that the converse is not true in general. For instance, look here for a counterexample!. However, it is easier to apply l'Hospital than Stolz Cesaro, and if we find that the limit exists from l'Hospital rule, we are fine.
As $k$ is n-dependent first try to find limit $\lim\limits_{n \to \infty} (k_{n}-1) = \lim\limits_{n \to \infty}((\frac{1}{12} (108n+12 \sqrt{768+81n^2})^{1/3}-\frac{4}{ (108n+12 \sqrt{768+81n^2})^{1/3}}+1)-1)$ - it should make clear what is the limit of argument of $arccos$, next use fact that $\lim\limits_{n \to \infty}a_{n}b_{n}=\lim\limits_{n \to \infty}a_{n}\lim\limits_{n \to \infty}b_{n}$.
