From my interpretation of the answers and comments to this question, one gets countable models $L_\alpha$ of second-order arithmetic $Z_2$, just if "$\alpha$ is $\Sigma_n$-admissible for all $n$, and $L_\alpha$ is locally countable in the constructible hierarchy (meaning that each element of $L_\alpha$ can be injected into $\omega$ via a function in $L_\alpha$)".
Are there large countable ordinals $\alpha$, so that $L_\alpha$ is a model of higher order arithmetic?
Throughout, $\alpha$ will be an infinite countable ordinal.
Yes in a way for essentially trivial reasons; however, to be precise it's not $L_\alpha$ itself that you're asking about. E.g. $L_\alpha$ is never a model of even second-order arithmetic, since the languages are different.
The correct focus-shift is the following. Fix some countable ordinal $\gamma$ (e.g. $\gamma=\omega$). By downward Lowenheim-Skolem, let $M$ be a countable elementary submodel of $L_{\beth_\gamma^L}$ with $\gamma\in M$. By condensation there is a countable ordinal $\alpha$ with $L_\alpha\cong M$. Now the "$\gamma$th-order arithmetic structure" associated to $L_\alpha$ will, in every first-order-expressible respect, mirror that associated to $L_{\beth_\gamma^L}$ itself, and so $L_\alpha$'s version of $\gamma$th-order arithmetic is as good as that proved by $\mathsf{ZFC}$ in the first place.
In a bit more detail, here's what I mean by the $\gamma$th-order arithmetic structure associated to a level of $L$.
Fix a countable infinite ordinal $\alpha$. For each ordinal $\beta<\alpha$ we define a structure $F_{\beta,\alpha}$ as follows:
$F_{0,\alpha}$ is just the standard model of arithmetic, $(\mathbb{N};+,\times,0,1,<)$.
$F_{\lambda,\alpha}$ is the union in the appropriate sense of the $F_{\beta,\alpha}$s for $\beta<\lambda$ (I say "appropriate sense" since the languages are increasing).
$F_{\beta+1,\alpha}$ is the expansion of $F_{\beta,\alpha}$ by adding a new sort with underlying set $\mathsf{Func}(F_{\beta,\alpha}, \mathbb{N})\cap L_\alpha$. Here "$\mathsf{Func}(X,Y)$" is the set of functions from $X$ to $Y$. We moreover add to the language of the structure a binary function symbol for function application, to apply elements of our new sort to elements of $F_{\beta,\alpha}$ in the obvious way.
Note that a previous version of this answer used a much smaller $\alpha$, namely $L_\alpha\prec L_{\omega_1}$. This is actually quite unsatisfactory, since such an $L_\alpha$ doesn't think $\mathbb{R}$ exists as a set! My bad.
