This can be done using a variation of the construction of the algebraic closure of a finite field $\mathbb{F}_q$. Namely, instead of adjoining roots of all irreducible polynomials we can fix a prime $\ell$ and adjoin only roots of irreducible polynomials of degree a power of $\ell$. Equivalently, in the algebraic closure we consider the increasing union of the subfields
$$\mathbb{F}_q \subset \mathbb{F}_{q^{\ell}} \subset \mathbb{F}_{q^{\ell^2}} \subset \dots .$$
This is the maximal $\ell$-extension of $\mathbb{F}_q$, and it is infinite and algebraic but not algebraically closed because it does not contain roots of irreducible polynomials of any other degrees except powers of $\ell$. (This is an exercise: more generally, $\mathbb{F}_{q^n}$ consists of $\alpha \in \overline{\mathbb{F}_q}$ whose minimal polynomials have degree dividing $n$.)
In Galois theory terms this is the subfield of $\overline{\mathbb{F}_q}$ corresponding to the quotient map $\widehat{\mathbb{Z}} \to \mathbb{Z}_{\ell}$, where $\widehat{\mathbb{Z}} \cong \text{Gal}(\overline{\mathbb{F}_q}/\mathbb{F}_q)$ is the profinite integers and $\mathbb{Z}_{\ell}$ is the $\ell$-adic integers.