This problem from book Dummit and Foote "Abstract algebra":
Exhibit nonnormal subgroup of $Q_8 \times Z_4.$
Let $q \in Q_8$ and $z \in Z_4.$ Let $N$ be nonnormal subgroup of $Q_8 \times Z_4 $. Then there exist $g \in Q_8 \times Z_4$ and $n=(q_n,z_n) \in N$ such that $g^{-1}n g = (q^{-1},z^{-1}) (q_n,z_n) (q,z)=(q^{-1}q_nq,z^{-1}z_nz) \notin N.$ In the other hand every subgroup of $Q_8$ and every subgroup of $Q_8$ are normal.
I get tripped up on this problem!
I'll use Goursat's lemma to give a slightly more technical and methodical answer, with hopes that this introduces techniques that can generalize to other similar situations. Goursat's lemma characterizes all subgroups of a direct product and then we can ask which subgroups can possibly be normal in the case of $Q_8 \times Z_4$.
The statement of the lemma is that for groups $G, H$, the set of all subgroups of $G \times H$ is in bijective correspondence with the set of tuples $(G_1, H_1, N, M, \varphi)$, where $N \trianglelefteq G_1 \leq G, M \trianglelefteq H_1 \leq H$, and
\begin{align} \varphi: G_1/N \rightarrow H_1/M \end{align}
is an isomorphism. To use this in practice, the point is that any $K \leq G \times H$ is given by
\begin{align} K_{\varphi} = \{(g,h) \,|\, g \in G_1, h \in H_1, \varphi(gN) = hM \} \end{align}
for each $\varphi$ satisfying the isomorphism condition above (see also this answer and this answer for more info on Goursat's lemma).
Now let's apply Goursat's lemma in the case of $Q_8 \times Z_4$. We first consider all possible subgroups of $Q_8$ and $Z_4$ and their respective normal subgroups, and then find all cases where we have $G_1/N \cong H_1/M$, as in the statement of the lemma.
\begin{align} \text{Subgroups of } Q_8 &: Q_8, Z_4, Z_2, 1\\ \text{Subgroups of } Z_4 &: Z_4, Z_2, 1 \end{align}
Let's consider what happens when we have trivial quotient groups (for example, $G_1 = Q_8, N = Q_8, H_1 = Z_4, M = Z_4$ and then we obtain $Q_8/Q_8 \cong 1 \cong Z_4/Z_4$). In this case, following the statement of Goursat's lemma above, we only have a single isomorphism $\varphi$ to the trivial group. Then this corresponds to $K_{\varphi} \leq G \times H$ as
\begin{align} K_{\varphi} = \{(g,h)\,|\,g\in G_1, h \in H_1\} \end{align}
because there is only a single coset in $Q_8/Q_8$ and in $Z_4/Z_4$, and so every $g \in Q_8 = G_1$ satisfies the condition $\varphi(g Q_8) = h Z_4$ for every $h \in Z_4 = H_1$.
In other words, the quotient groups are trivial if and only if $K_{\varphi}$ is the direct product of a subgroup of $G$ and a subgroup of $H$. Since every subgroup of $Q_8$ and $Z_4$ is normal, every $G_1 \times H_1 \trianglelefteq G \times H$.
Next, it's quite quick to write all possible nontrivial quotient groups such that $G_1/N \cong H_1/M$ (noting that every subgroup of both $Q_8$ and $Z_4$ is normal):
\begin{align} G_1/N &\cong X \cong H_1/M\\ \hline Q_8/Z_4 &\cong Z_2 \cong Z_4/Z_2\\ Q_8/Z_4 &\cong Z_2 \cong Z_2/1 \\ Z_4/Z_2 &\cong Z_2 \cong Z_4/Z_2 \\ Z_4/Z_2 &\cong Z_2 \cong Z_2/1 \\ Z_4/1 &\cong Z_4 \cong Z_4/1 \\ Z_2/1 &\cong Z_2 \cong Z_4/Z_2 \\ Z_2/1 &\cong Z_2 \cong Z_2/1 \\ \end{align}
(notice $Q_8/Z_2$ can't appear in the list, because it is the Klein four-group and so can't be isomorphic to any quotient of $Z_4$). The condition for $K_{\varphi}$ to be normal in $G \times H$ is
\begin{align} (x g x^{-1}, y h y^{-1}) \in K_{\varphi},\quad \forall\, (x,y) \in Q_8 \times Z_4, (g,h) \in K_{\varphi} \end{align}
$Z_4$ is abelian and so this reduces to
\begin{align} (x g x^{-1}, h) \in K_{\varphi},\quad \forall\, (x,y) \in Q_8 \times Z_4, (g,h) \in K_{\varphi} \end{align}
The condition $(x g x^{-1}, h) \in K_{\varphi}$ means that $\varphi(x g x^{-1} N) = h M = \varphi(g N)$. In other words, $x g x^{-1}$ needs to be in the same coset as $g$, or equivalently $x g x^{-1} g^{-1} \in N$. It's convenient to use the notation of the group commutator:
\begin{align} [x,g] \in N, \quad \forall g \in G_1, \forall x \in G = Q_8 \end{align}
Now we refer to our list above of nontrivial quotients and check when it's possible for $[x,g] \in N$. It's useful to check the commutator subgroup (group generated by all commutators) of $Q_8$, and we find that it is $Z_2$. So in every quotient of the form $G_1/Z_4$ or $G_1/Z_2$ in the list above, any commutator of $[x,g], \, g \in G_1, x \in Q_8$ will be in $Z_2 \leq N$ for these groups. Altogether, all of these subgroups are normal in $Q_8 \times Z_4$.
We could have checked the commutator subgroup and used this observation earlier when we were considering trivial quotients, but it was useful to show that case on its own, to get a feel for what Goursat's lemma is actually saying and to prepare for other similar questions where possibly things don't line up so nicely.
Lastly, according to our analysis, the only subgroups that can possibly fail to be normal in $Q_8 \times Z_4$ are those corresponding to the $G_1/N$ quotients $Z_4/1$ or $Z_2/1$. These are resolved as follows:
$Z_2$ is the center of $Q_8$ and so the commutators $[x,g], \, g \in Z_2, x \in Q_8$ will all lie in the trivial group $1$, which corresponds to $N$ for this case. So the subgroups corresponding to $Z_2/1$ are all normal in $Q_8 \times Z_4$.
The groups corresponding to $Z_4/1$ are not normal if and only if we can find $[x,g] = -1 \iff xg = -gx$ (we know that the commutators have to be -1 or 1 because $Z_2$ is the commutator subgroup of $Q_8$, and if they are always 1, then they lie in $N = 1$ and the corresponding groups are normal). Certainly $xg = -gx$ occurs, for example $ij = -ji$.
Altogether, we find that $K \not\trianglelefteq Q_8 \times Z_4$ if and only if $K$ corresponds to $G_1/N = Z_4/1$. One can check that one such subgroup is $\{(1,0),(i,1),(-1,2),(-i,3)\}$ as was found in David Wheeler's answer.
Consider $N = \langle (i,1)\rangle$ which is cyclic of order $4$:
$(i,1)^2 = (-1,2)\\(i,1)^3 = (-i,3)\\(i,1)^4 = (1,0).$
Now $(j,0)(i,1)(j,0)^{-1} = (jij^{-1},0+1-0) = (ji(-j),1)$
$= ((-k)(-j),1) = (kj,1) = (-i,1) \not\in N$.
