In fact, $H$ is complete iff for every closed subspace $K\subseteq H$, one has that $H=K\oplus K^\perp$. Actually, we will prove
something a bit stronger:
Theorem. Let $H$ be a separable, infinite dimensional inner-product space such that $K^\perp\neq \{0\}$, for every proper closed subspace $K\subseteq H$. Then
$H$ is complete.
Proof. Find an orthonormal basis $\{e_n\}_n$ for $H$ (the existence of bases
relies only on Graham-Schmidt and hence works on any inner-product space).
As usual we may then use $\{e_n\}_n$ to view $H$ as a subspace of
$\ell ^2$ containing the compactly supported sequences.
If $H$ is not complete, then $H\subsetneq \ell ^2$, so we may pick a vector $\xi \in \ell ^2\setminus H$. It is then obvious that
$$
K:= H\cap \{\xi \}^\perp
$$
is a proper closed subspace of $H$, and I claim that the orthogonal complement of $K$ within $H$ coincides with $\{0\}$.
In other words, we claim that
$$
K^\perp\cap H=\{0\}.
$$
To see this, suppose by contradiction that $\eta $ is a nonzero vector in $K^\perp\cap H$.
Fix any $m$ such that $\langle e_m, \xi \rangle \not=0$, and for every $n$ consider the vector
$$
u_n = e_n - \frac{\langle e_n, \xi \rangle }{\langle e_m, \xi \rangle }e_m.
$$
Observing that $\langle u_n, \xi \rangle =0$, we see that $u_n\in K$, so
$$
0 = \langle u_n, \eta \rangle =
\langle e_n, \eta \rangle - \frac{\langle e_n, \xi \rangle }{\langle e_m, \xi \rangle }\langle e_m, \eta \rangle ,
$$
hence
$$
\langle e_n, \eta \rangle = \frac{\langle e_n, \xi \rangle }{\langle e_m, \xi \rangle }\langle e_m, \eta \rangle = \langle e_n, \xi \rangle \frac{\langle e_m, \eta \rangle }{\langle e_m, \xi \rangle }.
$$
Since $m$ is fixed we see that the coefficients of $\eta $ and $\xi $ are proportional, meaning that also $\eta $ and $\xi $
themselves are proportional, but this is a contradiction because $\eta $ lies in $H$ and $\xi $ does not.
