For metric spaces, the following theorem is true:
Let $X$ be any metric space, and $K$ be a compact subset of $X$. Then, for any infinite subset $E$ of $K$, $E$ has a limit point in $K$.
My question is: Is the converse true? If yes, can we take this as a definition of compactness for metric spaces?
The answer is yes.
If a metric space X, any of its infinite subsets has a limit point, X is compact.
The proof is a bit involving, but you should be able to find it easily on the web.
In terms of definition, since this is equivalent (true for both directions), logically it has no problem to be a definition for metric space.
However, this equivalent relation might not be true for general topological space, but the concept of compactness is still important there, thus I would not suggest to use this as the definition for compactness.
Plus, original definition of compact set is more intuitive to me as it captures the nature more closely that a compact set is "small" and "finite" in some sense, and it is more general so that we do not need a metric exist to define compact sets.
One more point is that you probably noticed a big part of Analysis is wrestling with "infinite sets". If the set is finite, it is very easy to deal with (e.g. we could just take the max of elements, we know it is bounded etc.). And compactness extend the finite sets into a type of important sets that is infinite but is the closest to "finite sets" - the best thing if we cannot have a finite set. And the original definition of compact set directly reveal this "finite cover" feature of a compact set
It is indeed equivalent for metric spaces, but it's good to keep the notions separate.
For all topological spaces we distinguish a few compact-like properties:
Every infinite subset $E$ of $X$ has a limit point is called "limit point compact"
Every countable open cover of $X$ has a finite subcover is called "countably compact"
Every sequence in $X$ has a convergent subsequence is called "sequentially compact"
and of course plain old compact: every open cover of $X$ has a finite subcover. (there are even more variants, but these are the most common ones).
For $T_1$ spaces $X$: $X$ countably compact iff $X$ limit point compact. For first countable spaces (or more generally sequential spaces) $X$: $X$ seq. cpt iff $X$ limit point compact. Compactness always implies limit point compact and countably compact.
As metric spaces are $T_1$ and first countable :
for metric spaces $X$ we already know now (based on these general facts):
$X$ compact $\rightarrow$ $X$ limit point compact $\leftrightarrow$ $X$ countably compact $\leftrightarrow$ $X$ sequentially compact.
If a metric $X$ is not Lindelöf, it has a uncountable closed and discrete set, so $X$ is not limit point compact. Hence limit point compactness implies Lindelöf (every open cover has a countable subcover) for metric spaces. So as countably compact Lindelöf spaces are compact (trivially), limit point compactness actually implies compactness for metric $X$, making all of them equivalent:
$X$ compact $\leftrightarrow$ $X$ limit point compact $\leftrightarrow$ $X$ countably compact $\leftrightarrow$ $X$ sequentially compact.
But the weaker countable compact-like notions are useful in larger classes of spaces and many non-compact countably compact spaces exist. But even a product of two limit point compact spaces need not be limit point compact in general Tychonov spaces, while any product of compact spaces is compact. So if we want to prove that e.g. $[0,1]^2$ is compact, we cannot really work with the limit point compact definition, but it's better to work with "real" compactness.
