Suppose $M$ is a manifold. The tangent space at a point $p$, $T_pM$, is defined to be the set of all derivations: $$T_pM = \{D: C^\infty(M) \rightarrow \mathbb{R} ~|~ D(fg)(x) = f(x)D(g)(x) + g(x)D(f)(x)\}.$$ A vector field $X$ is then defined as assigning $d \in T_pM$ to each $p \in M$.
When defining vector fields and tangent spaces, I am not surprised that we impose some degree of smoothness due to how tangent spaces are defined using derivatives in basic calculus. But I have been wondering what would happen if we were to relax the domain of $T_pM$ and hence of $X$ in the more abstract definition above. For example, instead of defining the derivations to be over $C^\infty$, what if we were to define it to be over $C^0$? This would allow us to define vector fields and tangent spaces on any topological manifold, but I read a comment that this is impossible and they can only be defined on smooth manifolds. Is there any particular reason for this and we define the derivations to be over $C^\infty$?