Suppose $f:X\to\mathbb{R}$ is a function and we want to define its derivative in a similar way as that we do when the function is defined in $\mathbb{R}^n$, i.e. we want to mimic the following limit (where $g:U\subset\mathbb{R}^n\to\mathbb{R}$) $$\lim_{t\to 0}\left|\frac{g(x+ty)-g(x)}{t}-g'(x)y\right|=0 \tag{1}$$
From $(1)$ we see that we need two things: the first one is a way to measure the length of the expression $\left|\frac{g(x+ty)-g(x)}{t}-g'(x)y\right|$. The other one, is the fact that $g(x+y)$ must be well defined. We can solve these two issues by asking that $X$ is a normed vector space.
Now, we can say that a function has Gâteaux derivative in the point $x$ with direction $y$ if and only if there exist a bounded linear functional $f'(x):X\to\mathbb{R}$ such that $$\lim_{t\to 0}\left\|\frac{f(x+ty)-f(x)}{t}-f'(x)y\right\|=0$$
where $\|\cdot\|$ is the norm in $X$. If $f$ is Gâteaux differentiable in some point $x$ and the Gateaux derivative is continuous in this point $x$, then we say that $f$ is Fréchet differentiable in $x$.
Now, there is plenty of examples of normed spaces $X$ which are infinite dimensional and you can derivate, for example, you can take the set $X=C([0,1])=\{u:[0,1]\to\mathbb{R}:\ \mbox{$u$ continuous}\}$. Define $f:X\to X$ by $$f(u)=u^2$$
You can verify that $f'(u)v=2uv$, $u,v\in X$.
Remark: Note that the space $\ell^p=\{x=(x_1,...,x_n,...):\ \sum_{i=1}^\infty |x_i|^p<\infty\}$ for $p\in [1,\infty)$ is included here.