Considering the spectrum of unbounded operators as a very special definition that doesn't quite fit into the general definition of the spectrum as follows...
The basic ingredient for spectral theory in general is a unital algebra $1\in\mathcal{A}$.
Denote the set of invertibles by $\mathcal{A}^\ast$.
Then, the spectrum of an element is nothing but:
$$A\in\mathcal{A}:\quad\sigma(A):=\{\lambda\in\mathbb{C}:A-\lambda 1\notin\mathcal{A}^\ast\}$$
Now, the bounded linear operators on a Banach space form a unital Banach algebra:
$$1\in\mathcal{B}(E):=\{T:E\to E:T\text{ bounded, linear}\}$$
It is important, though, to have an identity operator; that forces the target space to agree with the domain! So though the bounded linear operators between different Banach spaces form a Banach algebra they miss an identity:
$$1\notin\mathcal{B}(E,F)$$
Note also that not the Banach space itself is the structure being studied but the algebra of operators acting on the Banach space...
Of course, one could as well consider the bounded linear operators on a normed spaces:
$$1\in\mathcal{B}(X):=\{T:X\to X:T\text{ bounded, linear}\}$$
or one could even consider the merely linear operators on a vector space:
$$1\in\mathcal{L}(V):=\{T:V\to V:T\text{ linear}\}$$
However, the one lacks of completeness being ridiculously important and the other even misses topological structure at all.