What LL says is exactly right, so this is just to add a one or two points which might help make the formal definitions fit better with the informal notions you were using in posing your question.
As LL says, a distribution is a real-valued linear map with domain $\mathcal T$, the space of smooth functions supported on an interval $[-R,R]$ for some $R>0$. If $f$ is any locally integrable function (i.e. integrable on any finite interval) and $\phi\in \mathcal T$, then $f.\phi$ is is integrable and supported in $[-R,R]$ where we assume $\phi$ is supported in $[-R,R]$. Hence we may associate to $f$ a distribution $\delta_f$ where
$$
\begin{split}
\delta_f(\phi) &= \int_{-\infty}^{\infty} f(t)\phi(t) dt,\\
&= \int_{-R}^R f(t)\phi(t)dt <\infty.
\end{split}
$$
Now suppose that
$$
s_n = \left\{ \begin{array}{cc}n(1+nt) & \text{ if } -1/n \leq t \leq 0 \\
n(1-nt), & \text{ if } 0<t \leq 1/n
\end{array}
\right.
$$
so that $s_n$ is nonzero only on $(-1/n,1/n)$ and in that interval is a triangular spike with height $n$ and base $2/n$, so that $\int_{-\infty}^{\infty} s_n(t)dt = 1$. Each $\phi_n$ thus defines a distribution $\delta_n$ by taking $f = s_n$ in the above. Now if $\phi \in \mathcal T$,
$$
\lim_{n\to \infty} \delta_n(\phi) = \lim_{n \to \infty} \int_{-\infty} ^\infty s_n(t).\phi(t)dt = \phi(0),
$$
since $\phi$ is continuous at $t=0$. Thus in some suitable sense $s_n\to \delta_0$, the Dirac delta function, as $n \to \infty$.
But now notice that for any $n$, $s_n(t)\phi(t)$ vanishes for $t \notin (-1,1)$, and in fact
$$
\begin{split}
\int_{-\infty}^{\infty} s_n'(t).\phi(t)dt &= [s_n(t).\phi(t)]_{-\infty}^{\infty} - \int_{-\infty}^{\infty} s_n(t).\phi'(t)dt \\
&\to 0 - \phi'(0) = -\phi'(0)
\end{split}
$$
so that $s_n'$ tends to $-\delta_0\circ \frac{d}{dx}$, which is exactly what your informal ``integration by parts'' heuristic led you to.