Given:
$$A = \begin{bmatrix} 5 & -3 & -2 \\ 8 & -5 & -4 \\ -4 & 3 & 3 \end{bmatrix}, ~~ F[t] = \begin{bmatrix} - \sin t \\ 0 \\ 2 \end{bmatrix}$$
We can find the solution to this system using:
$$X(t) = e^{At}X_0 + \int_{t_0}^t e^{A(t-s)}F(s)~ds$$
If we find the Jordan Normal Form of the matrix, we have:
$$A = PJP^{-1} = \begin{bmatrix} 0 & -2 & 0 \\ 1 & -4 & 0 \\ -\dfrac{3}{2} & 2 & 1 \end{bmatrix}\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} -2 & 1 & 0 \\ -\dfrac{1}{2} & 0 & 0 \\ -2 & \dfrac{3}{2} & 1 \end{bmatrix} $$
Update How did we find these eigenvectors?
$A$ has only two Jordan blocks for only one eigenvalue $\lambda = 1$, so:
$$A-I \ne 0 \\ (A-I)^2 = 0$$
Thus, we can take any vector $v_3$ such that $(A-I)v_3 \ne 0$, which becomes an eigenvector of rank $2$. So, take:
$$v_3 = (0,0,1) \implies (A-I)v_3 \ne 0$$
Now, we have:
$$v_2 = (A-I)v_3 = (-2,-4,2)$$
Lastly, we need a third linearly independent eigenvector such that:
$$(A-I)v_1 = 0 \implies v_1 = \left(0,1,-\dfrac{3}{2}\right)$$
Note: $P$ is made up of three generalized eigenvectors, $P = [v_1~|~v_2~|~v_3]$, from the single eigenvalue, since we need three linearly independent eigenvectors.
We can now find $e^{Jt}$ as:
$$E^{Jt} =e^t\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & t \\ 0 & 0 & 1 \end{bmatrix}$$
We can now write the fundamental matrix as:
$\phi(t) = Pe^{Jt}P^{-1} = \begin{bmatrix} 0 & -2 & 0 \\ 1 & -4 & 0 \\ -\dfrac{3}{2} & 2 & 1 \end{bmatrix} e^t\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & t \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} -2 & 1 & 0 \\ -\dfrac{1}{2} & 0 & 0 \\ -2 & \dfrac{3}{2} & 1 \end{bmatrix}$
So,
$$ \phi(t) = e^t\begin{bmatrix} 4t+1 & -3t & -2t \\ 8t & 1-6t & -4t \\ -4t & 3t & 2t+1 \end{bmatrix}$$
We now have:
$$\phi^{-1}(t) = e^{-t}\begin{bmatrix} 1-4t & 3t & 2t \\ -8t & 6t+1 & 4t \\ 4t & -3t & 1-2t\end{bmatrix}$$
- Find $\phi^{-1}(t) \cdot F(t) = \begin{bmatrix}
e^{-t} (4 t+(4 t-1) \sin (t)) \\
8 e^{-t} t (\sin (t)+1) \\
e^{-t} (-4 \sin (t) t-4 t+2) \\
\end{bmatrix}$
- Now we integrate the previous result, that is $w = \int \phi^{-1}(t) F(t)$ and this gives us: $w$
- Next, we find:
$X(t) = X_h + X_p = \begin{bmatrix} x(t) \\ y(t) \\z(t) \end{bmatrix} = e^{At}X_0 + \phi(t) w = \begin{bmatrix} c_1 e^t (4 t+1)-3 c_2 e^t t-2 c_3 e^t t-4 t (2 t+t \sin (t)+(t+1) \cos (t)+1)+12 t (2 t+t \sin (t)+(t+1) \cos (t)+2)-\frac{1}{2} (4 t+1) (8 (t+1)+(4 t-1) \sin (t)+(4 t+3) \cos (t)) ~~~\\ ~~~8 c_1 e^t t-c_2 e^t (6 t-1)-4 c_3 e^t t-8 t (2 t+t \sin (t)+(t+1) \cos (t)+1)+4 (6 t-1) (2 t+t \sin (t)+(t+1) \cos (t)+2)-4 t (8 (t+1)+(4 t-1) \sin (t)+(4 t+3) \cos (t)) ~~~ \\ ~~~-4 c_1 e^t t+3 c_2 e^t t+c_3 e^t (2 t+1)+2 (2 t+1) (2 t+t \sin (t)+(t+1) \cos (t)+1)-12 t (2 t+t \sin (t)+(t+1) \cos (t)+2)+2 t (8 (t+1)+(4 t-1) \sin (t)+(4 t+3) \cos (t)) \end{bmatrix}$