What is the intuition behind the method of undetermined coefficients?

Question

Our teacher has recently begun teaching second-order differential equations and the methods used for solving them.

The method which we are taught to solve linear differential equations is currently the method of undetermined coefficients. I would like to know if there is a reason as to why the method works. My chief question is about why we're able to just "step-up" our guesses by an x term (i.e. if $$e^{ax}$$ does not work we're simply able to amend our guess to $$xe^{ax}$$ Secondly, why is it that when doing so the terms in between seem to nicely cancel out? An example is the differential equation in one of our tutorials $$\frac{d^2y}{dx^2}-6\frac{dy}{dx}+9y=e^{3x}$$

As the solution to the characteristic polynomial has repeated roots 3, I understand why guesses of the form $$Ae^3x$$ $$Axe^3x$$ fail to work (because they get "absorbed" into the general solution of the complementary solution) and as a result why my "guess" has to be $$Ax^2e^{3x}$$ but I do not understand why when plugging in the solved integrals that somehow I get this mess of an equation $$A(9x^2e^{3x}+12xe^{3x}+2e^{3x}-18x^2e^{3x}-12xe^{3x}+9x^2e^{3x})=e^{3x}$$ that somehow resolves nicely to $$A(2e^{3x})=e^{3x}$$

Thank you for taking the time to read this.

Answer 1

Let's consider your example $$L[y]=y''-6y'+9y=e^{kx}$$ In operator notation $$D^2y-6Dy+9y=e^{kx}$$ $$(D^2-6D+9)y=e^{kx}$$ $$(D-3)(D-3)y=e^{kx}\tag1$$ What makes the "undetermined coefficients" method work here is the fact that $y=e^{kx}$ is a solution of the homogeneous linear equation $(D-k)y=0$. Therefore, multiplying the equation by $D-k$ will "annihilate" the input $e^{3x}$ and turn our nonhomogeneous second order equation into a homogeneous third order equation. (This is called the annihilator method.) $$(D-k)(D-3)(D-3)y=0\tag2$$ Assuming $k\ne3$, the general solution of the homogeneous equation $(2)$ is $$y=Ae^{kx}+(Bx+C)e^{3x}$$ where $A,B,C$ are arbitrary constants. However, not all solutions of $(2)$ are solutions of $(1)$; multiplying by $D-k$ introduced extraneous solutions; so we have to plug $(3)$ back into the original equation $(1)$ to see which values of the constants solve the original problem. $$L[Ae^{kx}+(Bx+C)e^{3x}]=e^{kx}$$ $$L[Ae^{kx}]+L[(Bx+C)e^{3x}]=e^{kx}$$ But we know that $L[(Bx+C)e^{3x}]=0$, because $y=(Bx+C)e^{3x}$ is the general solution of the homogeneous equation $L[y]=0$. Therefore $B$ and $C$ are arbitrary, and we have to determine $A$ by solving $$L[Ae^{kx}]=e^{kx}$$ In other words, the nonhomogeneous equation $(1)$ has a particular solution of the form $y=Ae^{kx}$.

That's assuming $k\ne3$. On the other hand, if $k=3$, then the general solution of $(2)$ is $$y=Ax^2e^{3x}+(Bx+C)e^{3x}$$ and in this case the particular soution takes the form $y=Ax^2e^{3x}$.

Answer 2

You can think of infinitely differentiable functions $f:\mathbb{R}\rightarrow \mathbb{R}$ as a vector space. It is definitely an infinite dimensional vector space. It has lots of finite dimensional subspaces, like the solution space to a homogeneous linear differential equation with constant coefficients. Call a function $f$ {\bf finite type} if the subspace $V$ that comes from taking linear combinations of all order derivatives of $f$ is finite dimensional. For instance, polynomials are finite type, the sine and cosine, $e^{ax}$ and their products are finite type. The method of undetermined coefficients works for solving linear constant coefficient differential equations where the function on the right hand side is of finite type.

The left hand side of the differential equation $$ D:=\frac{d^2}{dx^2}+b\frac{d}{dx}+c$$ should be thought of as a linear map from the vector space of infinitely differentiable functions to itself. Let $V$ be the vector space spanned by all derivatives of $f$. By definition, the restriction of $D$ to $V$ maps $V$ into itself. Hence if $D:V\rightarrow V$ is onto, you can definitely solve the equation $$Dg=f$$ with some $g\in V$, you just choose $g$ in the inverse image of $f$. If $D:V\rightarrow V$ isn’t onto, then since a linear map from a finite dimensional vector space to itself is onto if and only if it is one-to-one, that means that $V$ has nontrivial intersection with the solution space of $Dg=0$. So, let $W=\{x^sg|g\in V\}$. This is a new vector space of the same dimension as $V$, and it has smaller intersection with the solution space of $D$.

If you choose too high a power $s$, $D$ will not map $W$ into $V$, but if you choose the smallest $s$ so that the intersection of $W$ with the solution space of $D$ is $\{0\}$, then $D:W\rightarrow V$ is onto, and there is a unique $g\in W$ with $Dg=f$.