Take, for example, the polynomial $15x^2 + 5xy - 12y^2$.
6 Answers
What you have is a homogeneous polynomial of degree $2$ in $x$ and $y$ (homogeneous means that the total degree is the same in each monomial).
Divide through by $y^2$, and set $Z=\frac{x}{y}$. (This is called "de-homogeneization"). This will give you the one-variable degree $2$ polynomials $$aZ^2 + bZ + c.$$ If you know how to factor these (which is not hard), then after factoring $$aZ^2 + bZ + c = (rZ+s)(tZ+u)$$ you can multiply through by $y^2$ by multiplying one $y$ in each factor on the right hand side, to get back to the homogeneous form ("homogeneization"): $$ax^2 + bxy + cy^2 = (rx+sy)(tx+uy).$$ So it all comes down to factoring degree $2$ polynomials.

- 398,050
D'you know how to factor $15x^2+5x-12$? Can you see how to go from there to what you want?

- 179,216
-
After reading Arturo's answer, what you asked made more sense; I was thinking of the steps I took to factor ax^2 + bx + c rather than the logic behind what I did in factoring that. :) – Randall Ma Jun 26 '12 at 04:33
For degree $2$ polynomials we can use the quadratic formula. This will solve for $x$ in terms of $y$.

- 7,612
- 4
- 33
- 66
For general $a x^2 + b x y + c y^2,$ with integers $a,b,c,$ we calculate the discriminant $$ \Delta = b^2 - 4 a c. $$ If $b^2 - 4 a c$ is nonnegative and a perfect square, such as $0,1,4,9,16, \ldots,$ then the form can be factored, otherwise not.

- 139,541
I will assume that the coefficients $a$, $b$, and $c$ are real, with $a \ne 0$, and we are factoring over the reals.
We can write our polynomial as $$\frac{1}{4a}\left(4a^2x^2+4abxy+4ac\right).$$ Complete the square. We get $$\frac{1}{4a}\left((2ax+by)^2-(b^2-4ac)y^2\right).$$ If we are trying to factor over the reals, and $b^2-4ac$ is negative, there will be no factorization. But if $b^2-4ac \ge 0$, then we can factor as $$\frac{1}{4a}\left(\left(2ax+\left(b-\sqrt{b^2-4ac}\right)y\right)\left(2ax+\left(b+\sqrt{b^2-4ac}\right)y\right)\right).$$ If we wish, we can absorb $\frac{1}{4a}$ in various ways into the "main" factors.
For your particular example, the factorization is $$\frac{1}{60}\left(30x+(5-\sqrt{745})y\right)\left(30x+(5+\sqrt{745})y\right).$$

- 507,029
-
Ok so I wanted to ask how to factor $ax^2+bxy+cy^2$ explicitly snd you did it here but I really do not see the intuition to pulling out $\frac{1}{4a}$ could you go over how you factor that thing plz – homosapien Mar 07 '20 at 22:55
If you're curious, see my answer here for comparison. Note that the factorizations won't necessarily be integer factorizations. However, the methods for factoring $ax^2+bxy+cy^2$ are basically the same as those for factoring $ax^2+bx+c$, but with one extra step at the end.

- 102,994