10x² - 7xy - 5y²
The factorization, if there is one, has to be of the form
(x y)(x y)
Since there is just one factorization of the number 5 in the
last term as 5·1, we can put this:
(x 5y)(x 1y)
The 10 can be factored as 10·1, 5·2, 2·5 and 1·10, so we try all these
using tho O and I of "FOIL"
(10x 5y)(1x 1y) <--- outers = 10xy, Inners = 5xy, difference = 5xy
(5x 5y)(2x 1y) <--- outers = 5xy, Inners = 10xy, difference = 5xy
(2x 5y)(5x 1y) <--- outers = 2xy, inners = 25xy, difference = 23xy
(1x 5y)(10x 1y) <--- outers = 1xy, inners = 50xy, difference = 49xy
Since the last sign of 10x² - 7xy - 5y² is -, we look through
those to find one which two have a difference of the middle term,
7xy, ignoring the sign.
There are none, so the trinomial 10x² - 7xy - 5y² is PRIME, i.e., it
cannot be factored using integer coefficients.
Edwin