6x³+3x²y-2xy–y²

First way:

Factor first two terms 6x³+3x²y as 3x²(2x+y).
Factor last two terms -2xy-y² as -y(2x+y).

3x²(2x+y)-y(2x+y)

Let U = (2x+y)

3x²U-yU

U(3x²-y)

Replace U by (2x+y)

(2x+y)(3x²-y)   

=======================

6x³+3x²y-2xy–y²

Second way:

Swap middle two terms:

6x³-2xy+3x²y–y²


Factor first two terms 6x³-2xy as 2x(3x²-y).
Factor last two terms +3x²y-y² as +y(3x²-y).

2x(3x²-y)+y(3x²-y)

Let U = (3x²-y)

2xU+yU

U(2x+y)

Replace U by (3x²-y)

(3x²-y)(2x+y)   

Edwin