Set them identically "≡" (not conditionally "=") equal.

6x-x2 ≡ a-(x+b)2

6x-x2 ≡ a-(x2+2bx+b2)

6x-x2 ≡ a-x2-2bx-b2)

Add x2 to both sides:

6x ≡ a-2bx-b2

Now here is where we can do something in identity equations 
that we cannot do in a conditional equations.  That is, in 
identity equations we can equate coefficients of like powers 
of x (as well as constant terms).

The coefficient of x on the left is 6 and the coefficient
of x on the right is -2b, and since this is an identity
equation and not a conditional equation, we can set 6 and
-2b (conditionally) equal to each other:

 6 = -2b
-3 = b

Now we substitute -3 for b

6x ≡ a-2bx-b2
6x ≡ a-2(-3)x-(-3)2
6x ≡ a+6x-9
 9 ≡ a

Now let's check to see if we have an identity equation by 
substituting:

6x-x2 ≡ a-(x+b)2

6x-x2 ≡ 9-[x+(-3)]2

6x-x2 ≡ 9-(x-3)2

6x-x2 ≡ 9-(x2-6x+9)

6x-x2 ≡ 9-x2+6x-9

6x-x2 ≡ -x2+6x

6x-x2 ≡ 6x-x2

Yep, both sides are identical!

Edwin