.
Let x be the original side of the square measure.
Then from the condition, you have this quadratic equation for the new area
(x+7)*(x-5) = 64.
From this point, you can easily guess the solution x+7 = 16, x = 16-7 = 9 and
then CHECK (9+7)*(9-5) = 16*4 = 64.
Alternatively, you can solve equation (*) formally
x^2 + 2x - 35 - 64 = 0
x^2 + 2x - 99 = 0
(x+11)*(x-9) = 0 etc.
Solved.