We start with an x cm by x cm square



Then we extend it by 8 cm horizontally and 3 cm vertically, and
have this:



So the length of the extended rectangle is x+8 and the width is x+3

     Area = (length)(width)
      204 = (x+8)(x+3)
      204 = x²+3x+8x+24
      204 = x²+11x+24
Get 0 on the left
        0 = x²+11x-180
        0 = (x-9)(x+20)
Use zero factor property:
           x-9=0  x+20=0
             x=9     x=-20

Discard the negative answer.

Each side of the original square is 9 cm.

So the area of the original square is

     Area = (length)(width)
     Area = (9)(9)
     Area = 81 sq.cm.

Edwin