Let x be the size of the square to cut at each corner (in inches).


Then the dimensions of the base of the box will be  (20-2x) inches and (16-2x) inches;

the height of the box will be x inches.


So, the volume of the box is  x*(20-2x)*(16-2x) cubic inches.


It gives you an equation

     x*(20-2x)*(16-2x) = 384.


Simplify by dividing both sides by 4

    x*(10-x)*(8-x) = 96.


It is a cubic equation.


You can solve it graphically, or by trial and error method, or using the Rational root test.


From the plot below, you can see that the roots of the last equation are x= 2, x= 4  and  x= 12.  


It is clear that the roots x= 2 and x= 4 are the solutions to the problem, while the root x= 12 is not.   


Having it, you may compute the possible areas of the cut squares.


    


            Plot y = x*(10-x)*(8-x) - 96