Question 1058420
The cross section would be a squared up U, with vertical sides {{{x}}} inches tall,
and a middle horizontal section {{{12-2x}}} inches wide.
We really want to maximize the cross section area of the {{{x}}} by {{{12-2x}}} rectangle,
because the 8 inch depth of the file is determined by the width of the 8 in by 12 in sheet of plastic.
The area of that cross section is 
{{{A(x)=2x(12-2x)}}} or {{{A(x)=4x(6-x)}}}
That is a polynomial of degree 2, or in other words a quadratic function.
We know that those are symmetrical functions with a minimum or maximum at their axis of symmetry (at the vertex of the parabola).
That maximum has to be exactly halfway between the zeros of the function,
which we knew would be at {{{x=0}}} and (((x=6}}} before writing anything,
because those values correspond to no vertical sides, and no horizontal bottom respectively.
So, the file should be {{{x=(0+6)/2=highlight(3)}}} inches tall.
 
What did your teacher expect?
Maybe calculating volume as
{{{V(x)=8*x*(12-2x)}}}
{{{V(x)=8*(12x-2x^2)}}}
{{{V(x)=-16x^2+96x}}}
and after that,
either calculating the derivative to find the {{{x}}} value for the maximum,
or using the "formula" {{{x=-b/2a}}}
to find the axis of symmetry of a quadratic function
{{{f(x)=ax^2+bx+c}}} .