If she bought 10n dresses at $10n she would pay $100n² with $5n² 20s and get no
change back. So no change at all is less than $10, so 10n dresses at $10 with
no change may be an acceptable solution. You'll have to ask your teacher.
Now we show that if she bought 10n±k dresses, she'd get the same change back as
if she bought k < 5 dresses.
Suppose the change she got from k < 5 dresses at $k per dress was $c.
Then
k²-20 = c where brackets indicate the greatest integer function.
(1) k² = 20 + c
If she bought k±10n dresses, her change would be
(k±10n)^2-20
k2±20nk+100n^2-20[(k±10n)^2/20]
k2±20(nk+5n^2-[(k+10n)^2/20])
If we add ±20(nk+5n^2-[(k+10n)^2/20]) to both sides of (1),
k^2 ± 20(nk+5n^2-[(k+10n)^2/20]) = c ± 20(nk+5n^2-[(k+10n)^2/20])
we have added or subtracted a whole number of $20 bills and so her
change will still be $c
Let's assume she did get some change back.
If she bought 1 dress for $1 each, she'd get $20-$1=$19 change. Not acceptable
If she bought 2 dresses for $2 each, she'd get $20-$4=$16 change. Not acceptable
If she bought 3 dresses for $3 each, she'd get $20-$9=$11 change. Not assceptable.
If she bought 4 dresses for $4 each, she'd get $20-$16=$4. Acceptable!
If she bought 5 dresses for $5 each, she'd get $40-$25=$15. Not acceptable.
Every other number of dresses differs from those by a multiple of 10 and thus
has one of those same amounts of change.
The only possible amounts she can get in change are $19,$16,$11,$4,$15.
There are only two amounts she can get in change less than $10, $0 and $4.
If she must get back some change, then the only possible amount of change is $4.
Edwin