Question 689216
If "the longer leg of a right triangle is 3 cm less than 5 times the shorter leg,"   
you are correct in translating
{{{a^2 + b^2 = c^2}}} into {{{b^2 + (5b - 3)^2 = c^2}}}, calling the shorter leg {{{b}}}.
 
However, if the problem says exactly that the "hypotenuse is 3 cm longer than the shorter leg," it should be
{{{ b^2 + (5b - 3)^2 = (b + 3)^2}}} instead of 
{{{b^2 + (5b - 3)^2 = (5b + 3)^2}}}.
Then,
{{{ b^2 + (5b - 3)^2 = (b + 3)^2}}} --> {{{26b^2 - 30b +9 = b^2 + 6b +9}}} --> {{{25b^2 -36b=0}}} --> {{{25b(b-36/25)=0}}}
and as {{{b=0}}} makes no triangle, it must be
{{{highlight(b=36/25)}}}cm, or, if you prefer {{{highlight(b=1.44)}}}cm.
 
You made the hypotenuse 3 cm longer than 5 times the shorter leg in
{{{b^2 + (5b - 3)^2 = (5b + 3)^2}}}.
If that was really what the problem stated, then
{{{26b^2-30b+9=25b^2+30b+9}}} --> {{{b^2-60b=0}}} --> {{{b(b-60)=0}}}
and the solution would be {{{highlight(b=60)}}}cm.