You have three groups of books, and you can arrange them as whole groups in 
3! = 3*2*1 = 6 different ways on te shelf.


After that, you can arrange 5 Math books in 5! = 5*4*3*2*1 = 120 different ways 
inside this group;

            you can arrange 4 Science books in 4! = 4*3*2*1 = 24 different ways 
inside this group;

            you can arrange 3 English books in 3! = 3*2*1 = 6 different ways 
inside that group.


Doing this way, you obtain 6*120*24*6 = 103680 different possible arrangements
under given restrictions.