Question 1120791
To rotate axes by a radians (note that a is confined within (0, pi))(Personally I'd like to do it counterclockwise, but you're free to do otherwise), you'll need to convert the coordinates accordingly, which can be proved with a little simple geometry.


x=x'cosa-y'sina
y=x'sina+y'cosa


Put them into the equation you gave:
sina*cosa*(x'^2-y'^2)+((cosa)^2-(sina)^2)x'y'-2(x'sina+y'cosa)-4(x'cosa-y'sina)=0
Aha! To succeed, we just need to get the coefficient of the term x'y' equal to 0.


(cosa)^2-(sina)^2=cos(2a)=0.
a=pi/4 or 3pi/4.


And it's done.
//SORRY FOR THE ERROR WITH THE FOMULA GENERATOR. 

Wanna know more? check this out http://www.sparknotes.com/math/precalc/conicsections/section5/#_motz_