Since this is an advanced limit problem, you are probably in Calculus II
studying L'Hopital's rule.



Since both numerator and denominator approach 0,
we may use L'Hopital's rule.  The above is equal to











,



In the remaining limit, the numerator and denominator both approach 0,
so we can use L'Hopital's rule again, differentiating top and bottom:

               since cos(0)=1

So now our equation becomes:





  <-- correct answer, 2 values for k

You may get an heuristic check that this is the correct answer by 
using your calculator (TI-84 in radian mode) to calculate:



where k=±2 when x = 0.0001, for you get 10.00000003, which is close enough 
to 10 to convince anybody that we have the correct answer.

Edwin

As x is small and close to zero, we can use the decompositions for functions cos(2x) and cos(2kx)

    cos(2x) = 1 - 2x^2 + . . . ,  cos(2kx) = 1 - 2k^2x^2 + . . . ,  sin^2(x) = x^2 + . . . 


where ellipsis denote the terms of higher degrees, which do not make influence on my calculations.


So,  (1-cos(2x)*cos(2kx)) = 1 - (1-2x^2)*(1-2k^2x^2) = 2x^2 + 2k^2x^2 + . . . = 2x^2*(1+k^2) + . . . 


Therefore,   =  + . . . = 2*(1+k^2) + . . . 


From it, we conclude that  

    2*(1+k^2) = 10,

    1+k^2 = 10/2 = 5

       k^2 = 5 - 1 = 4,

       k = +/-  = +/- 2.


ANSWER.  There are two possible values for k, namely,  k = +/- 2.