Let's see if we can factor that using synthetic division:

1 | 1  0 -6  8 -3
  |    1  1 -5  3 
    1  1 -5  3  0

so 


1 | 1  1 -5  3
  |    1  2 -3
    1  2 -3  0

so now 

and

When the determinant is not 0.  And since



That will be when 

We row-reduce this augmented matrix:



And we get:



So the only solution is



That's a unique solution. There are no arbitrary constants, unless 
you want to say that "a" itself 
is the only arbitrary constant and there is 1 arbitrary constant.

Edwin

A = 

I will calculate the determinant det (A) in two steps.


Step 1


Subtract row 1 from the rows 2, 3, 4 and 5. You will get modified 5x5-matrix B

B = 

Under these transformations (and after these transformations), det (A) = det (B).


Step 2


In matrix B, add all the columns 2, 3, 4 and 5 to the column 1. You will get modified 5x5-matrix C

C = 

Under these transformations (and after these transformations), det (A) = det (B) = det (C).

Now calculate the determinant det (C) using cofactoring along the 1-st column.

You will get  det (C) = .


Thus  det (A) = .


In this way, you can calculate the determinant of the given form matrix for ANY order "n".


So, the question #1 is answered.



The answer to the question #2 is:  


    Ax = 0 has nonzero solutions for a = -4  and for a = 1.



The question #3 is formulated INCORRECTLY, in my view.

The correct formulation is THIS:


    when Ax = 0 has nonzero solution, how many arbitrary constants are there in the general solution?


And the answer to this question is:

    when a = 1,  the general solution has  4  arbitrary constants;

    when a = -4, the general solution has  1 arbitrary constant.