By analyzing the connection between complex Hadamard matrices and spectral sets, we prove the direction "spectral double right arrow tile" of the Spectral Set Conjecture, for all sets A of size \A\ <= 5, in any finite Abelian group. This result is then extended to the infinite grid Z(d) for any dimension d, and finally to R-d. It was pointed out recently in  that the corresponding statement fails for \A\ = 6 in the group Z(3)(5), and this observation quickly led to the failure of the Spectral Set Conjecture in R-5 , and subsequently in R-4 . In the second part of this note we reduce this dimension further, showing that the direction "spectral double right arrow tile" of the Spectral Set Conjecture is false already in dimension 3. In a computational search for counterexamples in lower dimension (one and two) one needs, at the very least, to be able to decide efficiently if a set is a tile (in, say, a cyclic group) and if it is spectral. Such efficient procedures are lacking however and we make a few comments for the computational complexity of some related problems.