Most worlds from books or novels are set up to be 'interesting' which means lots of conflict and trouble. That's not a place you'd want to live in. While Banks' books cover 'flashpoints' of trouble they are exceptional circumstances in a universe that by and large is pretty damn awesome.
I'm not so enamoured of fantasy that I'm willing to move to a world without indoor plumbing and proper medicine, nor does the idea of a galactic war appeal since I know that I'd be next to useless in such situations. Bumming around the galaxy as a tourist however... that appeals.