I think I'm not the one who can judge that for you. I live close to basel and that's perfect for me. One of the bigger cities in reach, close to the border and still much countryside around. Zurich is far the biggest and pretty expensive to live in. In my opinion too many people to get comfortable, if you know what I mean. Lucerne is a very beautiful city, you're right, but much tourism.
Everybody has their own opinions. I can asure you there are many people who would say Basel or Zurich for example are the worst one. But I can say that's because there is a big rivalry between the main cities so you probably have to find someone with living experience in different ones.