Nope, Ginza smells fine. Where in Tokyo do you want to stay? There are many nice places to stay and the subway system is such that wherever you stay, it's never that far from any place in Tokyo.
Before my first trip there, I've been 3 times now in the past 8 months, I was really concerned where I was going to stay because I didn't want to be too far from certain areas. But now that I've been there 3 times, I can honestly say that you will be astounded by the precision and vast coverage of the subway system throughout Tokyo.
People who live in Tokyo really do not need cars...it's that simple.