I can't speak for Tokyo since I haven't been there. But when I was in Osaka and Okayama recently, there were plenty of foreignors in both cities. In fact, Mad Pierrot and I hung out together!
It will help you greatly to know some conversational Japanese. You might not be treated as well if you always insist on speaking English and that they understand everything you say. I know you wouldn't do that, but some Americans have that attitude.