Oh yes, the vast majority of the time people are great, but I would not put things in as broad of terms as you did regarding how smooth things go if you speak/read Japanese.
I have had friends refused service and I have been held to what I think is the credit card double standard as soon as they find out I am not Japanese. I have also been flat out lied to by one inn keeper in particular on an issue I really don't feel like going into.
But for example, the credit card thing. Book a hotel in Japan from overseas and you will be asked to hold a room with a credit card. If you book from Japan, even on-line, you don't have to give out a credit card or even place a deposit (which is what, in theory, you do when you leave a number to hold a room). So what's the deal? Why when my wife calls the exact same place she is not asked for a card number?
I'm not saying this is out-and-out racism, but why do you need to make special rules for foreigners, even ones living in Japan? Do foreigners change their hotel reservations more often than Japanese people? I think that is quite unlikely. Just because you speak Japanese, doesn't mean you are going to be an honorary Japanese.
I grant you that I have been happy with my treatment more than 90% of the time in Japan. 90 is not the same as 100. It is more of a case of a few bad apples giving everyone the wrong impression of the rest. Customer service in Japan is great, and I hope my posts don't put people off coming to Japan.