I live less in NJ, less than 15 miles from NYC. Near me, when you really stop to look at most areas it's just straight disgusting here. There are a few towns that are exceptions, but they're generally pretty wealthy areas. So in NJ/NYC $$$ = cleaner streets.
I just got back from Japan less than a month ago. There is some littering to be sure, but I really only noticed it in a few areas (primarily in certain parts of Tokyo, like Shinjuku and some in Osaka, near Dotonburi-dori). However unlike NYC, there are plenty of areas in Tokyo that are pretty much spotless. I compare Shinjuku to the amount of trash/litter you'd expect to find in Times Square NYC. As you leave the major cities I saw even less trash/liter. While I was in Hakone I don't think I saw any at all!
So Japan as a whole is not entirely spotless and clean, but in comparison to NJ and NYC, it might as well be.