Perhaps like many, everywhere, some, but not all, and probably not by a majority, they are perverted in their bedrooms, but the scorn people levy at them seems to compare them to ancient Rome.
Interesting you mention them.
The ancient Romans actually had few sexual taboos (among them incest) that were actually considered offensive to the gods themselves. Sex was often an expression of power. It was also something that (depending on the time period) people never talked about. Cato the Elder once joked that he only embraced his wife when he was scared by thunder. Sex was everywhere, it just wasn't discussed in "polite company" or under certain circumstances because the Romans always maintained a very stoic facade in public.
The Greeks, on the other hand, were open and flaunted their sexuality and their proclivities.
Anyway, I will say that the impression about the Japanese that I have is more similar to the Roman idea, but without the taboos (when it comes to pornography at least). Salarymen are busy. Some even sleep in their offices. They have little time to seek sexual release, and so they often indulge in pornography or affairs during the brief periods of respite they get during their hurried work schedules. The various and deviant styles of pornography are an expression of their inadvertant sexual repression.
One girl I was with once told me that I would never have gone far in a Japanese company. I asked, "Why?" Her response was, "You always leave the bar early to be with me when you could be out drinking with your co-workers."
To the Japanese, that timespent with one's co-workers is very important to advancing one's career, and so it takes time away from girlfriends or wives.
As for the female standpoint, I have no idea, although I will admit I've seen a number of pornographic manga drawn by Japanese women, and they are far, far raunchier than anything I've seen done by Japanese men. It seems that there is a definite interest in "shotacon" especially "straight shotacon" among many Japanese women (possibly because young boys are impressionable and the woman can easily control the situation). Compare this to lolicon.
Interestingly enough, amongst the "chans" (i.e. 4chan and its satellite imageboards) there is a definite interest among Western males in "straight shotacon", but it is more like a teenage boyhood fantasy. "Boy, I wish women were like this when
I was twelve," is a common sentiment.
I also happen to know two guys who were "molested" by older girls (teenage and college level) in their preteen years, and they look back on it as awesome experiences, seeing them as sexual awakenings with the girls as "tutors" or "guides". These guys are somewhat proud of these experiences, and cherish them and still have feelings of appreciation and gratitude for the women who "assaulted" them.
Anyway, I've digressed. The point is, the conditions of Japanese society have made their sexuality contrary to their economic and social demands.
As an aside note, I do not partake in any of the aforementioned pornographic activities. There is a magical place called jail that I never wish to visit.