I'd say Japanese is very hard to learn. And by learn I mean learn properly.
Just today in class we were reading an interesting piece that discussed the follies Japanese presents for its learners.
Japanese has three main elements: Japanese itself (和語）wago, Chinese (漢語) kango and various others (mostly but not limited to English) loosely grouped as (外来語) gairaigo. Linguistically speaking, it's said that it's possible to carry on an every day conversation in English and many other Latin-based languages with a vocabulary comprised of as few as 2000 words. With Japanese this is not the case, in fact the average Japanese grade-schooler has a vocabulary of 6000 words. This isn't light years ahead of other linguistic groups/cultures or anything but its quite high and more importantly points out the 'technical requirements' of speaking Japanese.
Another statistic the article mentioned was that to read 80% of a Japanese newspaper you need to know around 10000 words. That varies of course depending on the complexity of the content but again, on average, statistically, this is the general criteria. With English I think it was something like around 6000 words and you're set.
In English there are many synonyms but they're not always used so much, whereas in Japanese there are many synonyms and they get used a lot more. Japanese culture has remained essentially unmolested by exterior meddling/colonization and its history can be thought of as layered, where old traditions and disappearing customs manage to retain a certain level of permanence since nothing is carelessly tossed away or trampled by outsiders. So even old and outdated words find daily application in Japanese culture.
On example the article mentioned was the idea of canceling something. There's キャンセル (cancel) which is used for canceling things like dinner reservations. Then there's 解約 (kaiyaku) used for canceling bank transactions and other important or business related things. Then there's 取り消す (torikesu) whose natural application I can't recall but also essentially means cancel or literally translated 'pick up and extinguish' or what have you.
Anyway I'm not trying to sound smart here this is just what the article was saying but as it was written by a Japanese person I'm inclined to believe it quite readily.
There are many examples like the above. Japanese is a very high context language syntactically and along with that comes a very rigid set of nuances where words of similar meaning, so similar that they'd pop up in an electronic dictionary interchangeably, end up actually having very specific applications.
So to make a long story short, sure you can pound a phrase book, learn your kana and pick up a few kanji and get away with saying 'I can speak Japanese, it's not so hard', but speaking Japanese properly is really hard and not something casual learners would probably bother doing.
