I've lived in Japan for two years and been studying it for almost the same amount of time. At the moment I'm probably at the "low intermediate", been getting around 80% for the JLPT level 3 practice tests (if you want to use that as a guide). I'm a fairly slow learner, or rather, I like to take my time
At the moment I can have decent conversations with people, as long as they don't go using words that I don't know, otherwise I'm thrown right off. I can write quite well too - basic stories using my limited vocabulary.
I can read most emails that my Japanese friends send me, as I've got a pretty good dictionary program for Firefox that shows you the meaning of words as soon as you put your mouse over the top of them (it's called Rikaichan for anyone who's interested).
The toughest thing you'll find is learning the vocabulary as, naturally, there's so much of it. Plus you have to be able to automatically decipher all the conjugated verbs. Sometimes someone will say the most basic thing to me but I won't understand it because I didn't recognise the verb in that particular form.
Don't get scared away from kanji, they're not that hard. In fact, I find them one of the easier parts of the language.
As far as your question, I'd estimate about five years of constant study until you'll reach a high level of fluency, but after three you'll be able to say pretty much anything you want, just in your own warped, limited way.