Hmm... hard to believe I haven't replied to this thread yet so here goes...
I consider myself fluent in conversational Japanese and it is something I pride myself on having learnt "the hard way" so to speak. Prior to my arrival in Japan I had virtually no direct exposure to the language (had not taken classes or bought audio tapes, etc.). I am a firm believer of the age-old adage "When in Rome..." therefore one of my first priorities was to immerse myself in the language as much as possible. This was no easy task and gaining even a basic comprehension was not something I was able to do early on. Only after acclimating to my environment and making a dedicated effort to commit to memory as many vocabulary and phrases as possible was I then on my way to gaining a level of comprehension.
My daily routine those first six months or so consisted of me carrying a small notepad with me where ever I went. As I would hear a common word or phrase repeated often, I would jot it down phonetically into my pad as best I could for future reference. Then, when I could afford the time (usually in-between classes at work), I would leisurely look up the meanings to everything I had written down. Soon I began forming lists and these I would eventually commit to memory as best I could. Interestingly enough, the more comfortable with the language I became, the more I wanted to learn. It was at this point then that I actually began taking classes from some of my private adult students (language exchange) to further polish my conversational skills.
For the entire duration of my stay, I made an active effort to use Japanese as much as possible and surround myself with the language. I did not actively seek out any native English speakers although having been in Sapporo I certainly could have had the opportunity. Nevertheless, I had met several native English speakers who had been there longer than me yet whose mastery of the Japanese language was considerably lacking as they tended to surround themselves with English speaking friends ever since their arrival. I resolved myself to not fall into the same trap so to speak.
At some point, i'm not sure when really... but everything just seemed to "click" and I found myself very comfortable using Japanese as often and as freely as possible. It wasn't until I was seated next to an elderly woman when I was returning home to the States for the Holidays that this realization finally dawned on me. This very kind Grandmother happened to be on her way to visit her relatives in California and she did not speak a lick of English. My ability to maintain a conversation with her without repeating myself and sounding redundant for the entire duration of that flight is still one of those personal milestones embedded in my memory.
As it stands, my Japanese has gotten a bit rusty having been away from Japan for a few years now. Although as I look forward to graduation this Fall, my plans to return to Japan soon have reinvigorated my efforts to pick up where I had left off. My advice to any beginners is this: IMMERSE yourself as much as humanly possible. Yes, you will get frustrated but don't give up! There IS a light at the end of the tunnel and comprehension is its name. Believe it or not, through sheer exposure, you WILL learn even if it is the "hard way". And ultimately, in the end it makes it all the more worthwhile. Keep practicing!