Lots of good advice from all the above posters. Regarding your current mindset: I suggest looking at this a different way. You talk about sticking around too long, but as Mike says, Japan will still be here in three years. If your wife cracks the books and actually graduates in two years, you only have 24 months to start getting your act together; and that means getting your degrees and job prospects in order, getting your finances in order, getting your visa requirements in order, getting your Japanese language foundation (maybe starting by taking the Japanese Language Proficiency exam while still in the States?), etc.... 24 months is nothing.
Resist the urge to just go to Japan for the experience of it. Find out what makes you and your wife valuable to a US company or school, and then that will most likely be your selling point for you here in Japan as well. If it takes you 24 or 36 months to nail that down, it will be time well spent. In the interim, you have plenty of time to research jobs and visas, and research living conditions here in Japan, study the language, etc...