Yes. Japan will tax ROTH IRAs and 401ks. 401ks are taxed on distribution in US too. ROTH IRAs are not taxed on distribution in the US but are in Japan. There's some question on how they are taxed, some file it as pensions (a lower tax rate) and some say that is incorrect.
Your first five years in Japan or until you take permanent residency you are not taxed on global income only income coming into Japan so you have 5 years to figure it out (if you do not take permanent residency sooner). Social security is taxed as a pension. While one has to file US taxes typically Japanese taxes are higher so foreign tax credits normally eliminate US taxes except in some edge cases. There is some arguments on how retirement accounts like ROTH IRAs and 401ks would be correctly taxed on some forums I am on. So yes taxes are the biggest deal as you say. When I look at cost of housing in Japan especially in smaller cities, when I look at cost of health care in Japan vs US even my father on medicare (he is 80s) is terrified of the doctor due to medical experiences he had with pneumonia and appendicitis, when I look at the cost of eating out, the cost of many groceries (Japanese diet so lots of imported stuff here) with a big exception of fruits I still believe it is a win in Japan financially. Your mileage may vary.
To make sure you know 401ks are taxed in US on distribution as are IRAs but not ROTH IRAs in US (but yes in Japan).
One tax question is weather these should be taxed on distribution in Japan or with the holdings, people argue on it and often get away with distribution. In those 5 years one is not taxed on global income it may be necessary to make some moves, certainly more research is needed. The worst part is the confusion due to no clear answers but perhaps that gives many wiggle room and relief.
I also have a pension (something no company offers any more) from that first tech company.
Oh yea one other trick on US taxes to make sure they stay low is to not have your wife give up her green card at first so you file taxes jointly if needed. The green card will eventually be revoked if deemed not to be abandoned. Green card holders like US citizens must file taxes anywhere in the world until they fill out a form to revoke it the card being expired does not mean one does not have to file US taxes. Fun right? Just like taking permanent residency too soon can increase your taxes on global assets.