I hate these discussions because the guy explaining the rules ends up sounding like a finger-wagging Karen, and the guy on the other end is
ALWAYS able to say, "
I know a guy who gamed the system for years and still got away with it without any consequences 
".
Anyway, since I'm the one who brought all this up;
Immigration asks you, every time you leave, how long you intend to be outside of Japan (via the orange re-entry cards). These cards are for people who intend to return within a year. So Immigration checks this once when you leave, and they check it once again when you return. I don't know what you use if you intend to be away for longer than a year. But anyway, it isn't the orange cards.
If the OP is gone for over three years...well, good luck to the OP.
For sure Japanese citizens can leave and then be relieved of tax obligations while they live overseas. But the whole point of the
residence rules for foreigners is that Japan actually assumes you to
reside in Japan. The permanent resident visa isn't for people who at one time lived in Japan,
but then left yet still would like the option to live in Japan at some point in the future. When you leave Japan with the intent to reside somewhere else, you aren't a resident of Japan anymore. And if you leave Japan while you are undecided about your future, Japan is saying to you, "figure it out within the year".
"But what if *
insert unusual circumstance* happens while you are outside of Japan?"
"Surely you can't be expected to follow the rules when *
unusual circumstance* happens."
The MOJ already answers some of this in #149, #152, and #161 on their site. The short version is: you've got a year to sort out your affairs. Figure it out within the year.
www.moj.go.jp