I would say that getting a job in the IT sector should be easier for a foreigner in Japan. After majoring in math, the options may be mainly in teaching. And, if you go for a teaching job without native level Japanese, you will have limited possibilities of getting a job at an international school. On the other side, with your interest in mathematics background, you may land an excellent job in niche areas like data engineer or data science if you add an information technology qualification. You may find some good information for an IT career in "How to get an Information Technology Job in Japan".
However, you need to see if you have a passion for information technology because success ultimately comes from love and interest. However, on the other side, I feel that a person interested in mathematics should easily find interest in information technology.
I wish you all the very best.