Many gyms do not accept people with tattoo, it doesn't matter whether the person is Japanese or non-Japanese.
I've checked on the net and found many similar questions - "is there any gym who allows people with tats?" - posted by Japanese who have tattoo.
I also found that some people say Central and KONAMI are OK, though I'm not 100% sure and it might depend on each branch.
Central Sports (they have 35 clubs in Tokyo and 19 in Kanagawa.)
KONAMI Sports Club (42 in Tokyo, 22 in Kanagawa)
Both of the above are mentioned by two different persons saying "I've got tats and am member at this club, so far I have no problem, was never pointed out by the staff." etc.