You have to understand that manga (and anime) are simply a medium (i.e. animation and comic books) in Japan and that there can be different degrees of being "a fan".
This is like saying "How are cartoon and comic book fans seen in the U.S. and Canada?"
If you're talking about someone who enjoys the Simpsons/South Park/Family Guy/Rick and Morty/whatever, or likes to read Marvel Comics or graphic novels or whatnot, that's probably going to be seen a as a fairly normal hobby/interest.
If they're obsessed with some cartoon or comic, have action figures or posters or whatever all over, and that's all they ever talk about, that's probably going to be seen as obsessive/nerdy/a bit weird.
Just like your questions on the language, you really need to understand that there is nuance to things like this. There's no one single definition of "an anime/manga fan", and there is no one way in which any given "anime/manga fan" will be perceived by _all_ other people in Japan.