In the U.S., at least, and presumably in many other countries as well, the Nazi swastika is the only swastika people are familiar with. WWII documentaries are probably some of the most popular documentaries out there, and the Nazis are still villains in movies, books, and video games.
The memory of Nazi Germany is kept alive both purposefully by people saying 'never again', and incidentally just because they make such great fictional villains.
Meanwhile, there's really no other use of swastikas that Americans, at least, are knowledgeable about. Through my own independent reading I'm aware of some, but I don't believe it was ever covered in class. A fair number of well-read Americans do know that the Nazis took the symbol, reversed the 'spin' of it to make it their own 'unique' symbol. (Of course it turns out there are also reverse-swastikas unrelated to the Nazi swastika, but they are fairly rare. I'm not sure if they always were or if they've fallen out of use to avoid being mistaken for a Nazi symbol.)
I think I've come across non-Nazi swastikas maybe two or three times in novels and adventure movies, and pretty much not at all in documentaries, news accounts, or non-fiction books except when I specifically looked information up on the swastika.
I actually have no idea why you have a swastika on your user icon, I just saw that it was the reverse of the Nazi swastika and wondered a bit.
Edit: There is a swastika that is a kanji, but it's reverse of the Nazi one and doesn't mean 'temple', of course. 卍 (まん) ; the only time I've seen it used in the wild was in the manga Bleach.