Your reason for your interest and visit sounds like a very good start, albeit a bit vague still. Plenty of time to flesh it out as you look more into things.
I assume by your comment about losing 1.5 years of college you mean that you wouldn't go to Japan if you had to lose all of that? Hmm... if so, it would show me that you aren't that serious. It took me 8 years to graduate from college with my two degrees, each year was of value, classes or not. There is no such thing as wasted time if you are spending it doing something. If you have a goal, and are working towards it, that is the true purpose. Right now, you're in community college which is a great place to start in my opinion. I spent almost 4 years in community college before transferring. The fact you've only gone for 1.5 years means it should be quite easy to turn your grades around and get them up to at LEAST 3.4 by the time you finish college if you work at it.
I also didn't know what I truly wanted to do till I was 23 years old. Try things, follow your interests. If you are interested in Japan, the hell with whatever one else says or tells you, go for it full blast and try it out. That is the purpose of college I believe. This is missing in Japan and other countries, and is probably the most shameful part of university study in Asia.
What if you spend the next 3 years studying in your home country? That's fine! You can learn more about Japan, follow your interest, up your grades and of course, learn the language (which is a never ending journey itself). After you transfer to a college here, you can easily do a study abroad program with full US financing to almost any college that is connected in Japan for a year, just check the colleges you plan to transfer to. By that time you may know more specifically what interests you (rather that JUST a general "Japan").
Explore. You have the chance now, use it. You'll regret it for the rest of your life if you don't. It is hard work, but NOTHING easy is worth squat. You may think 1.5 years is a lot of work and a long time but it is NOTHING, so don't fret over it.
Just so you know, it took me 1.5 years after graduating and HARD work studying like a maniac, rejection letters, and tons of applications before I got into graduate school in Japan. I'm now accepted as a grad student at the University of Tokyo and start around October.
Simply put, everything is up to you and your choices that you make every single day and how you spend your time, and don't lie to yourself that you're past a point of no-return.