That sounds like exactly what happened when I took Japanese as well. Everybody dropped out except the Asian exchange students and a very few others.
These days, I make my living using only Japanese, read Japanese books, and have raised my children as monolingual Japanese speakers....but way back when I FLUNKED the first Japanese course I took.
Stick with it. Start a thread in the Learning Japanese section and ask questions about things you need specific help with.