Hi Atashi,
Well, I only started learning Japanese a short time ago, so... I would encourage, even though writing is difficult! In fact, I haven't yet started to learn kanji. I have learned only hiragana. There are 2 phonetic alphabets, hiragana and katakana. Kanji are symbols, there are many, thousands!
It's possible to learn to speak/listen but not to read/write, I would think, because when I consider it, there are some English people who find it difficult to read and write their language, so... For starting, I am concentrating more on speaking, but also learning the hiragana. The phonetic alphabets are not hard, and you can make yourself understood in writing if you know them. Personally, I think speaking is most useful if you are going to Japan, but... some day, I hope to learn kanji, too! Of course, if you go to live in Japan, kanji would be necessary.
In speaking, I find grammer and pronunciation not so hard, so, go for it!
Good luck, I hope you enjoy learning!