There are kanji that can be used, too, although this is not so common in modern usage.
I don't think I agree with you, Jimmy Seal. Granted it's been a long time since I formally this stuff, but I don't think it's incorrect to have 4ケ月 and the others you put. I do think, however, that they are not nearly so common, and yeah, a bit strange perhaps.
Personally, I always use 4ヶ月 except when I change it for stylistic purposes.