は could work as a contrastive marker, so it's not appropriate there.
を or が are sometimes omitted especially in casual conversations. That's just the omission of を, in other words, there is no problem with あの人の名前を、きのう聞いたんですが、忘れてしまいました。
Note that there are cases where no particle is the most common.
e.g.
その服、かわいいね。
You can't use その服は/が here, since は/が unnecessarily emphasizes the meaning, so it's interpreted as "other cloths are not cute", "your shoes are not cute", or even "you are not cute".