Go for it. I'd be interested in your views on the matter as well. I have a two year old daughter that I'm trying to teach English but it's not as easy as I had invisioned to say the least.
As I see it, when it comes to kids that little, we can't "teach" them a language. We can facilitate to one degree or another their acquiring a language, but it is up to them whether they choose to acquire it or not.
When it comes to language acquisition little kids are the ultimate cold-blooded pragmatists. If they sense a
need for picking up a language, they will. If they don't, they won't. My children knew I understood Japanese, heard me speaking Japanese, and saw no point in acquiring what in their eyes was just some strange gibberish their father could speak when he felt like it. Trying to explain the benefits of learning a language useful around the world falls on deaf ears when the ears are attached to people who don't yet have any notion that the world extends more than a few blocks from their house.
When my kids were about 3 and about 1 and a half we went to the US to live for about a year and a half. The kids switched from monolingual Japanese to monolingual English in just a few months. My son picked up English at preschool and his sister then picked it up from him. Within about three months of moving back to Japan they were monolingual Japanese speakers again and have remained so until the present (now 17 and 15). Neither of them has any recollection that there was ever a time they spoke English.
The number one annoying thing that parents in a situation such as mine have to hear is the stereotypical gushing about ハーフがかわいい!! The number two annoying thing is people admonishing us for not "teaching" our children English, followed with a list of the benefits and several anecdotes of friends of friends they know whose children are bilingual or trilingual or whatever. These people have never been in a position to do this themselves....no no no....but they know all about it.
My kids were born here. They have grown up here, attending regular neighborhood schools. They will in all likelihood live their lives here. It strikes me as natural and appropriate that Japanese should be their first language. They are required to study English from junior high, but to what degree they decide to apply themselves and make practical use of it is entirely up to them.