My husband and I speak only English together too. For one, because I don't know much Japanese and secondly, he has to use English at his job everyday, so it helps him there. As for our children, he uses only Japanese when speaking to them and I only use English with them.