Japanese national health coverage will cover expenses outside the country the same as within Japan. You need to supply translated documentation of the receipts and explanations, though. Point that out to your HR people or to the ward office/city hall that handles your insurance, if they hesitate.
How much this applies to childbirth expense itself, I don't know, as childbirth is not covered by insurance. Reimbursement, as you seem to know, is a separate thing.