で indicates the location where the action is done, so it's the location it is shipped from when it's used with おくる.
e.g.
ゆうびんきょくでてがみをおくりました。
I sent a letter at the post office.
The action "to ship/send" is done at Japan, not your house, right? That's why で is wrong there. (Incidentally, から is more common to indicate the location it is shipped from.) に is used to indicate the delivery destination, as mdchachi-san wrote.
By the way, おくる also means "to present/gift (贈る in kanji)" other than "to ship/send (送る in kanji)". That's why I initially interpreted your sentence as "(my friend) presented (the shirt) to me at my home".