に would sound odd here. As Toritoribe-san explained, で is the location of the action 殺してしまっている, not the location of the "possibility".
The 可能性 is not what exists in the (abstract) place described as 私の記憶がない部分.
It's something the speaker is consciously aware of and considering now.