yeah, a grammarian would probably mark it as incorrect, and people who had it beaten into them as students might flinch, but no one would wonder what you meant. And you'll hear a lot of native speakers making this mistake.
It's like conbini keigo; at what point is it no longer considered "wrong?"