I think in this case you should think of あまり as a softener.
It isn't a one-for-one translation, but its kind of like "not many" vs. "not that many". The difference is in the nuance, but essentially the meaning is the same.
I think your translated sentence still needs a bit of work though. It is understandable, but the syntax is a bit garbled, and you've got a double negative in it, which is confusing.