Cloa,
You need to talk to a learning disabilities specialist. Is there one in your area? Is there one associated with your school?
You also need to talk to someone at your school. Does your school have a learning disabilities specialist? If not, you may need to talk to someone like the principal. I also feel very strongly you need to talk to your son's teacher, both his present teacher and his future first grade teacher. They have probably seen this before and they probably have specific ideas on how to deal with the problem. (They may also have some bad ideas about this. If so, you need to find out about this right away.)
I feel your learning disabilities specialist, your son's teachers, and you need to get together and come up with a clear plan as to how this is going to be dealt with. Teamwork will be important here. Get the teamwork started as soon as possible.
It doesn't matter if your son feels this is not important. He is not a learning disabilities specialist. Find someone who is.
Another factor: This whole thing may be upsetting to him, and him saying it doesn't matter might just be a defense mechanism. No doubt a lot of compassion, understanding, and gentle encouragement from you will help a great deal.