I surfed around and found the following, which says basically there are situations where the third party's right to information concerning the employee's health outweights the employee's right to privacy, and in this case your employer may provide your health information to a third party.
The situation below describes someone in the health care profession, so your situation may not be analagous to this. If you are a teacher, and you are getting sent to a school from a dispatch agency, I can see how the dispatch agency might have a compelling reason to verify to the school that you are fit to work there.
派遣労働者の健康診断について - 『日本の人事部』
Guidelines from the MInistry of Health, Labour and Welfare are somewhat vague.
http://www.mhlw.go.jp/new-info/kobetu/roudou/gyousei/anzen/dl/091130-1h.pdf
I think if you are positioning yourself for a legal fight you will find it tough going. Better to resolve this some other way, or move to a different company.