何の~ (なんの) seems to be asking to identify a specific item out of a set.
どんな~ seems to be asking for a more general description
e.g. 何の本? might be answered with the name of a book or a generic classification, どんな本? by a description of it.
So 何の楽器 if you want to ask which instrument somebody can play, どんな楽器 if you want to ask them "and what kind of instrument is a flugelhorn?"