作家 > Writer, as in, "I want to be a writer".
著作 > Author. May be the author of a self-written book, may be the author of a book that is derived from the works of other people (textbook, dictionary, encyclopedia, etc...).
作者 > Author of a book, novel, etc...
作家 is a generic term for writer - or could be any artist who creates something, but by itself it generally means writer. 著者 and 作者 are both used to identify a work with a particular person, and start to take on a more specific meaning, including legal meanings. 著作権 for example is "copyright", and so the 著作者 will be the copyright holder.
The words are not interchangeable, but there are similarities.