You don't need to learn all or most compound words and their readings upfront. You can learn them as you encounter them. For most characters, it is enough to learn and memorize the most common on and kun readings (if there are any), and learn special readings or rarer compound words (or compound words with special readings) only as you encounter them.
Also, it is helpful to understand that most two-character compound words are either of the on-on (the majority) or the kun-kun type. This helps with guessing the readings of newly-encountered words.
Finally, note that "ruling pen" is an uncommon English term. I didn't know precisely what it was, and would have looked it up if I needed to know. Similarly, you can and should do this with Japanese words written with kanji.