I haven't "mastered" Japanese and never will. Just the other day, I had to ask for help on this forum concerning a simple dialogue line from a game, despite me having passed the JLPT N1 and the line being completely obvious for native speakers. I, too, once thought my knowledge of Japanese grammar was pretty good - but it didn't take long to realize that this was only because I wasn't even aware of what I didn't know.
What I can say though, is that Genki is definitely not enough. The books have the word "elementary" on their cover for a reason. They only explain the absolute basics which aren't enough to understand even a simple anime series.
I'd recommend to first spend more time on textbooks (to build grammar knowledge) and reading (to build vocabulary). Looking up words while reading is much easier and much less frustrating than repeatedly rewinding a video, trying to catch what a character is saying. Then, when you start doing listening practice, you'll be able to quickly recognize spoken words that you've never heard before, but did read.