No. To me it is saying that you studied an hour at a time. Actually I don't think anyone would say they studied like that. maybe these....I studied a long time. I studied a lot. I studied for hours. I studied for many hours.
Oh I just thought if you meant that you studied for certain amout of time such as 4 hours. It could be...I studied for four long hours.
"I studied by the hour" sounds more like studying off and on, here and there, for an hour at a time. But then you'd probably add "I studied by the hour when I wasn't working", or something like that.
It's a very underused phrase but I agree wholeheartedly with Goldie here as well as your dictionary definition. The term "by the hour" you can think of in contrast to "study by the minute" which implies a highly sporadic and inconsistent effort.