Furigana is a crutch. We use crutches to help us overcome our weaknesses and shortcomings, but they are to be cast aside when we gain strength.
Once you learn to read along well without them, they can actually be distracting and slow you down as you're reading.
A problem with furigana in many books I have read is that they are used haphazardly. Many times I have encountered the furigana applied to not the first instance of an uncommon or hard-to-read word, but rather to perhaps the second or third instance of it. Conscientious editors/publishers will tend to apply the furigana to the first instance of such words in each chapter of a book where it may appear. Sometimes it may appear only the first time the word appears in the book. Seldom is it applied to each and every instance.
The usage of furigana is by no means universal. It sort of depends on the target audience, I suppose. I am currently concurrently reading 仏教聖典 and 座標軸としての仏教学. The first one uses the conscientious approach: applying furigana to the first instance per chapter for not-so-common words and writes lots of stuff in hiragana rather than kanji. The second one uses furigana exceedingly sparingly. I'd prefer an approach somewhere in between the two.
What will really slow you down, if you want my opinion on it, is developing a habit of looking up everything in a dictionary as you go.
Whether I think of an isolated kanji as on or kun reading depends on whether the kanji has a kun reading or not. Since kun readings are Japanese words on their own the natural thing to do would be to think of the kun reading. By way of illustration, imagine I asked, "When you see in isolation the word 'book', what do you think of? A book, a bookshelf, or a bookstore?"