I've been stopped on my bike 2 times in the span of a year and a bit. The first time, I was riding home from work (but I'd bought a fancy mountain bike from a friend who was leaving Japan) and was pursued by a bike gang of cops saying 'sumimasen!'. They asked me for the bike registration, and I explained to them that I'd bought it off someone else and hadn't had the time to go register it.
The lady wrote down my bike model/licence? number and basically told me that I should go and get the biker registered to me right away. They were very courteous the whole time and I was courteous back, so 5 minutes later I was on my way home with no trouble.
The third time, it was kind of my fault- myself and a Japanese friend were both riding on a single bike to pick up some late night snacks (2am) from the convenience store and stupidly rode in front of the Koban

. So the bike gang pursued us again, 2 junior officers and 1 old guy. Basically the junior officers were nice and polite and saying that we shouldn't do that, especially in front of the Koban. The old guy was a crass ***, and I couldn't understand about half of what he said, so I ended up kind of ignoring him. My poor buddy got lots of flak though- "You should teach people from other countries about Japan's law! What if little kids saw you riding 2 to a bike, they would do it too!" They asked my for my foreigner registration card - as the store was just out 3 minutes from my apartment, I'd not brought it, so I told them that. They also asked and other questions like "How long are you staying in Japan"- I told them that I was leaving the country soon (which was actually a couple months, but it's all relative I guess). With that they kind of relented and let us go on our way, warning us not to do it again and telling me to carry my passport/card at all time.
All in all, I don't think the experiences with them were too bad, they were fairly nice and didn't go on power trips or anything. Pretty much they were doing their jobs. I think if you are cordial with them and explain the situation properly, they will respond accordingly. I too have heard of them stopping some of my Japanese friends on bikes in similar manners- if anything its kind of just like a checkstop.