I spoke of courtesy, not legalities.
And legalities may very well be involved, since whether a use is commercial or not is irrelevant to copyright violation.
In the U.S. at least, section 110(1) allows the performance of musical (and other) works during a classroom session. The time the students are working on the project is allowed by fair use, as it's permissible to make derivative works like this in private, it's only a copyright violation to distribute them.
That's probably enough on its own, but given the likelihood that there is music played at school outside of the classroom, the school also very likely has a school would have an ASCAP blanket license.
Now, I've never lived in Norway or had reason to learn anything about Norweigan law but I would expect that they have similar arrangements. Most Berne convention signatories have extremely similar laws.
In any case, it's really the responsibility of the instructor to make sure everything is legal before assigning making a music video as a class project.
If the group wanted to distribute their video outside of class (by e.g., uploading it to youtube) that's when they need to be talking to the copyright holder. I don't know if in Japan that's more likely to be the artist or the record label.
For the U.S. model, there's a guide here
https://www.american.edu/library/documents/upload/Copyright_for_Teaching.pdf, but again, details could differ in Norway.
Well, and the OP didn't actually use the word 'classroom', but 'we have an experimental music video project' sounds like a classroom assignment. If it's not in a class setting, then they would need to check on copyright arrangements before sharing the video with anyone outside the project team. Blanket licenses are likely in place to cover club activities inside academic institutions though.