The gist of it is that butter is produced from raw milk and higher priority is given to the production of cheese and other dairy products, so if there's a shortage of raw milk production, butter production gets squeezed first. Recent falls in production due to elderly farmers leaving the business and high tariffs on imported dairy products (have you ever seen New Zealand butter in Japan?) have exacerbated this situation.
If you're thinking of doing any baking over the Christmas season, then it's a good idea to get your butter a month or two before because there is very little around at that time of year.
If you have any Seiyu supermarkets in your area, I find them to be quite reliable for butter.
Japan imported 8,000t of butter (12% of the total amount of consumption) in 2017.
We have a thread about the topic. To make a long story short, Japan limits importing butter to protect Japanese farmers. Butter | Japan Forum