IMO, nonstop population growth is about as wrongheaded as the Iraq War or market fundamentalism. And there's something wrong about capitalism's need for constant growth.
Population growth can't be infinite. A growing population will have its own problems-economic, social and ecological, and faith in future technology to solve all such problems is unrealistic. Otoh, having a smaller population can be beneficial. Japan has a pop. of 123 million although it's just the size of California, which has a population of only 35 million. Despite that smaller population, California has what would be the 8th largest economy in the world if it was a country. This is just a simple comparison but I think it shows that Japan shouldn't be afraid of a smaller pop. Japan could still have a respectable economy. There would be dislocation problems along the way but I think the country would adjust. Some of the fiscal deficit may be relieved by the extremely high inheritance tax rate that will transfer wealth from households to the government as the population shrinks, and standard of living may rise if GDP doesn't shrink as fast as the populaton does. Cities would be less congested and tourist spots less crowded. Japan could end up as the Switzerland of Asia, IF it resolves its differences with neighbouring countries.