Yes, it's an adverbial usage. For an English equivalent, you could say he's "internationally famous."
世界中(せかいじゅう)で would have roughly the same meaning. ("He's famous across/around the world.")
世界に doesn't work. The location where someone/something is famous takes で, not に. (世界的に is fine because it's not marking a location, it's an adverbial usage.)