If you are American and want to get married in Japan, you need to get a form (Affadavit of Competency to Marry
http://japan.usembassy.gov/pdfs/wwwf7114.pdf )from the U.S. embassy to swear that you are not already married. You sign it there in front of the embassy official, and he countersigns and stamps it to make it official.
You take that and your J boyfriend's copy of his family register (koseki) to the ward office or city hall near your home, and in 10 minutes you are officially married, as long as you have signatures from 2 other people. You don't even get a marriage license unless you want to pay extra for one (400 yen for the simple one, about 1200 yen for a fancier one). Any ceremony you may want is extra and an entirely different affair between you and hubby to work out.
After you get married, you will have to amend hubby's koseki to show that you are part of it.
Then, you take the necessary documents to immigration (including the COE) and file for a spousal visa. You do not have to leave the country for that.
COE form and other info can be found at this official immigration site:
Immigration Services Agency of Japan
The government has been tightening up on spousal visas lately, so expect a few serious questions to prove you are not in a marriage of convenience. That may include you supplying nonreturnable pictures of you two. Your hubby will also have to complete a Japanese essay describing how you met and your courtship.
Want more official stuff?
Here's what the Sapporo office of the U.S. embassy says about getting married and a spousal visa. 2 links
http://japan.usembassy.gov/e/acs/tacs-7114.html
http://japan.usembassy.gov/e/acs/tacs-7114a.html
By the way, if you live near a different embassy/consulate, go to this link instead, and click on the link for the maps. You will go to a site that has all the above info, too, including a map of that specific embassy/consulate.
http://japan.usembassy.gov/e/tvisa-main.html