Update Bye-bye mojibake!

[thomas]

After almost 11 months of arduous, mind-numbing clean-up efforts I am very glad (and relieved) to report that the 日本語 section here at the Japan Forum, including all of its subfora, has finally been cleared of mojibake! :emoji_v:

The task - unfortunately entirely manual - not only gave us the chance to get rid of garbled Japanese text, but also to update invalid and broken links. If you happen to find any leftover traces of mojibake, please do report them in this thread.

We will continue to stomp out ���� throughout all other forum sections, although at a less intensive pace.

Thank you for your patience! :

 

[Lomaster]

では、お言葉に甘えてこちらの文字化けをお訴えいたします。

 

[joadbres]

Much gratitude to those who labored to make the fixes. Feel free to identify yourselves here, so that you can be given the credit you deserve.

 

[thomas]

では、お言葉に甘えてこちらの文字化けをお訴えいたします。

De-garbled! :emoji_point_up_2:

I have just discovered some more mojibake in older threads of the Translation section. Tackling these two beauties

• Need a Kanji Translated? Just ask! | Japan Forum (63 pages, 1,559 posts)
• Need a word or phrase translated? | Page 604 | Japan Forum (605 pages, 15,100 posts)

will also require a little more effort.

 

[musicisgood]

De-garbled! :emoji_point_up_2:

I have just discovered some more mojibake in older threads of the Translation section. Tackling these two beauties

• Need a Kanji Translated? Just ask! | Japan Forum (63 pages, 1,559 posts)
• Need a word or phrase translated? | Page 604 | Japan Forum (605 pages, 15,100 posts)

will also require a little more effort.

Not familiar with what is going on here, was it all spam?

 

[Toritoribe]

お疲れさまでした、thomasさん！！:emoji_clap::emoji_clap:

Not familiar with what is going on here, was it all spam?

It's a problem on character encoding.
Mojibake - Wikipedia

 

[thomas]

Not familiar with what is going on here, was it all spam?

When we switched from vBulletin to Xenforo (our current software) in 2014 most of the Japanese-language threads and posts were not properly converted. Our vBulletin installation ran on SJIS, as UTF-8 wasn't supported back in 2002. Various upgrades resulted in a Gordian knot of encryption issues. I invested a lot of time and money in solving these issues, but wasn't satisfied with the results. In the end I decided to go ahead and clean up the garbled text manually, with the kind assistance of @Toritoribe -san and @nekojita (who mysteriously disappeared about a year ago).

お疲れさまでした、thomasさん！！:emoji_clap::emoji_clap:

こちらこそ、サポートをありがとうございました。　:emoji_slight_smile: :emoji_pray:

 

[mdchachi]

ご苦労様でした。

 

[OoTmaster]

I didn't know there was a difference in Unicode encoding. I figured since it was all base 16 they only had 8 bit Unicode. I didn't even know 16 bit Unicode was a thing.

 

[thomas]

I didn't know there was a difference in Unicode encoding. I figured since it was all base 16 they only had 8 bit Unicode. I didn't even know 16 bit Unicode was a thing.

The problem is that PHP doesn't have native Unicode support and had to rely on extensions like mbstring and iconv. Also, UTF-8 only became the default charset as of PHP 5.4.0 if I remember correctly.