Why AI Struggles with JapaneseStructural Barriers, English-Centric Bias, and the Future of Perfect Translation

Why AI Struggles with Japanese

— Structural Barriers, the "English-Centric" Wall, and the Future of Perfect Translation

Updated October 23, 2026: About a year after publication, this update adds measured Japanese token efficiency, a Japanese benchmark (Nejumi LLM Leaderboard 4), domestic models (LLM-jp-4.1, Rakuten AI 3.0) and Japan's AI Basic Plan, plus a one-year check of the original predictions (3-3). It also corrects an error: the "AI Plus Action Plan" described in the first version as a Japanese government plan is in fact a Chinese government initiative.

---

Section 1: Introduction — The "Strange Japanese" from AI Around Us

1-1. Hook: A Question to Readers and "AI Japanese Quirks"

Have you recently been puzzled by the "strange Japanese" created by AI (artificial intelligence)? This is a phenomenon you encounter more frequently as you use the latest technology. In fact, the following "symptoms" are appearing.

Symptom 1: Delayed Support Apple's addition of Japanese to "Live Translation" in the AirPods Pro OS update (iOS 26.1) made headlines, but compared to English and European languages, the support came later. iOS 26.1 shipped on November 4, 2025, adding Japanese at the same time as Korean, Chinese, and Italian, and only as a beta feature^[1]. Why is Japanese always left behind?

Symptom 2: Bizarre Misreading AI like Google's NotebookLM sometimes reads "和む (nagomu)" as "wawamu" or cannot read "清少納言 (Sei Shōnagon)" at all. When it comes to historical proper nouns, AI may even fall silent.

Symptom 3: Inexplicable Misrecognition When generating Japanese subtitles with AI, they sometimes appear in "Chinese (Simplified or Traditional)" for some reason. This phenomenon was seen in Microsoft's older software and is being repeated even with the latest AI.

Against this background, some YouTube channels have temporarily suspended video uploads to accurately convey Japanese culture. It's understandable to judge that it's premature to have AI that can't read "Sei Shōnagon" talk about the Heian period.

---

1-2. Problem Statement: This Is Not a "Bug" but a "Structural" Issue

These phenomena are not mere programming mistakes. They are the result of a deep mismatch between the "design philosophy" of modern AI technology (especially Large Language Models = LLMs) and the "unique structure" that Japanese possesses.

In this article, we will unravel why AI struggles with Japanese from the perspectives of linguistics, statistics, and AI development. And we will predict when "perfect Japanese AI" will emerge.

---

Section 2: The "Double Barrier" Blocking AI — Why Is Japanese Difficult?

2-1. Overview: The "Two Barriers" for AI

The challenges AI faces when handling Japanese can be classified into the following two categories:

1. The Language Barrier: The structural difficulty inherent in Japanese itself
2. The Data Barrier: The reality that AI training data is biased toward English

People tend to think "because Japanese is special," but in reality, the structural problem of "AI raised in an English-centric environment" is more serious.

---

2-2. The First Barrier: The "Linguistic and Structural" Complexity of Japanese

(A) The Nightmare of "Writing Systems" — Three Types of Characters Coexist

While English is complete with 26 letters of the alphabet, Japanese mixes three systems: "hiragana," "katakana," and "kanji."

When AI converts speech to text, for example, it needs to decide whether the sound "kyō" should be written as "今日," "きょう," or "キョウ." This significantly reduces accuracy.

---

(B) Infinite Branching of "Readings" — Why "和む" Is Read as "wawamu"

Japanese kanji have polyphony (multiple readings), with numerous on-readings and kun-readings existing.

Example: "生" → "sei," "shō," "nama," "ikiru," "umareru," and more.

AI prioritizes readings with high occurrence probability, so based on the statistic that "和 = wa," it misreads "和む" as "wawamu." In other words, AI decomposes it as "和 + む" rather than treating "和む" as a single word.

Historical proper nouns (e.g., "清少納言") have low data occurrence frequency, so they are treated almost as "unknown words" by AI.

---

(C) The Ultimate Challenge: "Ambiguity" and "High Context"

Japanese has a culture of omitting subjects and objects. Please look at the following sentence:

頭が赤い魚を食べる猫。

This sentence has two possible interpretations:

1. What is "red" is the fish
2. What is "red" is the cat

Humans judge by context, but AI can only process both as "probabilistically possible." This is "ambiguity," AI's greatest enemy.

---

2-3. The Second Barrier: "Data Divide (Language Gap)"

(A) AI Is Raised in "English"

AI performance is proportional not to "language difficulty" but to "the amount of training data." However, almost all of that data is in English.

According to a report from Johns Hopkins University, out of approximately 7,000 languages in the world, AI can handle only about 100 languages. Nearly all of GPT-3's training data of approximately 250 billion words was in English.

AI is "born as an English native speaker."

---

(B) Is Japanese a "Low-Resource Language"?

While Japanese has a large speaker population and abundant information on the internet, in AI development, it tends to be treated as "non-English = everyone else."

The government's AI Basic Plan (Phase II), adopted by the Cabinet on July 14, 2026, lists "developing and expanding high-quality Japanese-language data" as a policy measure^[2]. Setting it as a goal is itself an admission that such data is not yet sufficient. In other words, Japanese AI is still in its "growing phase."

---

(C) Case Study: Why Do Japanese Subtitles Become "Chinese"?

AI mistakenly recognizes "kanji = Chinese" because it makes such simplifications in the early stages of learning.

Afterward, it cannot sufficiently learn the "mixed structure of Japanese," resulting in misdetecting Japanese subtitles as "Chinese."

This is the result of the structural barrier + the data barrier working simultaneously.

---

(D) A Less Visible Third Barrier: Japanese Costs More in "Tokens"

LLMs read and write text in units called "tokens." The rules for splitting text (the tokenizer) are often designed around English, so the same content in Japanese tends to use more tokens.

In an independent study published in May 2026, ITmedia (@IT) fed the same content in eight languages to five models and compared input token counts against English^[4].

If Japanese needs 1.48 times as many tokens as English, the article estimates, the same context window holds only about 68% as much information as it would in English. Token counts also translate directly into cost.

There are caveats. Only input tokens were compared; output and reasoning tokens were excluded. The samples were texts the author prepared, and the results are a snapshot from May 2026. A lower ratio to English does not mean a model is cheaper, either: the article notes that Claude Opus 4.7 has a small Japanese penalty but the highest absolute token count in every language^[4].

Chinese, which also uses kanji, averages 1.01x, while Japanese averages 1.48x. This study alone cannot pin down the cause, but the mix of hiragana, katakana, and kanji (the first barrier) and how tokenizer vocabularies are allocated may play a role. The structural barrier and the data barrier show up not only in accuracy but also in price.

---

Section 3: AI's "Thinking" and "Future" — Roadmap to Perfect Japanese

3-1. AI (LLM) Is a "Kindergarten Child" — An Entity That Speaks Through Probability

AI does not understand grammar. When input with "Today is, good...," it simply calculates the probability of what follows and chooses "weather." It does not "understand" like humans do.

In other words, for AI to improve its Japanese means "increasing the accuracy of probability."

---

3-2. Future Prediction: When Will AI Speak "Perfect Japanese"?

(A) Six Months to 1 Year: Eradication of Obvious Misreading Issues like "和む = wawamu," "cannot read Sei Shōnagon," and "Chinese subtitles" are expected to improve within a year through additional training with high-quality data.

(B) 3 to 5 Years: Natural Japanese Generation The first version of this article said that the government's "AI Plus Action Plan" would bring natural Japanese conversation and translation to practical levels around 2027. However, the "AI Plus (AI+)" action is an initiative of China's government (the State Council) with milestones in 2027, 2030, and 2035, not a Japanese government plan^[3]. We apologize for the error. Japan's actual national plan is the AI Basic Plan under the AI Promotion Act (Phase I adopted December 23, 2025; Phase II adopted July 14, 2026)^[2]. It calls for developing Japanese-language data and trustworthy AI, but it sets no target year for when natural Japanese will reach a practical level. The outlook that natural Japanese conversation and translation will become practical in 3 to 5 years remains, but as the author's own forecast rather than a government target.

(C) 5 to 10 Years: Overcoming Ambiguity and Context Understanding High-context problems like "the cat that eats the fish with a red head" require next-generation AI that goes beyond the limitations of probabilistic LLMs. True context understanding will require evolution on a scale of 10 years.

---

3-3. One Year Later: Japanese AI in October 2026 (added October 23, 2026)

(A) A "Report Card" for Japanese Now Exists

Nejumi LLM Leaderboard 4, released by Weights & Biases Japan on August 27, 2025, evaluates models in Japanese across reasoning, expert knowledge (such as JMMLU-Pro), coding and tool use (such as JHumanEval), and safety and reliability (M-IFEval, HalluLens)^[5].

A third party, Qualiteg, compiled the leaderboard data as of October 22, 2026. Part of the result^[6]:

Global frontier models hold the top spots, about 0.15 points ahead of the best Japanese-made open model. The same article says the newest models, including Claude Opus 5.5, GPT-6 Astra, and Claude Fable 5.1, had not yet been evaluated^[6]. I could not read the numbers directly from the official page (nejumi.ai), so this section relies on the third-party compilation. Check the official page for current rankings.

(B) Domestic Models Are Moving Toward "Open Down to the Recipe"

• LLM-jp-4 (National Institute of Informatics, April 3, 2026): Two models, 8B and 32B-A3B (MoE), trained on about 12 trillion tokens. On a Japanese MT-Bench judged by GPT-5.4, NII reports 7.54 for the 8B and 7.82 for the 32B-A3B, above GPT-4o (7.29)^[7].
• LLM-jp-4.1 (September 28, 2026): Three models, 8B, 32B-A3B, and 33B, released under Apache 2.0, with better math and science performance and new support for calling external tools^[8].
• Rakuten AI 3.0 (Rakuten, March 17, 2026): A roughly 700-billion-parameter MoE model released under Apache 2.0, developed with support from the GENIAC program (third term) run by METI and NEDO^[9].
• GENIAC fourth term (June 4, 2026): METI and NEDO selected 16 foundation-model development projects^[10].

Domestic models still trail the top global models on total score, but more of them now publish their training data and methods and can run on your own servers. That is a significant change for companies and researchers who cannot send Japanese data outside.

(C) How Did Last Year's Predictions Hold Up?

• Prediction (A), "obvious misreadings improve within a year": Written Japanese ability can now be compared numerically, as shown above. However, I could not find official announcements or evaluations confirming that text-to-speech misreadings (reading 和む as "wawamu") or Japanese subtitles being misdetected as Chinese have been resolved. Nejumi evaluates text, not reading-aloud accuracy. So this prediction can be called neither a hit nor a miss: it remains unconfirmed.
• Prediction (B): Its basis, the "AI Plus Action Plan," turned out to be a Chinese government initiative, as corrected in 3-2.
• What is new: Apart from accuracy, measurements now show a "cost barrier": Japanese uses more tokens than English and is therefore more expensive (2-3 (D)).

---

Section 4: Conclusion — "Barriers" Are "Fuel for Evolution"

4-1. The Changing Human Side: The Need for "AI Literacy"

When giving instructions to AI, do not omit subjects and avoid ambiguous expressions. By explicitly structuring sentences like "the cat eats the fish with a red head," AI can understand more accurately. This is "language design literacy."

---

4-2. Japanese Evolves AI

The complexity of Japanese is both a "barrier" and a "trial for growth." By learning Japanese, English-centric AI approaches "human-like intelligence" through context understanding.

On the day AI can understand the cultural background of "清少納言 (Sei Shōnagon)," AI should have evolved as humanity's "cultural interpreter."

---

4-3. Closing Message

The future when AI can fully understand Japanese. That is the dawn of an era when culture, history, and emotions nurtured in Japanese can be instantly conveyed to people around the world.

If Japanese is the final challenge for AI, overcoming it is the moment humanity gains "true intelligence." We are now standing at its entrance.

---

References

References

1. [1]Gizmodo Japan, "Live Translation on some AirPods now supports Japanese; iOS 26.1 rolls out" (Nov 2025, in Japanese). Gizmodo Japan ↩
2. [2]Cabinet Office of Japan, "AI Basic Plan" (Phase I adopted Dec 23, 2025; Phase II adopted Jul 14, 2026). The quotation is from the Phase II text, section "Development of trustworthy AI foundation models." Cabinet Office ↩
3. [3]JETRO Business Short News, "China issues opinion on thoroughly implementing the 'AI+' action, strengthening six priority areas" (Sep 3, 2025; State Council issued Aug 26, 2025; in Japanese). JETRO ↩
4. [4]ITmedia @IT, "Independent study: is using AI in Japanese about 1.5 times more expensive? Comparing token efficiency" (May 13, 2026, in Japanese). @IT ↩
5. [5]Weights & Biases Japan, "Nejumi LLM Leaderboard 4 released" (Aug 27, 2025, in Japanese). PR TIMES / Official leaderboard: nejumi.ai ↩
6. [6]Qualiteg Journal, "Japanese LLM Rankings 2026" (Oct 22, 2026 edition; third-party compilation of Nejumi LLM Leaderboard 4 data). Qualiteg ↩
7. [7]National Institute of Informatics, news release on LLM-jp-4 (Apr 3, 2026, in Japanese). NII ↩
8. [8]LLM-jp, "LLM-jp-4.1 series released" (Sep 28, 2026, in Japanese). LLM-jp ↩
9. [9]Rakuten Group press release, "Rakuten AI 3.0" (Mar 17, 2026). Rakuten ↩
10. [10]METI, "GENIAC computing-resource support (fourth term) selections" (Jun 4, 2026, in Japanese). METI ↩