awesome-japanese-nlp-resources

A curated list of resources dedicated to Python libraries, llms, dictionaries, and corpora of NLP for Japanese

• Listed information on 890 GitHub repositories
• Listed information on 454 Hugging Face repositories (models and datasets)
• Listed information on 156 multilingual libraries, models, and datasets that also support Japanese

English | 日本語 (Japanese) | 繁體中文 (Chinese) | 简体中文 (Chinese)

🎉 The latest additions

Python

• natural-japanese-writing - An agent skill for Claude Code, Codex and Cursor that rewrites AI-sounding Japanese into natural Japanese without changing the facts. 59-item checklist across six layers, plus an automated checker.
• TextToVoicevox - Free offline desktop app (Windows/macOS) that reads Japanese PDFs, images, EPUB and Aozora Bunko aloud with VOICEVOX voices (Zundamon etc.), with SRT subtitles. ／ PDF・画像・青空文庫の日本語を VOICEVOX（ずんだもん等）で朗読・音声化する無料デスクトップアプリ。オフラインOCR・行ごとの話者切替・字幕出力。
• natural-japanese - 仕事の日本語を、読みやすくわかりやすく書く・直すための Agent Skill です。

_Updated on Oct 20, 2026_

Claude Code & Codex Plugin

Search, discover, and track Japanese NLP resources directly from Claude Code or Codex using the awesome-japanese-nlp-resources plugin. Both tools use the same skills and the same bundled data.

Claude Code:

# Add the marketplace and install
/plugin marketplace add taishi-i/awesome-japanese-nlp-resources
/plugin install awesome-japanese-nlp-resources@awesome-japanese-nlp-resources
/reload-plugins

Codex (run in your terminal, then start a new Codex session):

codex plugin marketplace add taishi-i/awesome-japanese-nlp-resources
codex plugin add awesome-japanese-nlp-resources@awesome-japanese-nlp-resources

The plugin ships four skills:

In Codex, call the same skills with $ instead of / (e.g. $awesome-japanese-nlp-resources:search <query>).

All skills detect the query language and respond in kind — English by default, Japanese when the query contains Japanese characters.

`search` example:

/awesome-japanese-nlp-resources:search What tutorials are recommended for beginners learning Japanese NLP?

Output includes a Use-case Selection Guide table:

For full documentation, see the plugin README.

Contents

• Hugging Face
• Models
• Datasets
• Python library
• Morphology analysis
• Parsing
• Converter
• Preprocessor
• Sentence splitter
• Sentiment analysis
• Machine translation
• Named entity recognition
• OCR
• Tool for pretrained models
• Others
• C++
• Morphology analysis
• Parsing
• Others
• Rust crate
• Morphology analysis
• Converter
• Search engine library
• Others
• JavaScript
• Morphology analysis
• Converter
• Others
• Go
• Morphology analysis
• Others
• Java
• Morphology analysis
• Others
• Pretrained model
• Word2Vec
• Transformer based models
• ChatGPT
• Dictionary and IME
• Corpus
• Part-of-speech tagging / Named entity recognition
• Parallel corpus
• Dialog corpus
• Others
• Tutorial
• Research summary
• Reference
• Contributors

Python library

Morphology analysis

Libraries that split Japanese text into words or morphemes and assign part-of-speech and base forms

• sudachi.rs - SudachiPy 0.6* and above are developed as Sudachi.rs.
• Janome - Japanese morphological analysis engine written in pure Python
• mecab-python3 - mecab-python. you can find original version here:http://taku910.github.io/mecab/
• mecab - This repository is for building Windows 64-bit MeCab binary and improving MeCab Python binding.
• fugashi - A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.
• nagisa - A Japanese tokenizer based on recurrent neural networks
• pyknp - A Python Module for JUMAN++/KNP
• Mykytea-python - Python wrapper for KyTea
• konoha - Konoha: Simple wrapper of Japanese Tokenizers
• natto-py - natto-py combines the Python programming language with MeCab, the part-of-speech and morphological analyzer for the Japanese language.
• rakutenma-python - Rakuten MA (Python version)
• python-vaporetto - Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.
• dango - An easy to use tokenizer for Japanese text, aimed at language learners and non-linguists
• rhoknp - Yet another Python binding for Juman++/KNP
• python-vibrato - Viterbi-based accelerated tokenizer (Python wrapper)
• jagger-python - Python binding for Jagger(C++ implementation of Pattern-based Japanese Morphological Analyzer)
• Mecari - Mecari (Japanese Morphological Analysis with Graph Neural Networks)

Parsing

Libraries that analyze syntactic and dependency structures of Japanese sentences

• ginza - A Japanese NLP Library using spaCy as framework based on Universal Dependencies
• cabocha - Yet Another Japanese Dependency Structure Analyzer
• UniDic2UD - Tokenizer POS-tagger Lemmatizer and Dependency-parser for modern and contemporary Japanese
• camphr - Camphr - NLP libary for creating pipeline components
• SuPar-UniDic - Tokenizer POS-tagger Lemmatizer and Dependency-parser for modern and contemporary Japanese with BERT models
• depccg - A* CCG Parser with a Supertag and Dependency Factored Model
• bertknp - A Japanese dependency parser based on BERT
• esupar - Tokenizer POS-Tagger and Dependency-parser with BERT/RoBERTa/DeBERTa models for Japanese and other languages
• yomikata - Heteronym disambiguation library using a fine-tuned BERT model.
• jdepp-python - Python binding for J.DepP(C++ implementation of Japanese Dependency Parsers)
• lightblue - A CCG parser for Japanese with DTS-representations
• natsume-simple - natsume-simpleは日本語の係り受け関係検索システム
• jdeppy - Python wrapper for J.DepP, fast Japanese Dependency Parser

Converter

Libraries that convert between character types such as kana, romaji, and full-width/half-width forms

• pykakasi - Lightweight converter from Japanese Kana-kanji sentences into Kana-Roman.
• cutlet - Japanese to romaji converter in Python
• alphabet2kana - Convert English alphabet to Katakana
• Convert-Numbers-to-Japanese - Converts Arabic numerals, or 'western' style numbers, to a Japanese context.
• mozcpy - Mozc for Python: Kana-Kanji converter
• jamorasep - Japanese text parser to separate Hiragana/Katakana string into morae (syllables).
• text2phoneme - 日本語文を音素列へ変換するスクリプト
• jntajis-python - A fast character conversion and transliteration library based on the scheme defined for Japan National Tax Agency (国税庁) 's
• wiredify - Convert japanese kana from ba-bi-bu-be-bo into va-vi-vu-ve-vo
• mecab-text-cleaner - Simple Python package (CLI/Python API) for getting japanese readings (yomigana) and accents using MeCab.
• pynormalizenumexp - 数量表現や時間表現の抽出・正規化を行うNormalizeNumexpのPython実装
• Jusho - Easy wrapper for the postal code data of Japan
• yurenizer - Japanese text normalizer that resolves spelling inconsistencies. （日本語表記揺れ解消ツール）
• e2k - A tool for automatic English to Katakana conversion
• alkana.py - A tool to get the katakana reading of an alphabetical string.
• englishtokanaconverter - 英語文字列をカタカナに変換するプログラム
• kanjiconv - Kanji Converter to Hiragana, Katakana, Roman alphabet.
• kanjize - Kanjize(カンジャイズ): Easy converter between Kanji-Number and Integer

Preprocessor

Libraries that normalize and clean text before analysis

• neologdn - Japanese text normalizer for mecab-neologd
• jaconv - Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku
• mojimoji - A fast converter between Japanese hankaku and zenkaku characters
• text-cleaning - A powerful text cleaner for Japanese web texts
• HojiChar - 複数の前処理を構成して管理するテキスト前処理ツール
• utsuho - Utsuho is a Python module that facilitates bidirectional conversion between half-width katakana and full-width katakana in Japanese.
• python-habachen - Yet Another Fast Japanese String Converter
• kairyou - Quickly preprocesses Japanese text using NLP/NER from SpaCy for Japanese translation or other NLP tasks.

Sentence splitter

Libraries that automatically detect sentence boundaries and split text

• Bunkai - Sentence boundary disambiguation tool for Japanese texts (日本語文境界判定器)
• japanese-sentence-breaker - Japanese Sentence Breaker
• sengiri - Yet another sentence-level tokenizer for the Japanese text
• budoux - Standalone. Small. Language-neutral. BudouX is the successor to Budou, the machine learning powered line break organizer tool.
• ja_sentence_segmenter - japanese sentence segmentation library for python
• hasami - A tool to perform sentence segmentation on Japanese text
• kuzukiri - Japanese Text Segmenter for Python written in Rust
• ja-senter-benchmark - Comparison of Japanese Sentence Segmentation Tools
• fast-bunkai - Japanese sentence splitting(日本語文境界判定器), 40–250× faster via a Rust-accelerated Python library with near-perfect API compatibility with megagonlabs/bunkai.

Sentiment analysis

Libraries that detect emotions or polarity in text

• oseti - Dictionary based Sentiment Analysis for Japanese
• negapoji - Japanese negative positive classification.日本語文書のネガポジを判定。
• pymlask - Emotion analyzer for Japanese text
• asari - Japanese sentiment analyzer implemented in Python.
• kotobacore - Japanese semantic understanding engine for LLM, RAG, SNS analysis, and AI agents ? dependency-free tokenizer + Plutchik emotion + intent + RAG keywords.

Machine translation

Libraries that automatically translate text between languages

• jparacrawl-finetune - An example usage of JParaCrawl pre-trained Neural Machine Translation (NMT) models.
• JASS - JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation (LREC2020) & Linguistically Driven Multi-Task Pre-Training for Low-Resource Neural Machine Translation (ACM TALLIP)
• PheMT - A phenomenon-wise evaluation dataset for Japanese-English machine translation robustness. The dataset is based on the MTNT dataset, with additional annotations of four linguistic phenomena; Proper Noun, Abbreviated Noun, Colloquial Expression, and Variant. COLING 2020.
• VISA - An ambiguous subtitles dataset for visual scene-aware machine translation
• plamo-translate-cli - A command-line interface for translation using the plamo-2-translate model with local execution.

Named entity recognition

Libraries that extract names of people, places, and organizations from text

• namaco - Character Based Named Entity Recognition.
• entitypedia - Entitypedia is an Extended Named Entity Dictionary from Wikipedia.
• noyaki - Converts character span label information to tokenized text-based label information.
• bert-japanese-ner-finetuning - Code to perform finetuning of the BERT model. BERTモデルのファインチューニングで固有表現抽出用タスクのモデルを作成・使用するサンプルです
• joint-information-extraction-hs - 詳細なアノテーション基準に基づく症例報告コーパスからの固有表現及び関係の抽出精度の推論を行うコード
• pygeonlp - pygeonlp, A python module for geotagging Japanese texts.
• bert-ner-japanese - BERTによる日本語固有表現抽出のファインチューニング用プログラム
• huggingface-finetune-japanese - Examples to finetune encoder-only and encoder-decoder transformers for Japanese language (Hugging Face) Resources
• novelanalysisbyner - BERTのfine-tuningによる固有表現抽出

OCR

Libraries that recognize and extract text from images

• Manga OCR - About Optical character recognition for Japanese text, with the main focus being Japanese manga
• mokuro - Read Japanese manga inside browser with selectable text.
• handwritten-japanese-ocr - Handwritten Japanese OCR demo using touch panel to draw the input text using Intel OpenVINO toolkit
• OCR_Japanease - 日本語OCR
• ndlocr_cli - NDLOCRのアプリケーション
• donut - Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022
• JMTrans - manga translator - get japanese manga from url to translate manga image
• Kindai-OCR - OCR system for recognizing modern Japanese magazines
• text_recognition - NDLOCR用テキスト認識モジュール
• Poricom - Optical character recognition in manga images. Manga OCR desktop application
• owocr - Optical character recognition for Japanese text
• yomitoku - Yomitoku is an AI-powered document image analysis package designed specifically for the Japanese language.
• findtextcenternet - Japanese OCR with CenterNet
• simple-ocr-for-manga - A simple OCR for manga (Japanese traditional and Japanese vertical)
• jp-ocr-evaluation - 日本語の文章画像に対するOCRの性能を評価
• paddleocr-vl-sft-for-japanese-manga-on-rtx-3060 - Fine-tune PaddleOCR-VL on the Manga109s dataset for Japanese manga text recognition. The base model struggles with vertical Japanese text reading order in manga. After fine-tuning, the model correctly handles manga-specific text layouts.
• MangaOCR - A lightweight OCR model for Japanese text, especially in Manga
• meikiocr - high-speed, high-accuracy, local ocr for japanese video games
• meikipop - universal japanese ocr popup dictionary for windows, linux and macos

Tool for pretrained models

Libraries that utilize pretrained models to improve accuracy and efficiency

• JGLUE - JGLUE: Japanese General Language Understanding Evaluation
• ginza-transformers - Use custom tokenizers in spacy-transformers
• t5_japanese_dialogue_generation - T5による会話生成
• japanese_text_classification - To investigate various DNN text classifiers including MLP, CNN, RNN, BERT approaches.
• Japanese-BERT-Sentiment-Analyzer - Deploying sentiment analysis server with FastAPI and BERT
• jmlm_scoring - Masked Language Model-based Scoring for Japanese and Vietnamese
• allennlp-shiba-model - AllenNLP integration for Shiba: Japanese CANINE model
• evaluate_japanese_w2v - script to evaluate pre-trained Japanese word2vec model on Japanese similarity dataset
• gector-ja - BERT-based GEC tagging for Japanese
• Japanese-BPEEncoder - Japanese-BPEEncoder
• Japanese-BPEEncoder_V2 - Japanese-BPEEncoder Version 2
• transformer-copy - 日本語文法誤り訂正ツール
• nagisa_bert - A BERT model for nagisa
• jglue-evaluation-scripts - Training and evaluation scripts for JGLUE, a Japanese language understanding benchmark
• jptranstokenizer - Japanese Tokenizer for transformers library
• jp-stable - JP Language Model Evaluation Harness
• compare-ja-tokenizer - How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese-ACL SRW 2023
• lm-evaluation-harness-jp-stable - A framework for few-shot evaluation of autoregressive language models.
• llm-lora-classification - llm-lora-classification
• rinna_gpt-neox_ggml-lora - The repository contains scripts and merge scripts that have been modified to adapt an Alpaca-Lora adapter for LoRA tuning when assuming the use of the "rinna/japanese-gpt-neox..." [gpt-neox] model converted to ggml.
• japanese-llm-roleplay-benchmark - このリポジトリは日本語LLMのキャラクターロールプレイに関する性能を評価するために作成しました。
• japanese-llm-ranking - This repository supports YuzuAI's Rakuda leaderboard of Japanese LLMs, which is a Japanese-focused analogue of LMSYS' Vicuna eval.
• llm-jp-eval - このツールは、複数のデータセットを横断して日本語の大規模言語モデルを自動評価するものです．
• llm-jp-sft - This repository contains the code for supervised fine-tuning of LLM-jp models.
• llm-jp-tokenizer - LLM勉強会（LLM-jp）で開発しているLLM用のトークナイザー関連をまとめたリポジトリです．
• japanese-lm-fin-harness - Japanese Language Model Financial Evaluation Harness
• ja-vicuna-qa-benchmark - Japanese Vicuna QA Benchmark
• swallow-evaluation - Swallowプロジェクト 大規模言語モデル 評価スクリプト
• swallow-evaluation-instruct - Swallowプロジェクト 事後学習ずみ大規模言語モデル 評価フレームワーク
• pretrained_doc2vec_ja - pretrained doc2vec models on Japanese Wikipedia
• pl-bert-ja - A repository of Japanese Phoneme-Level BERT

Others

General-purpose tools supporting Japanese language processing

<details markdown="1"> <summary>Show 212 items</summary>

• namedivider-python - A tool for dividing the Japanese full name into a family name and a given name.
• asa-python - A curated list of resources dedicated to Python libraries of NLP for Japanese
• python_asa - python版日本語意味役割付与システム（ASA）
• toiro - A comparison tool of Japanese tokenizers
• ja-timex - 自然言語で書かれた時間情報表現を抽出/規格化するルールベースの解析器
• JapaneseTokenizers - A set of metrics for feature selection from text data
• daaja - This repository has implementations of data augmentation for NLP for Japanese.
• accel-brain-code - The purpose of this repository is to make prototypes as case study in the context of proof of concept(PoC) and research and development(R&D) that I have written in my website. The main research topics are Auto-Encoders in relation to the representation learning, the statistical machine learning for energy-based models, adversarial generation net…
• kyoto-reader - A processor for KyotoCorpus, KWDLC, and AnnotatedFKCCorpus
• nlplot - Visualization Module for Natural Language Processing
• rake-ja - Rapid Automatic Keyword Extraction algorithm for Japanese
• jel - Japanese Entity Linker.
• MedNER-J - Latest version of MedEX/J (Japanese disease name extractor)
• zunda-python - Zunda: Japanese Enhanced Modality Analyzer client for Python.
• AIO2_DPR_baseline - https://www.nlp.ecei.tohoku.ac.jp/projects/aio/
• showcase - A PyTorch implementation of the Japanese Predicate-Argument Structure (PAS) analyser presented in the paper of Matsubayashi & Inui (2018) with some improvements.
• darts-clone-python - Darts-clone python binding
• jrte-corpus_example - Example codes for Japanese Realistic Textual Entailment Corpus
• desuwa - Feature annotator to morphemes and phrases based on KNP rule files (pure-Python)
• HotPepperGourmetDialogue - Restaurant Search System through Dialogue in Japanese.
• nlp-recipes-ja - Samples codes for natural language processing in Japanese
• Japanese_nlp_scripts - Small example scripts for working with Japanese texts in Python
• DNorm-J - Japanese version of DNorm
• pyknp-eventgraph - EventGraph is a development platform for high-level NLP applications in Japanese.
• ishi - Ishi: A volition classifier for Japanese
• python-npylm - ベイズ階層言語モデルによる教師なし形態素解析
• python-npycrf - 条件付確率場とベイズ階層言語モデルの統合による半教師あり形態素解析
• unsupervised-pos-tagging - 教師なし品詞タグ推定
• negima - Negima is a Python package to extract phrases in Japanese text by using the part-of-speeches based rules you defined.
• YouyakuMan - Extractive summarizer using BertSum as summarization model
• japanese-numbers-python - A parser for Japanese number (Kanji, arabic) in the natural language.
• kantan - Lookup japanese words by radical patterns
• make-meidai-dialogue - Get Japanese dialogue corpus
• japanese_summarizer - A summarizer for Japanese articles.
• chirptext - ChirpText is a collection of text processing tools for Python.
• yubin - Japanese Address Munger
• jawiki-cleaner - Japanese Wikipedia Cleaner
• japanese2phoneme - A python library to convert Japanese to phoneme.
• anlp_nlp2021_d3-1 - This repository contains codes related to the experiments in "An Experimental Evaluation of Japanese Tokenizers for Sentiment-Based Text Classification"
• aozora_classification - This project aims to classify Japanese sentence to how well similar to some Japanese classical writers, such as Soseki Natsume, Ogai Mori, Ryunosuke Akutagawa and so on.
• aozora-corpus-generator - Generates plain or tokenized text files from the Aozora Bunko
• JLM - A fast LSTM Language Model for large vocabulary language like Japanese and Chinese
• NTM - Testing of Neural Topic Modeling for Japanese articles
• EN-JP-ML-Lexicon - This is a English-Japanese lexicon for Machine Learning and Deep Learning terminology.
• text-generation - Easy-to-use scripts to fine-tune GPT-2-JA with your own texts, to generate sentences, and to tweet them automatically.
• chainer_nic - Neural Image Caption (NIC) on chainer, its pretrained models on English and Japanese image caption datasets.
• unihan-lm - The official repository for "UnihanLM: Coarse-to-Fine Chinese-Japanese Language Model Pretraining with the Unihan Database", AACL-IJCNLP 2020
• mbart-finetuning - Code to perform finetuning of the mBART model.
• xvector_jtubespeech - xvector model on jtubespeech
• TinySegmenterMaker - TinySegmenter用の学習モデルを自作するためのツール．
• Grongish - 日本語とグロンギ語の相互変換スクリプト
• WordCloud-Japanese - WordCloudでの日本語文章をMecab（形態素解析エンジン）を使用せずに形態素解析チックな表示を実現するスクリプト
• snark - 日本語ワードネットを利用したDBアクセスライブラリ
• toEmoji - 日本語文を絵文字だけの文に変換するなにか
• termextract - - 専門用語抽出アルゴリズムの実装の練習
• JDT-with-KenLM-scoring - Japanese-Dialog-Transformerの応答候補に対して、KenLMによるN-gram言語モデルでスコアリングし、フィルタリング若しくはリランキングを行う。
• mixture-of-unigram-model - Mixture of Unigram Model and Infinite Mixture of Unigram Model in Python. (混合ユニグラムモデルと無限混合ユニグラムモデル)
• hidden-markov-model - Hidden Markov Model (HMM) and Infinite Hidden Markov Model (iHMM) in Python. (隠れマルコフモデルと無限隠れマルコフモデル)
• Ngram-language-model - Ngram language model in Python. (Nグラム言語モデル)
• asr - Automatic Speech Recognition with deepspeech2 model in pytorch with support from Zakuro AI.
• neural_ime - Neural IME: Neural Input Method Engine
• neural_japanese_transliterator - Can neural networks transliterate Romaji into Japanese correctly?
• tinysegmenter - tokenizer specified for Japanese
• AugLy-jp - Data Augmentation for Japanese Text on AugLy
• furigana4epub - A Python script for adding furigana to Japanese epub books using Mecab and Unidic.
• PyKatsuyou - Japanese verb/adjective inflections tool
• jageocoder - Pure Python Japanese address geocoder
• nksnd - New kana-kanji conversion engine
• JaMIE - A Japanese Medical Information Extraction Toolkit
• fasttext-vs-word2vec-on-twitter-data - fasttextとword2vecの比較と、実行スクリプト、学習スクリプトです
• minimal-search-engine - 最小のサーチエンジン/PageRank/tf-idf
• 5ch-analysis - 5chの過去ログをスクレイピングして、過去流行った単語(ex, 香具師, orz)などを追跡調査
• tweet_extructor - Twitter日本語評判分析データセットのためのツイートダウンローダ
• japanese-word-aggregation - Aggregating Japanese words based on Juman++ and ConceptNet5.5
• jinf - A Japanese inflection converter
• kwja - A unified language analyzer for Japanese
• mlm-scoring-transformers - Reproduced package based on Masked Language Model Scoring (ACL2020).
• ClipCap-for-Japanese - [PyTorch] ClipCap for Japanese
• SAT-for-Japanese - [PyTorch] Show, Attend and Tell for Japanese
• cihai - Python library for CJK (Chinese, Japanese, and Korean) language dictionary
• marine - MARINE : Multi-task leaRnIng-based JapaNese accent Estimation
• whisper-asr-finetune - Finetuning Whisper ASR model
• radicalchar - 部首文字正規化ライブラリ
• posuto - Japanese postal code data.
• tacotron2-japanese - Tacotron2 implementation of Japanese
• ibus-hiragana - ひらがなIME for IBus
• furiganapad - ふりがなパッド
• chikkarpy - Japanese synonym library
• ja-tokenizer-docker-py - Mecab + NEologd + Docker + Python3
• JapaneseEmbeddingEval - JapaneseEmbeddingEval
• shuwa - Extend GNOME On-Screen Keyboard for Input Methods
• japanese-nli-model - This repository provides the code for Japanese NLI model, a fine-tuned masked language model.
• tra-fugu - A tool for Japanese-English translation and English-Japanese translation by using FuguMT
• fugumt - ぷるーふおぶこんせぷと で公開した機械翻訳エンジンを利用する翻訳環境です。 フォームに入力された文字列の翻訳、PDFの翻訳が可能です。
• JaSPICE - JaSPICE: Automatic Evaluation Metric Using Predicate-Argument Structures for Image Captioning Models
• Retrieval-based-Voice-Conversion-WebUI-JP-localization - jp-localization
• pyopenjtalk - Python wrapper for OpenJTalk
• yomigana-ebook - Make learning Japanese easier by adding readings for every kanji in the eBook
• N46Whisper - Whisper based Japanese subtitle generator
• japanese_llm_simple_webui - Rinna-3.6B、OpenCALM等の日本語対応LLM(大規模言語モデル)用の簡易Webインタフェースです
• pdf-translator - pdf-translator translates English PDF files into Japanese, preserving the original layout.
• japanese_qa_demo_with_haystack_and_es - Haystack + Elasticsearch + wikipedia(ja) を用いた、日本語の質問応答システムのサンプル
• mozc-devices - Automatically exported from code.google.com/p/mozc-morse
• vits-japros-webui - 日本語TTS（VITS）の学習と音声合成のGradio WebUI
• ja-law-parser - A Japanese law parser
• dictation-kit - Japanese dictation kit using Julius
• julius4seg - Juliusを使ったセグメンテーション支援ツール
• voicevox_engine - 無料で使える中品質なテキスト読み上げソフトウェア、VOICEVOXの音声合成エンジン
• LLaVA-JP - LLaVA-JP is a Japanese VLM trained by LLaVA method
• bertjsc - Japanese Spelling Error Corrector using BERT(Masked-Language Model). BERTに基づいて日本語校正
• llm-leaderboard - Project of llm evaluation to Japanese tasks
• BLIP2-Japanese - Modifying LAVIS' BLIP2 Q-former with models pretrained on Japanese datasets.
• wikipedia-passages-jawiki-embeddings-utils - wikipedia 日本語の文を、各種日本語の embeddings や faiss index へと変換するスクリプト等。
• simple-simcse-ja - Exploring Japanese SimCSE
• gpt4-autoeval - GPT-4 を用いて、言語モデルの応答を自動評価するスクリプト
• t5-japanese - 日本語T5モデル
• japanese_llm_eval - A repo for evaluating Japanese LLMs　・　日本語LLMを評価するレポ
• jmteb - The evaluation scripts of JMTEB (Japanese Massive Text Embedding Benchmark)
• pydomino - 日本語音声に対して音素ラベルをアラインメントするためのツールです
• easynovelassistant - 軽量で規制も検閲もない日本語ローカル LLM『LightChatAssistant-TypeB』による、簡単なノベル生成アシスタントです。ローカル特権の永続生成 Generate forever で、当たりガチャを積み上げます。読み上げにも対応。
• clip-japanese - 日本語データセットでのqlora instruction tuning学習サンプルコード
• rime-jaroomaji - Japanese rōmaji input schema for Rime IME
• deep-question-generation - 深層学習を用いたクイズ自動生成（日本語T5モデル）
• magpie-nemotron - Magpieという手法とNemotron-4-340B-Instructを用いて合成対話データセットを作るコード
• qlora_ja - 日本語データセットでのqlora instruction tuning学習サンプルコード
• mozcdic-ut-jawiki - Mozc UT Jawiki Dictionary is a dictionary generated from the Japanese Wikipedia for Mozc.
• shisa-v2 - Japanese / English Bilingual LLM
• llm-translator - Mixtral-based Ja-En (En-Ja) Translation model
• llm-jp-asr - Whisperのデコーダをllm-jp-1.3b-v1.0に置き換えた音声認識モデルを学習させるためのコード
• rag-japanese - Open source RAG with Llama Index for Japanese LLM in low resource settting
• monaka - A Japanese Parser (including historical Japanese)
• jp-translate.cloud - A state-of-the-art open-source Japanese <--> English machine translation system based on the latest NMT research.
• substring-word-finder - 連続部分文字列の単語判定を行います
• heron-vlm-leaderboard - This project is a benchmarking tool for evaluating and comparing the performance of various Vision Language Models (VLMs). It uses two datasets: LLaVA-Bench-In-the-Wild and Japanese HERON Bench to measure model performance.
• text2dataset - Easily turn large English text datasets into Japanese text datasets using open LLMs.
• mecab-web-api - MeCabを利用した日本語形態素解析WebAPI
• mecab_controller - Mecab wrapper to generate furigana readings.
• vits - VITSによるテキスト読み上げ器&ボイスチェンジャー
• akari_chatgpt_bot - 音声認識、文章生成、音声合成を使って対話するチャットボットアプリ
• kudasai - Streamlining Japanese-English Translation with Advanced Preprocessing and Integrated Translation Technologies
• mecab-visualizer - MeCabの形態素解析結果を可視化するツール
• add-dictionary - OpenJTalkのユーザ辞書をGUIで追加するアプリ
• j-moshi - J-Moshi: A Japanese Full-duplex Spoken Dialogue System
• jatts - JATTS: Japanese TTS (for research)
• tsukasa-speech - a Frontier Japanese Speech Generation net
• symptom-expression-search - ElasticsearchやGiNZA、患者表現辞書を使った患者表現揺れ吸収する意味構造検索を試した
• llm-jp-judge - 生成自動評価を行うためのPythonツール
• asagi-vlm-colaboratory-sample - Colaboratory上でAsagi(合成データセットを活用した大規模日本語VLM)をお試しするサンプル
• llm-jp-eval-mm - This tool automatically evaluates Japanese multi-modal large language models across multiple datasets.
• manga109api - Simple python API to read annotation data of Manga109
• fastrtc-jp - fastrtc用の日本語TTSとSTT追加キット
• whisper-transcription - Pythonを使用したWhisperモデルによる音声文字起こしツール
• pocket-researcher - LLMを活用した自律調査エージェント。手軽に情報収集、概要把握。
• jtransbench - A tool to easily benchmark Japanese translation skills
• easyllasa - EasyLlasa は 5～15秒の日本語音声と日本語テキストから日本語音声を生成する TSTS (TextSpeechToSpeech) です。
• kanjikana-model - 氏名漢字カナ突合モデル
• deep-openreview-research-ja - OpenReview論文を自動で発見・分析する日本語対応AIエージェント
• pitchbench - Experimental Japanese pitch accent based LLM Benchmark
• mini-transformer-from-scratch - English to Japanese Transformer from scratch
• vv_core_inference - VOICEVOXのコア内で用いられているディープラーニングモデルの推論コード
• pyopenjtalk-plus - pyopenjtalk-plus: A Python wrapper for OpenJTalk with additional improvements
• japanese_spelling_correction - Japanese Spelling Correction
• py-kaomoji - python kaomoji
• [llm-jp-vila](https://github.com/llm-jp/llm-jp-vil