書物単位で分割したテスト集合（59,072 行）に対し、かな表記の揺れを正規化したうえで評価したところ、本文の文字単位正答率はおよそ 92.0%（plain micro CER 0.080）でした。語彙を 7,710 文字へ拡張し旧字・異体字を忠実保存しながらも、本文の認識精度はこの水準を維持しています。注釈構造の付与は、ふりがなの「(漢字, 読み) 完全一致」で F1 0.73、送り仮名で F1 0.71、返り点で F1 0.52（漢文の密な返り点を含む全件）、割書で F1 0.40 の水準で再現できています。
ふりがなは「どの漢字に何の読みを付けるか」という厳しい完全一致でも F1 0.73、送り仮名は F1 0.71 と良好です（送り仮名は前世代から改善）。割書の F1 が低めなのは、1 スパンが長い 2 列テキストで「1 文字でも違えば不正解」という厳しい span 完全一致の定義によるものです。返り点は、みんなで翻刻のテキストに漢文の返り点が密に並ぶ行が多く含まれるため、全件では F1 0.52 にとどまります。これは隣接する仮名や複数の文字を 1 つの返り点として取り込みすぎる過剰取り込みが主因で、引き続きの改善対象です。
Minna de Honkoku OCR is an OCR system that, given a cursive Japanese (kuzushiji) classical book image, generates not only the body text transcription but also the surrounding annotation structure—furigana (ruby readings), okurigana and kaeriten (Japanese-style reading marks for Chinese-text passages), and warigaki (interlinear two-column notes). The output is structured text with tags preserving these annotation relations, making downstream typesetting, search, and reuse straightforward.
Dataset Provenance
Training data is sourced from Minna de Honkoku, a citizen-driven transcription platform launched in 2017 and jointly operated by the National Museum of Japanese History, the Earthquake Research Institute of the University of Tokyo, and the Kyoto University Historical Earthquakes Research Group. The platform hosts classical-book images from the IIIF digital archives of multiple holding institutions, including the National Diet Library, National Institute of Japanese Literature, University of Tokyo Library, Kyoto University Library, National Museum of Japanese History, Fukui Prefectural Digital Archives, and University of the Ryukyus Library. Volunteer transcribers attach transcription and annotations (furigana, kaeriten, etc.) to those images.
From the licensed portion of these transcriptions, we paired per-line position information (coordinates on the IIIF Image API) with transcription text and extracted approximately 1.2 million line-image / transcription pairs, organizing them into our custom webdataset_v3 format. Train/validation/test splits are deterministic by a hash of the title (entryId), so different lines from the same book never cross splits—the system is evaluated for book-level generalization.
This dataset is a scale-up of the Minna de Honkoku Dataset published by NDL Lab, which is also derived from Minna de Honkoku transcriptions. Following the NDL Lab dataset’s design as a foundation, we broaden the source materials and strengthen automatic line-image extraction and annotation-structure preservation (furigana, kaeriten, etc.), expanding the scale to ~1.2 million line pairs.
For automatic line-image extraction we use a page-layout / line-detection model trained on the NDL-DocL Dataset (document-image layout dataset), published by NDL Lab. Line bounding boxes detected on each page are cropped via the IIIF Image API. For aligning each cropped line image with a single line of transcription text (i.e., which bbox corresponds to which line of text), we use the parseq ONNX model from NDL Kotenseki OCR-Lite as a recognition prior: we compute the edit distance (Levenshtein distance) between each tentative parseq output and each transcription line, normalize by length, and accept only pairs whose normalized distance falls below a fixed threshold. This filters out noisy or clearly misaligned correspondences and yields a mechanically validated text–image mapping.
The constructed dataset will be released under an open license in the near future, in a form usable for both academic and commercial purposes. Details will be announced in this repository.
Data Preprocessing
Transcriptions use a notation specific to Minna de Honkoku. For example, “漢字（かんじ）” or “《振り仮名：漢字｜かんじ》” encode furigana; “￣ニ” or “［ニ］” encode okurigana; “＿レ” or “｛レ｝” encode kaeriten; and “《割書：右｜左》” encodes warigaki. As a first step, all of these are normalized into special training tokens: <ruby>, <rt>, <OKURI>, <KAERI>, <WARI>, etc. To absorb kana-orthographic variation across transcribers, we fold isolated single-character katakana into hiragana. In contrast, classical-form (kyujitai) and variant characters are preserved faithfully rather than unified to modern (shinjitai) forms (the previous generation applied a 464-character kyujitai→shinjitai mapping, which is dropped in this version). This lets the model reproduce old and variant glyphs exactly as they appear in the source, and the vocabulary is expanded to cover them. Note that the rare “second-reading” ruby (a furigana on the left side of the kanji, encoded as <rt2>) appears very rarely in the gold and the model used to over-produce it; in the current version it is removed at preprocessing time and excluded from the vocabulary.
Model Architecture
The line recognizer is a Vision-Encoder-Decoder neural network that generates structured text from images, implemented on top of HuggingFace’s VisionEncoderDecoderModel framework.
The encoder uses ConvNeXt V2 at the Base scale (~88M parameters). ConvNeXt V2 adds self-supervised pretraining via a fully convolutional masked autoencoder (FCMAE) and a Global Response Normalization (GRN) layer; we start from weights fine-tuned on ImageNet-22k at 384px. Each vertical line is rotated 90° to a horizontal orientation and resized (aspect-preserving) to height 256px and width up to 2048px (preserving an 8:1 aspect ratio). With a cumulative stride of 32, the encoder produces an 8×64 feature map (512 vectors of 1024 dimensions). On top of this we add a custom learned 2D positional embedding indicating “which row and column of the line is this feature from” (8 row embeddings and 72 column embeddings; we add the appropriate row and column embeddings to each cell and apply LayerNorm).
The decoder is a small RoBERTa-based Transformer (6 layers, hidden size 512, 8 heads). It is initialized from a RoBERTa pretrained on a ~57 million-character corpus assembled from Minna de Honkoku transcriptions, using a masked language modeling (MLM) objective. In other words, the decoder starts from a language model already adapted to the target domain. We then transfer it to the OCR task; the cross-attention is trained from scratch (in this version the OCR model is trained entirely from scratch, without inheriting a prior OCR checkpoint). Generation is autoregressive, one token at a time, from <CLS> until <SEP>. The vocabulary is a character-level tokenizer of the top 7,710 most frequent characters in the corpus—expanded (from 5,000 in the previous generation) to match the faithful-preservation policy for classical and variant glyphs—and it colocates 11 structural special tokens alongside those characters, so that character recognition and annotation tagging happen jointly in a single generation pass.
Figure: Model architecture of Minna de Honkoku OCR
Training
Optimization uses AdamW with two learning-rate groups: the newly initialized modules (2D positional embeddings, the encoder/decoder projection layer, and cross-attention) at a high rate of 1×10−3, and the encoder backbone and the pretrained decoder body at a low rate of 5×10−5. Training runs in bfloat16 autocast with effective batch size 64 for roughly 5 epochs on a single NVIDIA A100. The 256×2048 input is pixel-heavy and memory-intensive, so we shrink the micro-batch and increase gradient accumulation to keep the effective batch size unchanged. The learning rate is annealed linearly in the final segment, and the validation CER drops a further notch during that anneal.
The loss is cross-entropy with label smoothing 0.1, augmented with weighted CE (weight 2.0 on structural tokens). We additionally add a focal-style regularizer that penalizes high <SEP> probability at non-terminal positions (coefficient 0.5) to discourage the model from cutting lines short. To compensate for the low frequency of kaeriten and okurigana, we oversample the corresponding training lines by 2.0× each. Augmentation via albumentations randomly applies elastic distortion, morphological operations, Gaussian noise, resolution reduction, and JPEG compression artifacts.
Evaluation
On a book-level test split of 59,072 lines, with kana variation normalized, plain text CER is approximately 0.080 (~92.0% character-level accuracy). The body-text accuracy holds at this level even though the vocabulary was expanded to 7,710 characters to faithfully preserve classical and variant glyphs. For annotation tagging, F1 reaches 0.73 for the strict “exact (kanji, reading) pair” furigana match, 0.71 for okurigana, 0.52 for kaeriten (all spans, including densely-marked kanbun), and 0.40 for warigaki.
About the CER figures (important). The CER reported here is measured against the volunteer transcriptions of Minna de Honkoku as ground truth. However, those transcriptions have not yet undergone expert review: a separate sampling survey over ~100,000 characters estimated the transcriptions’ own accuracy at about 98.5%. The numbers above are therefore agreement rates against an imperfect reference—errors in the transcription can be counted as model errors and vice versa—and do not strictly equal the model’s true recognition accuracy. We plan a more rigorous CER measurement using an expert-reviewed test set in the future.
Methodologically, structural tokens are best evaluated with multiset-token F1 rather than region CER. F1 is order-invariant and outlier-robust, and gives a more realistic picture.
Per-structure Precision / Recall / F1 (multiset match; kana-fold normalized, spurious <rt2> stripped = deployed configuration; measured on the v16 test split of 59,072 lines):
Structure
Unit definition
gold
P
R
F1
Furigana
exact (kanji, reading) pair match
29,279
0.764
0.697
0.729
Okurigana
exact <OKURI> span string match
4,474
0.685
0.726
0.705
Kaeriten
exact <KAERI> span string match (all spans)
11,455
0.535
0.499
0.516
Warigaki
exact span match (right + separator + left)
2,203
0.453
0.356
0.399
Furigana keeps F1 0.73 even under the strict “which kanji gets which reading” exact match, and okurigana reaches F1 0.71 (improved over the previous generation). Warigaki F1 is relatively low because its unit is a long two-column string and any single-character mismatch fails the exact span match. Kaeriten sits at F1 0.52 over all spans, because Minna de Honkoku contains many lines of densely-marked Chinese (kanbun) text; the model tends to over-capture adjacent kana or multiple characters into a single return mark—a continuing improvement target.
Per-host plain micro CER (hosts with n ≥ 300; measured on the v16 test split), showing the range of per-institution difficulty:
Holding institution (IIIF host)
n
CER
Fukui Prefectural Digital Archiveswww.digital-archives.pref.fukui.lg.jp
6,656
0.046
ADEACdcfs.trc-adeac.co.jp
1,809
0.062
University of Tokyo Libraryiiif.dl.itc.u-tokyo.ac.jp
8,376
0.068
Kyoto University Libraryrmda.kulib.kyoto-u.ac.jp
1,190
0.068
National Diet Librarydl.ndl.go.jp
20,367
0.070
University of the Ryukyus Libraryshimuchi.lib.u-ryukyu.ac.jp
Kyushu University Librarycatalog.lib.kyushu-u.ac.jp
1,039
0.100
Tokyo Gakugei University Libraryd-archive.u-gakugei.ac.jp
1,071
0.102
National Museum of Japanese Historykhirin-a.rekihaku.ac.jp
3,214
0.103
NIJL Classical Bookskotenseki.nijl.ac.jp
1,050
0.120
National Diet Library (legacy)www.dl.ndl.go.jp
3,590
0.125
National Institute of Japanese Literaturekokusho.nijl.ac.jp
1,981
0.129
Among higher-volume hosts, the largest sample—the National Diet Library—reaches 0.070, the University of Tokyo Library 0.068, and the Fukui Prefectural Digital Archives 0.046, whereas the National Institute of Japanese Literature (both classical and regular collections) sits around 0.12, clearly reflecting the range of script and page-layout difficulty. The overall plain micro CER across all 59,072 lines is 0.080.
Inference
Processing in the browser is a two-stage pipeline: first layout recognition detects the body-text line regions in the input image, then line OCR (enc-dec recognition) runs on each detected line. For layout recognition we use the RTMDet-s model from NDL Kotenseki OCR-Lite (switchable in settings to this system’s own YOLOv8 model). The reading order of the detected line boxes is fixed during the layout stage by a text-free XY-Cut.
The trained model is exported to ONNX and compressed via quantize_dynamic (dynamic int8 quantization). The encoder is shipped as a single file (~89 MB), while for inference speed the decoder is split into two files: a prefill graph (~32 MB) that runs once on <CLS> and constructs the internal Key/Value cache, and a step graph (~28 MB) that consumes the cache and a single new token at each iteration. This means decoder self-attention shifts from “recompute over all tokens generated so far” to “extend by one new token,” giving a theoretical 5–10× speedup on greedy decoding. The graphs are executed in the browser’s WebAssembly environment using onnxruntime-web.
Several post-processing steps run at inference time. First, small per-line tilts are estimated with a projection-profile method and corrected before recognition. Second, a safety guard terminates generation if the output begins repeating the same token or a short cyclic pattern (end-of-line collapse).
Future Work
Low-resolution images, severely deformed scripts, and elements written as two columns within a single line (warigaki) remain failure modes. The absolute level for warigaki is still low; we plan further resolution increases and targeted additional training on specific source materials. For kaeriten, the next target is suppressing over-capture—the tendency to absorb adjacent kana or multiple characters into a single return mark. We are also exploring grammar-constrained decoding at inference time (to enforce tag consistency) and integration with a language model (to correct visually confusable characters). As noted above, the current CER is measured against unreviewed transcriptions as ground truth; we plan to re-evaluate on a rigorous test set that has undergone expert review.
