Minna de Honkoku OCR is an OCR system that, given a cursive Japanese (kuzushiji) classical book image, generates not only the body text transcription but also the surrounding annotation structure—furigana (ruby readings), okurigana and kaeriten (Japanese-style reading marks for Chinese-text passages), and warigaki (interlinear two-column notes). The output is structured text with tags preserving these annotation relations, making downstream typesetting, search, and reuse straightforward.
Dataset Provenance
Training data is sourced from Minna de Honkoku, a citizen-driven transcription platform launched in 2017 and jointly operated by the National Museum of Japanese History, the Earthquake Research Institute of the University of Tokyo, and the Kyoto University Historical Earthquakes Research Group. The platform hosts classical-book images from the IIIF digital archives of multiple holding institutions, including the National Diet Library, National Institute of Japanese Literature, University of Tokyo Library, Kyoto University Library, National Museum of Japanese History, Fukui Prefectural Digital Archives, and University of the Ryukyus Library. Volunteer transcribers attach transcription and annotations (furigana, kaeriten, etc.) to those images.
From the licensed portion of these transcriptions, we paired per-line position information (coordinates on the IIIF Image API) with transcription text and extracted approximately 1.17 million line-image / transcription pairs (4,140 works), organizing them into our custom webdataset_v4 format. Train/validation/test splits are deterministic by a hash of the title (entryId), so different lines from the same book never cross splits—the system is evaluated for book-level generalization.
This dataset is a scale-up of the Minna de Honkoku Dataset published by NDL Lab, which is also derived from Minna de Honkoku transcriptions. Following the NDL Lab dataset’s design as a foundation, we broaden the source materials and strengthen automatic line-image extraction and annotation-structure preservation (furigana, kaeriten, etc.), expanding the scale to ~1.17 million line pairs.
For automatic line-image extraction we use the RTMDet-s line-detection model from NDL Kotenseki OCR-Lite (1024×1024 input). Line bounding boxes detected on each page are cropped via the IIIF Image API. For aligning each cropped line image with a single line of transcription text (i.e., which bbox corresponds to which line of text), we use the parseq ONNX model from NDL Kotenseki OCR-Lite as a recognition prior. A pair is accepted only if it satisfies all three criteria: length-normalized edit distance (Levenshtein) ≤ 0.4, a recognized-to-transcribed length ratio between 0.85 and 1.20, and a line-detection score ≥ 0.25. The length-ratio criterion complements edit distance specifically to reject pairs where only part of the line was cropped—cases where the transcription contains characters whose pixels are absent from the image. Such pairs teach the model the wrong behavior: to omit text it cannot see.
The constructed dataset will be released under an open license in the near future, in a form usable for both academic and commercial purposes. Details will be announced in this repository.
Data Preprocessing
Transcriptions use a notation specific to Minna de Honkoku. For example, “漢字（かんじ）” or “《振り仮名：漢字｜かんじ》” encode furigana; “￣ニ” or “［ニ］” encode okurigana; “＿レ” or “｛レ｝” encode kaeriten; and “《割書：右｜左》” encodes warigaki. As a first step, all of these are normalized into special training tokens: <ruby>, <rt>, <OKURI>, <KAERI>, <WARI>, etc. To absorb kana-orthographic variation across transcribers, we fold isolated single-character katakana into hiragana. In contrast, classical-form (kyujitai) and variant characters are preserved faithfully rather than unified to modern (shinjitai) forms (the previous generation applied a 464-character kyujitai→shinjitai mapping, which is dropped in this version). This lets the model reproduce old and variant glyphs exactly as they appear in the source, and the vocabulary is expanded to cover them. Note that the rare “second-reading” ruby (a furigana on the left side of the kanji, encoded as <rt2>) appears very rarely in the gold and the model used to over-produce it; in the current version it is removed at preprocessing time and excluded from the vocabulary.
Model Architecture
The line recognizer is a Vision-Encoder-Decoder neural network that generates structured text from images, implemented on top of HuggingFace’s VisionEncoderDecoderModel framework.
The encoder uses ConvNeXt V2 at the Base scale (~88M parameters). ConvNeXt V2 adds self-supervised pretraining via a fully convolutional masked autoencoder (FCMAE) and a Global Response Normalization (GRN) layer; we start from weights fine-tuned on ImageNet-22k at 384px. Each vertical line is rotated 90° to a horizontal orientation and resized (aspect-preserving) to height 256px and width up to 2048px (preserving an 8:1 aspect ratio). With a cumulative stride of 32, the encoder produces an 8×64 feature map (512 vectors of 1024 dimensions). On top of this we add a custom learned 2D positional embedding indicating “which row and column of the line is this feature from” (8 row embeddings and 72 column embeddings; we add the appropriate row and column embeddings to each cell and apply LayerNorm).
The decoder is a small RoBERTa-based Transformer (6 layers, hidden size 512, 8 heads). It is initialized from a RoBERTa pretrained on a ~57 million-character corpus assembled from Minna de Honkoku transcriptions, using a masked language modeling (MLM) objective. In other words, the decoder starts from a language model already adapted to the target domain. We then transfer it to the OCR task; the cross-attention is trained from scratch (in this version the OCR model is trained entirely from scratch, without inheriting a prior OCR checkpoint). Generation is autoregressive, one token at a time, from <CLS> until <SEP>. The vocabulary is a character-level tokenizer of the top 7,710 most frequent characters in the corpus—expanded (from 5,000 in the previous generation) to match the faithful-preservation policy for classical and variant glyphs—and it colocates 11 structural special tokens alongside those characters, so that character recognition and annotation tagging happen jointly in a single generation pass.
Figure: Model architecture of Minna de Honkoku OCR
Training
Optimization uses AdamW with two learning-rate groups: the newly initialized modules (2D positional embeddings, the encoder/decoder projection layer, and cross-attention) at a high rate of 1×10−3, and the encoder backbone and the pretrained decoder body at a low rate of 5×10−5. Training runs in bfloat16 autocast with effective batch size 64 for roughly 5 epochs on a single NVIDIA A100. The 256×2048 input is pixel-heavy and memory-intensive, so we shrink the micro-batch and increase gradient accumulation to keep the effective batch size unchanged. The learning rate is annealed linearly in the final segment, and the validation CER drops a further notch during that anneal.
The loss is cross-entropy with label smoothing 0.1, augmented with weighted CE (weight 2.0 on structural tokens). We additionally add a focal-style regularizer that penalizes high <SEP> probability at non-terminal positions (coefficient 0.5) to discourage the model from cutting lines short. To compensate for the low frequency of kaeriten and okurigana, we oversample the corresponding training lines by 2.0× each. Augmentation via albumentations randomly applies elastic distortion, morphological operations, Gaussian noise, resolution reduction, and JPEG compression artifacts.
Evaluation
On a book-level test split of 58,521 lines (~1.01 million characters), with kana variation normalized, plain text CER is approximately 0.075 (~92.5% character-level accuracy)—achieved while the vocabulary is expanded to 7,710 characters to faithfully preserve classical and variant glyphs. For annotation tagging, F1 reaches 0.75 for the strict “exact (kanji, reading) pair” furigana match, 0.82 for kaeriten and 0.43 for warigaki. Okurigana remains at 0.15; this stems from a defect in the training-time preprocessing that has since been identified, and retraining is planned.
About the CER figures (important). The CER reported here is measured against the volunteer transcriptions of Minna de Honkoku as ground truth. However, those transcriptions have not yet undergone expert review: a separate sampling survey over ~100,000 characters estimated the transcriptions’ own accuracy at about 98.5%. The numbers above are therefore agreement rates against an imperfect reference—errors in the transcription can be counted as model errors and vice versa—and do not strictly equal the model’s true recognition accuracy. We plan a more rigorous CER measurement using an expert-reviewed test set in the future.
Methodologically, structural tokens are best evaluated with multiset-token F1 rather than region CER. F1 is order-invariant and outlier-robust, and gives a more realistic picture.
Per-structure Precision / Recall / F1 (multiset match; kana-fold normalized, spurious <rt2> stripped = deployed configuration; measured on the test split of 58,521 lines. A span counts as correct only on an exact string match; its position within the line is not considered):
Structure
Unit definition
gold
P
R
F1
Furigana
exact (kanji, reading) pair match
26,424
0.776
0.720
0.747
Kaeriten
exact <KAERI> span string match
368
0.782
0.867
0.822
Warigaki
exact span match (right + separator + left)
835
0.460
0.404
0.430
Okurigana
exact <OKURI> span string match
325
0.104
0.274
0.151
Furigana keeps F1 0.75 even under the strict “which kanji gets which reading” exact match, and kaeriten reaches F1 0.82. Warigaki F1 is relatively low because its unit is a long two-column string and any single-character mismatch fails the exact span match. Okurigana sits at F1 0.15, driven by over-tagging: the model emits 852 spans against 325 in the gold. The cause has been traced to the training-time preprocessing, which normalizes okurigana to hiragana and thereby breaks the correspondence with the katakana actually written on the page; retraining is planned. Note also that kaeriten and okurigana occur only 368 and 325 times in this test split, so those figures should be read with care.
Per-host plain micro CER (hosts with n ≥ 300; measured on the test split of 58,521 lines), showing the range of per-institution difficulty:
Holding institution (IIIF host)
n
CER
Fukui Prefectural Digital Archiveswww.digital-archives.pref.fukui.lg.jp
4,519
0.046
University of the Ryukyus Libraryshimuchi.lib.u-ryukyu.ac.jp
1,284
0.055
Kyoto University Libraryrmda.kulib.kyoto-u.ac.jp
790
0.058
ADEACiiif.adeac.jp
1,580
0.064
National Diet Librarydl.ndl.go.jp
21,187
0.068
University of Tokyo Libraryiiif.dl.itc.u-tokyo.ac.jp
7,171
0.068
Tokyo Gakugei University Libraryd-archive.u-gakugei.ac.jp
765
0.078
amane projectourarchives.amane-project.jp
6,262
0.082
National Diet Library (legacy)www.dl.ndl.go.jp
1,465
0.093
National Institute of Japanese Literaturekokusho.nijl.ac.jp
1,712
0.094
NIJL Classical Bookskotenseki.nijl.ac.jp
1,134
0.095
National Museum of Japanese Historykhirin-a.rekihaku.ac.jp
Kyushu University Librarycatalog.lib.kyushu-u.ac.jp
1,004
0.113
ContentDM-hostedcdm16028.contentdm.oclc.org
946
0.119
Among higher-volume hosts, the largest sample—the National Diet Library—reaches 0.068, the University of Tokyo Library 0.068, and the Fukui Prefectural Digital Archives 0.046, whereas Kyushu University Library and the ContentDM-hosted materials sit at 0.11–0.12—more than a 2.5-fold spread across institutions, clearly reflecting differences in script, page layout, and imaging conditions. The overall plain micro CER across all 58,521 lines is 0.075.
Inference
Processing in the browser is a two-stage pipeline: first layout recognition detects the body-text line regions in the input image, then line OCR (enc-dec recognition) runs on each detected line. For layout recognition we use the RTMDet-s model from NDL Kotenseki OCR-Lite (switchable in settings to this system’s own YOLOv8 model). The reading order of the detected line boxes is fixed during the layout stage by a text-free XY-Cut.
The trained model is exported to ONNX and compressed via quantize_dynamic (dynamic int8 quantization). The encoder is shipped as a single file (~90 MB), while for inference speed the decoder is split into two files: a prefill graph (~34 MB) that runs once on <CLS> and constructs the internal Key/Value cache, and a step graph (~31 MB) that consumes the cache and a single new token at each iteration. This means decoder self-attention shifts from “recompute over all tokens generated so far” to “extend by one new token,” giving a theoretical 5–10× speedup on greedy decoding. The graphs are executed in the browser’s WebAssembly environment using onnxruntime-web.
Several post-processing steps run at inference time. First, small per-line tilts are estimated with a projection-profile method and corrected before recognition. Second, a safety guard terminates generation if the output begins repeating the same token or a short cyclic pattern (end-of-line collapse).
Future Work
Low-resolution images, severely deformed scripts, and elements written as two columns within a single line (warigaki) remain failure modes. The absolute level for warigaki is still low (F1 0.44); we plan further resolution increases and targeted additional training on specific source materials. For kaeriten, the next target is suppressing over-capture—the tendency to absorb adjacent kana or multiple characters into a single return mark. We are also exploring grammar-constrained decoding at inference time (to enforce tag consistency) and integration with a language model (to correct visually confusable characters). As noted above, the current CER is measured against unreviewed transcriptions as ground truth; we plan to re-evaluate on a rigorous test set that has undergone expert review.
