SemiHastakshar: Generalizable Indic Handwritten OCR

Abstract

Learning from the Wild, Unlabeled Web

Handwritten Text Recognition (HTR) remains a challenging task, especially in Indic languages, due to varied handwriting styles, complex character structures, and the scarcity of annotated data. While deep learning models have shown promising results on curated benchmarks, their performance often degrades in real-world scenarios with high variability in writing.

To address this, we propose a semi-supervised approach, SemiHastakshar, that leverages large-scale unlabeled word-level images collected from diverse sources across the Internet. We employ a high-confidence pseudo-labeling strategy to train on unlabeled samples iteratively. It allows the model to learn from a wider distribution of handwriting styles and improve generalization across writers. Our method demonstrates that combining a small amount of labeled data with a large unlabeled corpus leads to more robust HTR models for Indic scripts, advancing scalable recognition in low-resource settings.

Key Contributions

What We Contribute

1. Indic-HW-Wild dataset

A large-scale collection of unlabeled word-level handwritten images spanning nine Indic languages, sourced from diverse online repositories. Public on Hugging Face →

2. Semi-supervised training framework

A generalized multi-script OCR model that leverages unlabeled samples via pseudo-labeling to improve recognition across varied handwriting styles.

3. Dataset-specific fine-tuning without labels

Significant accuracy gains from fine-tuning on unlabeled target-domain data alone, enabling personalized recognition in low-resource settings.

4. Data–performance ablation

An extensive ablation quantifying how the amount of labeled data affects final performance, to understand the data–performance trade-off.

Method

High-Confidence Pseudo-Labeling

Starting from a base recognizer, SemiHastakshar runs the current best model over the unlabeled Indic-HW-Wild pool, keeps only predictions above a confidence threshold as pseudo-labels, and retrains on a mix of labeled and pseudo-labeled samples. This repeats for several cycles, each time refreshing the "best model" used to generate the next round of pseudo-labels.

SemiHastakshar self-training architecture diagram

Figure 4. Architecture of SemiHastakshar. Frozen components are denoted by a snowflake icon, trainable components by a fire icon. The process runs for N cycles, each selecting m samples from both labeled and pseudo-labeled data.

Results

Experimental Evaluation

SemiHastakshar is compared against a PARSeq HTR baseline trained only on labeled data, both in-domain (IIIT-INDIC-HW-WORDS) and, more importantly, out-of-domain on unconstrained real-world handwriting (IIIT-INDIC-HW-UC).

Out-of-Domain Comparison

WER and CER on the unconstrained IIIT-INDIC-HW-UC test set — where the gains matter most. Bold indicates our best result.

Language	PARSeq HTR (baseline)		SemiHastakshar
Language	WER↓	CER↓	WER↓	CER↓
Hindi	53.00	31.94	38.62	26.49
Bengali	41.71	21.58	29.09	14.86
Telugu	60.56	30.58	41.66	18.12
Gujarati	78.39	66.57	70.70	54.73
Punjabi	43.79	29.32	43.53	28.68
Kannada	65.06	45.71	60.53	41.18
Malayalam	59.38	55.43	55.29	44.61
Tamil	54.99	29.77	45.02	21.13

Qualitative comparison between PARSeq HTR baseline and SemiHastakshar predictions

Figure 6. Out-of-domain evaluation on the IIIT-INDIC-HW-UC dataset comparing PARSeq HTR with SemiHastakshar. Ground truth text is shown in blue, while incorrectly predicted characters are highlighted in red.

Ablation: Effect of Labeled Data Size

Reducing supervised pretraining data from 60k–80k to 30k samples, on the IIIT-INDIC-HW-UC dataset. Bold indicates the better (full-data) result.

Language	Full Data (60k–80k)		30k Samples
Language	WER (%)	CER (%)	WER (%)	CER (%)
Hindi	38.62	26.49	57.89	58.73
Telugu	41.66	18.12	66.16	54.29
Bengali	29.09	14.86	41.43	22.87

Reducing labeled data from 60k–80k to 30k leads to a consistent, substantial increase in WER and CER — sufficient supervised pretraining is crucial before the model can benefit from unlabeled data; semi-supervised fine-tuning alone cannot compensate for limited initial supervision.

Citation

BibTeX

@inproceedings{evani2025semihastakshar, abbr = {ICVGIP}, title = {SemiHastakshar: Generalizable Indic Handwritten OCR through Semi-Supervised Learning}, author = {Evani, Lalitha and Mondal, Ajoy and Jawahar, C. V.}, booktitle = {Proceedings of the 16th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP)}, pages = {1--9}, publisher = {ACM}, year = {2025}, doi = {10.1145/3774521.3774605}, url = {https://dl.acm.org/doi/abs/10.1145/3774521.3774605} }