OTAKU AI DŌJŌ was one of those competitions that looked straightforward at first glance and then quickly turned into a problem about everything except “just classification.” The task was to classify anime characters across 630 classes, but the real difficulty came from the fact that the test set was visually very different from the training set. The training data had two known styles, while the test data contained six unfamiliar styles. In other words, the model was not only being asked to recognize characters, but to survive a serious domain shift across art styles.
There was also a bonus objective hidden inside the dataset: identifying human images of alumni. We managed to find them, and we were the second team to do so, which was a nice early validation that the dataset exploration work was pointing in the right direction.
What made this competition interesting for me was how quickly the solution evolved. I did not begin with the final ViT pipeline. I started by trying to understand the data structure, then tested simple baselines, then tried more style-aware and retrieval-like approaches, and only after that settled on the final model that actually worked best.
My first useful breakthrough came from clustering the images by character and style. I created character-wise clusters and then used those cluster centers to measure distances and find outliers. That turned out to be much faster and more practical than some of the other ideas I explored, including CLIP-based similarity methods. From that analysis, I isolated 34 strong outliers from the 150 suspicious samples I had initially looked at. That was an important step because it helped clean the dataset early and made the training set less noisy.
More importantly, the clustering gave me a strong visual sense of the test distribution. I generated a style clustering visualization that revealed eight distinct test-set style groupings. That plot became one of the most important EDA artifacts in the whole competition, because it confirmed that the test images were not coming from a single uniform domain. The known training styles were easy to confirm: one was cel-shaded anime character illustration, and the other was black-and-white manga comic panels. The rest of the clusters were interpreted from appearance only, not from ground-truth labels, and they suggested a mixture of semi-realistic anime paintings, modern game-art-like styles, soft pastel styles, sketch-like illustrations, retro low-resolution images, and highly stylized fan-art-like images.
That clustering work was valuable even though the clustering itself did not become the final solution. It explained why the problem was hard: the model had to generalize across style boundaries, not just character boundaries.
Like most Kaggle workflows, the first thing I wanted was a baseline that would tell me where I stood. I tried standard ResNet-style classification and also a more generic ViT-style architecture. Those baselines were useful not because they were final contenders, but because they told me that the problem was not going to be solved by ordinary image classification alone.
The issue was not just model capacity. It was the mismatch between training style and test style. A model that looked decent on familiar styles could still fail badly when the art style changed. That pushed me away from plain supervised baselines and toward models that had a stronger prior for anime-specific visual structure.
Once I understood the domain shift, I explored more style-aware ideas. CLIP-based approaches were part of that exploration, along with other ways of measuring similarity between test images and training groups. These ideas made sense conceptually, but they were not the fastest path to a strong solution in the competition window. The clustering approach also made me think about whether I should explicitly route samples by style or train some kind of style classifier first. That looked elegant on paper, but in practice it was not giving the kind of reliable improvement I needed. It helped with analysis, but not enough with leaderboard performance.
That was the point where I realized that the final solution probably had to be simpler and more robust.
After the early baselines, I shifted toward anime-specific pretraining. I also explored DeepDanbooru-style tagging models because they carry strong priors for anime imagery and fine-grained visual patterns. That direction was promising as a complementary branch, especially for local texture and character-detail recognition. But the final notebook tells the real story: the strongest result came from the anime-pretrained ViT path, not from a complicated ensemble. The best code that survived into the final run was a single model pipeline built around hf_hub:animetimm/vit_base_patch16_224.dbv4-full.
That choice made sense. A ViT pretrained on anime-specific data is much better positioned for this competition than a generic image classifier, because it already knows a lot about the visual language of anime art. It can capture global structure, face layout, costume patterns, and character identity more effectively than a standard backbone trained on generic natural images.
The final notebook is very clear about the winning setup. The model was a custom wrapper around timm.create_model('hf_hub:animetimm/vit_base_patch16_224.dbv4-full', ...) with num_classes=630 and img_size=224. That image size was not just a detail; it was essential. If the image size was left inconsistent, weight loading could fail or the model could silently drift into an incompatible configuration. Keeping the resolution fixed at 224 was the correct choice and also made inference much faster.
The training objective was Focal Loss with gamma=2.0. That was a strong choice for a dataset with imbalance and many hard classes. Instead of treating all examples equally, Focal Loss emphasizes the difficult ones, which is useful when some classes are underrepresented or visually confusing. The optimizer was AdamW with learning rate 5e-5 and weight decay 1e-4. The scheduler was CosineAnnealingLR with T_max=5. The setup also used AMP or mixed precision, cudnn.benchmark=True, and GradScaler for speed and stability. These were not glamorous tricks, but they mattered a lot in a short competition.
The validation strategy used StratifiedKFold with n_splits=5, shuffle=True, and random_state=42. In the final notebook execution, only the first fold was run, but the code was built around the broader fold-based setup. Macro F1 was used as the validation metric, which was appropriate for the multi-class nature of the task.
Class imbalance was handled with a WeightedRandomSampler built from inverse class frequencies. That was another important stabilizer. Instead of allowing frequent classes to dominate every batch, the sampler forced the model to see rare classes more often, which made the training much more balanced.
The final training augmentations were strong, but not excessive. I used Resize to 224x224, HorizontalFlip with p=0.5, ShiftScaleRotate, ColorJitter, CoarseDropout, and ImageNet normalization. This was the right balance for the competition. The augmentations improved robustness to style variation and local corruption without damaging the character identity signals too much.
I had also explored more aggressive and more experimental augmentation ideas earlier, including stronger batch-level mixing and other domain-robustness tricks, but the final notebook shows that the stable core was the simpler image-level augmentation pipeline above. In a short competition, the best augmentation is often the one that improves generalization without making training unpredictable.
The final test-time pipeline was deliberately simple and fast. The test images were resized to 224x224, normalized with ImageNet statistics, and passed through the model in batches of 128. Inference used AMP as well, which kept it efficient.
For TTA, the final setup used only two passes: the original image and a horizontal flip. Their logits were averaged, and then the final class was chosen by argmax. Earlier, I had considered more stochastic TTA variants, including brightness-based noise, but those added randomness without giving a dependable gain. For this competition, deterministic flip-based TTA was the right trade-off between speed and robustness.
That simplicity mattered. The model had to be strong enough to generalize, but inference also had to be stable and fast enough to avoid submission headaches.
A big part of the final blog should honestly explain what did not make it. The character-wise clustering helped with EDA and outlier removal, but I did not turn it into a separate style-routing classifier. The CLIP experiments helped me think about the problem, but they were not the fastest path to a strong leaderboard score. The DeepDanbooru and ensemble direction was useful to explore, but the final best-performing submission came from the single anime-pretrained ViT rather than from a heavier blend.
That is actually an important lesson: in short competitions, a technically elegant idea is not automatically the best idea. The best idea is the one that is both valid and dependable under time pressure.
Looking back, the final solution worked because it matched the problem structure. The competition was not just about having a classifier. It was about handling style shift, class imbalance, and limited time. The anime-pretrained ViT gave strong prior knowledge about the image domain. The heavy but controlled augmentation improved style robustness. Focal Loss and weighted sampling helped with imbalance. The 224px input size kept the pipeline fast and stable. And the deterministic flip TTA gave a small but reliable inference boost without adding noise.
If I had to summarize the whole competition in one sentence, it would be this: I started by using clustering and outlier analysis to understand the style shift, tried standard ResNet and ViT baselines, explored CLIP and DeepDanbooru ideas, and then won by simplifying everything into a strong anime-pretrained ViT trained with Focal Loss, weighted sampling, heavy but controlled augmentation, and deterministic flip TTA.