Multi-Task and Cross-Domain Test-Time Training: Generalization Beyond Language

TL;DR: Can the TTT-E2E paradigm be generalized to other domains, like vision, audio, or multi-modal models? Let’s try adapting TTT-E2E for multi-task and cross-domain continual learning.

Research Question: Does E2E test-time training provide similar scaling and adaptation benefits outside language modeling, in vision, audio, or multi-modal tasks?

Hypothesis: TTT-E2E will enhance model adaptability and memory across diverse input modalities, but may require domain-specific adaptations (e.g., new loss functions, meta-learning objectives).

Experiment Plan: - Adapt TTT-E2E to U-Net or vision transformers for biomedical image segmentation (inspired by TTT-Unet).

• Apply TTT-E2E to time-series models for vehicle state estimation, leveraging continual learning techniques.
• Benchmark on domain transfer tasks (e.g., adapting image segmentation models to new imaging modalities at test time).
• Analyze scaling, generalization, and adaptation speed compared to domain-specific baselines.

References:

• Tandon, A., Dalal, K., Li, X., Koceja, D., Rod, M., et al. (2025). End-to-End Test-Time Training for Long Context.
• Zhou, R., Yuan, Z., Yan, Z., Sun, W., Zhang, K., et al. (2024). TTT-Unet: Enhancing U-Net with Test-Time Training Layers for Biomedical Image Segmentation. arXiv.org.
• Hosseinzadeh, A., Khoshnevisan, L., Pirani, M., Chenouri, S., & Khajepour, A. (2025). An Efficient Continual Learning Framework for Multivariate Time Series Prediction Tasks with Application to Vehicle State Estimation. arXiv.org.