PURER++: Data-Free, Cross-Architecture Meta-Learning with Multimodal Episode Curricula

Hu et al. (2023) introduced PURER to extract prior knowledge from collections of pretrained models via episode curriculum inversion, but focused on vision and parameter-space constraints. Let’s extend this idea in three directions:

• Cross-modality: combine inversion for vision backbones with pseudo-episode construction for language (using multilingual embeddings like LaBSE or text-embedding-ada-002, cf. Rahimi & Veisi, 2024) and audio (spectral matching/wav2vec-style decoders). This creates a multimodal “episode curriculum” that ramps up difficulty based on real-time adaptation feedback, as in PURER.
• Self-supervision priors: adopt HateMAML’s (Awal et al., 2023) self-supervised objectives to regularize the meta-model in the absence of ground-truth labels, reducing mode collapse in synthetic episodes.
• Architecture- and scale-agnosticism: meta-learn across heterogeneous backbones (e.g., CNN, ViT, GNN as in Maqsood et al., 2024), using inversion calibration at meta-test to narrow train–test gaps.
  This differs from prior data-free meta-learning by spanning architectures and modalities, and from HateMAML by removing the need for target-language/task data during meta-training. It’s particularly impactful for privacy-restricted domains (clinical imaging, enterprise logs) where raw data are inaccessible but pretrained models exist. The result is a deployable meta-learner that adapts in few shots without ever touching the original training data.

References:

1. Model-Agnostic Meta-Learning for Multilingual Hate Speech Detection. Rabiul Awal, Roy Ka-Wei Lee, Eshaan Tanwar, Tanmay Garg, Tanmoy Chakraborty (2023). IEEE Transactions on Computational Social Systems.
2. Integrating Model-Agnostic Meta-Learning with Advanced Language Embeddings for Few-Shot Intent Classification. Ali Rahimi, Hadi Veisi (2024). 2024 32nd International Conference on Electrical Engineering (ICEE).
3. Model Agnostic Meta-Learning (MAML)-Based Ensemble Model for Accurate Detection of Wheat Diseases Using Vision Transformer and Graph Neural Networks. Yasir Maqsood, Syed Muhammad Usman, Musaed A. Alhussein, Khursheed Aurangzeb, Shehzad Khalid, Muhammad Zubair (2024). Computers, Materials & Continua.
4. Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning. Zixuan Hu, Li Shen, Zhenyi Wang, Tongliang Liu, Chun Yuan, Dacheng Tao (2023). Computer Vision and Pattern Recognition.