Task-Specific Neural Thickets in Multi-Objective or Federated Learning Environments

TL;DR: What happens to neural thickets when you have lots of users or objectives—like in federated or multi-task learning? We’ll investigate if expert density and ensembling strategies can help reconcile conflicting tasks or non-IID data.

Research Question: How do neural thicket properties manifest in federated and multi-objective learning, and can ensembles of local experts improve robustness and generalization in these settings?

Hypothesis: In federated and multi-objective problem settings, the local expert density around a shared pretrained model will be heterogeneous, and ensembling top-performing local perturbations will outperform either naive averaging or full retraining.

Experiment Plan: Simulate federated learning with non-IID client data; for each client, sample parameter perturbations and identify local task experts. Evaluate global ensemble strategies (e.g., ensemble of top-K local experts) versus baseline federated aggregation (e.g., FedAvg). Apply to real-world federated datasets and multi-objective benchmarks. Measure task-specific accuracy, robustness to data shift, and communication efficiency. Expected outcome: Ensembles of local neural thickets provide more robust and adaptable federated models.

References:

• Gan, Y., & Isola, P. (2026). Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights.
• Wang, X., & Jin, Y. (2025). Distilling Ensemble Surrogates for Federated Data-Driven Many-Task Optimization. IEEE Transactions on Evolutionary Computation.
• Zhou, Y., Wu, X., Wu, J., Feng, L., & Tan, K. C. (2024). HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models. arXiv.org.