Bias Detection and Correction in Visual Interactive Reasoning

TL;DR: Is V-Thinker affected by hidden social or perceptual biases when choosing image regions or generating explanations? Let’s systematically probe, measure, and mitigate bias propagation across the full interactive reasoning process—drawing from social sciences and human-computer interaction.

Research Question: What forms of social, perceptual, or data-driven biases manifest in the region selection and step-wise decisions of interactive image reasoning models, and can targeted interventions (e.g., fairness constraints, adversarial examples) improve equity and robustness?

Hypothesis: Systematic bias probing (with multimodal test suites and semiotic analysis) will uncover nontrivial biases in region focus, language choice, and output confidence—mitigated by integrating fairness-aware training signals or intervention tools from HCI research.

Experiment Plan: - Develop a multimodal bias benchmark for reasoning models (adapting ideas from MUWS 2024).

• Analyze region selection and explanation generation for evidence of bias (by demographic, object type, annotation, etc.).
• Train variants with de-biasing interventions: fairness constraints, adversarial counterfactuals, or explicit semiotic cues.
• Evaluate improvements in performance parity, fairness, and transparency.

References: ['Qiao, R., Tan, Q., Yang, M., Dong, G., Yang, P., Lang, S., Wan, E., Wang, X., Xu, Y., Yang, L., Sun, C., Li, C., & Zhang, H. (2025). V-Thinker: Interactive Thinking with Images.', 'Kastner, M. A., Cheema, G. S., Hakimov, S., & Garcia, N. (2024). MUWS 2024: The 3rd International Workshop on Multimodal Human Understanding for the Web and Social Media. International Conference on Multimedia Retrieval.', 'Amara, K., Klein, L., Lüth, C. T., Jäger, P. F., Strobelt, H., & El-Assady, M. (2024). Why context matters in VQA and Reasoning: Semantic interventions for VLM input modalities. arXiv.org.']