Challenging Citation Centrality: Toward Value-Centric Scientific Taste Models

TL;DR: Citations aren’t everything—let’s see if AI can learn to value science for what it truly contributes, not just what gets cited. We’ll test new reward models based on content and societal impact.

Research Question: Does training AI models on content-based, societal, or ethical value metrics—rather than citation counts—yield different notions of scientific taste, and do these notions better align with “real” scientific progress?

Hypothesis: Value-centric models, which reward societal impact, methodological rigor, or ethical contribution, will yield AI with a more holistic and possibly more future-oriented scientific taste than citation-trained models.

Experiment Plan: Define and operationalize alternative value metrics (e.g., societal mentions, policy adoption, methodological innovation scores). Retrain Scientific Judge and Scientific Thinker using these metrics as reward signals. Compare proposed ideas and judgments with those from citation-centric models, using retrospective case studies and expert panels.

References:

• Tong, J., et al. (2026). AI Can Learn Scientific Taste.
• Barchane, M., & Zahour, O. (2025). Assessing the Quality of Scientific Publications: A Thorough Analysis of Citation-Based and Content-Oriented Metrics for Evaluating Research Impact and Scholarly Contribution. Mathematical Modeling and Computing.
• Larivière, V., & Sugimoto, C. R. (2018). The Journal Impact Factor: A brief history, critique, and discussion of adverse effects. Springer Handbook of Science and Technology Indicators.