Compression-Aware Scaling Laws for Protein and Genomic Sequence Models

TL;DR: Let’s see if the magic of DLCM’s compression-aware scaling applies to protein and genome modeling! By adapting hierarchical compression and scaling laws to models like Evo, we can test if concept-level reasoning and resource allocation translate to biological sequences. A first step could involve defining “biological concepts” (e.g., motifs, domains) as variable-length units for compression.

Research Question: Can compression-aware scaling laws, as developed in DLCM, be effectively transferred to protein/genome sequence models to optimize compute allocation and functional reasoning?

Hypothesis: Hierarchical compression of biological sequences into meaningful “concepts” (e.g., motifs, structural domains) will enable better scaling of functional prediction and sequence generation under fixed compute, mirroring the benefits in language modeling.

Experiment Plan: Adapt DLCM’s semantic compression mechanism to identify variable-length biological concepts in sequence data (using Evo’s datasets). Define and measure concept-level reasoning capacity in tasks like zero-shot protein function prediction or CRISPR system design. Derive and empirically validate scaling laws linking token- and concept-level capacity, compression ratio, and task performance.

References:

• Qu, X., Wang, S., et al. (2025). Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space.
• Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S., Brixi, G., Sullivan, J., Ng, M. Y., Lewis, A., Lou, A., Ermon, S., Baccus, S., Hernandez-Boussard, T., Ré, C., Hsu, P. D., & Hie, B. L. (2024). Sequence modeling and design from molecular to genome scale with Evo. Science.