A Multi‑Objective Generator for Novel‑Yet‑Feasible Ideas

Problem: RLHF and conventional decoding push models toward safe, high‑likelihood, template answers (“same joke every time”). Pushing only for novelty yields incoherent or off‑task outputs.

Hypothesis: A generator that optimizes two objectives in parallel—(a) novelty (semantic distance, concept incongruence, anti‑template) and (b) feasibility (coherence, constraint satisfaction, style)—and that verbalizes its sampling choices via structured, non‑CoT sampler notes will produce diverse, high‑quality stories.
We operationalize this with Pareto search over candidates, logit/embedding perturbations, and a learned reward model for feasibility.

2. Core ideas (what’s new)

Pareto decoding: Maintain a frontier of candidates that are non‑dominated on {novelty, feasibility}. Don’t collapse to a single weighted sum until selection time.

Verbalized sampling (structured, not chain‑of‑thought): At beat‑level (every few sentences), the model emits a Sampler Card:

concept_sources: [“astronomy”, “cooking”]

relation: “analogy” | “juxtaposition” | “causal twist”

risk: medium

constraints_met: [genre, PG-13]
These cards explain concept combination and feed back into rewards; they’re short metadata, not step‑by‑step reasoning.

Concept‑incongruence objective: Encourage unlikely but meaningful pairings by rewarding semantic distance between concept clusters with a bridge relation (e.g., analogy/metaphor) so we don’t reward nonsense.

Controllable noise schedules: Inject noise where it matters (concept choice, scene beats), not at every token. Use a per‑segment temperature / top‑p schedule and low‑rank logit perturbations targeting over‑used templates.

Anti‑confirmation bias via logit/embedding search:

Baseline tail search: keep one beam near the vanilla RLHF distribution.

Anti‑template head: maintain another beam with penalty on tokens/phrases that dominate typical decodes (“chicken road” jokes, tired tropes).

(generated by the conversation between me and GPT Pro)