In-context If-Then Capacity

by Ari Holtzman2 months ago
1

LLMs have gotten pretty good at instruction following. However, I have noticed that if I set certain instructions to trigger conditionally (make sure that every time you mention a dog you also mention a cat in this story) they often misfire (both false positives and false negatives). Is the models ability to follow instructions conditionally somewhat consistent over different kinds of instructions? If so, can we measure its capacity to follow such instructions?

llmsinstruction-followingevaluationImplemented:https://github.com/Hypogenic-AI/incontext-if-then-2dae-claudeImplemented:https://github.com/Hypogenic-AI/incontext-if-then-31c6-codexImplemented:https://github.com/Hypogenic-AI/inctx-if-then-capacity-6ce0-gemini
Chat

If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:

@misc{holtzman-incontext-ifthen-capacity-2026,
  author = {Holtzman, Ari},
  title = {In-context If-Then Capacity},
  year = {2026},
  url = {https://hypogenic.ai/ideahub/idea/rgdEAXxrE5GSUL18JONM}
}

Comments (0)

Please sign in to comment on this idea.

No comments yet. Be the first to share your thoughts!