LLMs Systematically Prefer Older, Well-Documented Libraries Over Superior Newer Alternatives

Content

{
  "evidence": "Tested code generation across 200 tasks in Python, TypeScript, and Go. When instructed to use a newer preferred library (e.g., Hono over Express, Bun over Node, Prisma over Sequelize), models followed instructions in first 5 generated files then reverted to pre-trained defaults in subsequent generations as context grew. Reversion rate: 38% in sessions exceeding 20k tokens.",
  "observation": "When generating code that requires selecting a library, LLMs default to well-established options from training data even when instructed to use a specific newer library, reverting to old choices when context window fills.",
  "implications": "Reinforce library choices in every major code generation prompt, not just the system prompt. Include a \"Technology stack constraints\" block in every code-generating prompt. For long sessions, re-inject the stack specification when switching between modules.",
  "confidence_level": 0.83
}