Semantic layers over LLM APIs: architecture patterns for product teams

A semantic layer is an abstraction that sits between your data sources and your LLM calls. It:

• Translates raw database records into model-legible descriptions
• Normalises inconsistent data into consistent context
• Enforces business rules about what the model can and cannot see
• Caches and optimises context injection for cost and latency

Without a semantic layer, every LLM feature becomes its own bespoke prompt engineering exercise. With one, you build the translation logic once and reuse it across features.

At SEEK, the shift to semantic layers for career context — translating job records, application history, and behavioural signals into structured model context — was the architectural change that made the GenAI Career Feed perform consistently at scale.

The semantic layer determines what the model can know about your users and your domain. That is a product design decision, not a backend implementation detail.

Designers need to engage with semantic layer design because:

• Context shapes output quality — a poorly designed semantic layer produces mediocre AI features regardless of model capability
• Abstraction choices affect UX — how you represent a user's "career trajectory" to the model determines what the model can say about it
• Privacy and trust boundaries — which user data enters the semantic layer is a consent and trust question
• Explainability — if you want to show users why the system made a recommendation, you need a semantic layer that preserves reasoning chains

Product designers who treat the semantic layer as purely an engineering concern will find themselves unable to improve AI feature quality through design iteration alone.

Entity-centric context — build semantic representations around business entities (User, Job, Application, Company) rather than raw tables. Each entity has a standard context representation that any LLM feature can use.

Layered context assembly — assemble context for each LLM call in layers: static entity context → dynamic user context → session context → task-specific context. Each layer adds specificity without rebuilding from scratch.

Context versioning — treat your semantic layer representations like API contracts. When you change how you represent a User to the model, you need to understand the downstream effects on every feature that uses that representation.

Evaluation-driven iteration — measure the quality of semantic layer outputs, not just LLM outputs. If the model produces poor results, diagnose whether the problem is in the model, the prompt, or the semantic layer before changing anything.

If you are building your first LLM-powered feature, start with a simple semantic layer:

1. Define the three or four entities most relevant to your feature
2. Write a prose description of each entity that a smart person with no context would understand
3. Include the fields that matter for your use case; exclude the noise
4. Test the description manually before writing any LLM integration code

This is not a technical exercise. It is a clarity exercise. If you cannot write a crisp description of what a "User" means in your product context, the model cannot either.