The core claim
AI agents do not experience interfaces the way humans do. Humans infer meaning from layout, hierarchy, colour, copy, and context. Agents need meaning to be encoded.
This creates a new design problem: the interface must become readable as a contract, not just visible as a screen.
The web has semantic standards. Engineering has schema contracts. What product design lacks is a design-owned semantic contract that connects product meaning to interface expression.
That is the white space this essay names. It is not that semantics exist nowhere. It is that design systems do not yet treat semantic meaning as a first-class design output.
What already exists
This shift does not start from zero:
- ARIA already gives interfaces roles, states, and properties for assistive technologies.
- Schema.org already gives web content structured meaning, including ApplyAction, JobPosting, and the Job posting property.
- Google's job posting structured data defines direct apply in terms of the actions required from the user.
- Design tokens already separate design decisions from implementation, and the Design Tokens Community Group works on a shared format for design tokens across tools and codebases.
- MCP and similar protocols show that AI systems increasingly need structured access to tools, data, and context.
- Headless CMS content modelling has already separated semantic fields from display fields.
The missing piece is a design-owned layer that binds these ideas together at the level of product meaning.
Designers should be deciding what meaning needs to be encoded. Today, that work is usually scattered across accessibility, content modelling, schema markup, analytics, and engineering implementation.
Three layers, one missing
| Layer | What it encodes | Who reads it |
|---|---|---|
| Visual layer | Colour, spacing, shape, hierarchy | Human perception |
| Semantic layer | Meaning, intent, role, state, context | Agents, AI, screen readers, crawlers |
| Functional layer | Behaviour, interactions, state machines | Browsers, native runtimes |
Design systems are excellent at the visual layer. The functional layer is handled through component APIs and interaction specs.
The semantic layer is the missing contract.
It answers questions like:
- What is the domain role of this component? This is a commitment action, not just a primary button.
- What is the ontological relationship between these data fields? This salary range belongs to this job listing, not this recommended job.
- What is the confidence or provenance of this content? This result was AI-generated, not human-authored.
- What are the reversibility semantics of this action? This is destructive and irreversible.
These are design decisions. They have always been design decisions. We just never had to make them explicit before, because humans could infer them. AI cannot.
What the designer's semantic layer is not
The designer's semantic layer is not:
- just ARIA, though it should include accessibility semantics
- just schema.org, though it can publish structured data
- just design tokens, though tokens may carry semantic properties
- just an API contract, because it describes interface meaning, not only system behaviour
- just metadata, because it is governed as part of the design system
The designer's semantic layer is the governance layer that decides what meaning should be encoded, sitting above any specific technical implementation.
Why now: the agent reading problem
Agents and AI copilots interact with product interfaces in three primary modes:
- DOM traversal — reading the accessibility tree and HTML structure
- Screenshot parsing — interpreting rendered UI from vision models
- API or schema consumption — reading structured data directly
The reliability order runs in reverse: schema is most reliable, DOM is moderate, and screenshot parsing is most fragile.
Here is the design crisis: most product interfaces are optimised for screenshot parsing by humans, and almost entirely unoptimised for schema consumption by agents.
When a vision model reads a job listing, it sees text that looks like a salary, a title, and a location. But it has no way to know:
- whether the salary is gross or net
- whether the job is still accepting applications
- whether the employer is verified
- what Apply will do to the user's account state
- whether this listing is the canonical record or a promotional duplicate
These facts are in the heads of the designers and product managers who built the interface. They are not in any machine-readable contract that agents can consume.
The designer's semantic layer makes them explicit.
The three pillars of semantic interface design
1. Semantic tokens: from pixels to meaning
The next evolution of design tokens extends from visual tokens to semantic tokens: tokens that carry domain meaning alongside presentation.
# Current token: visual
color.action.primary: "#0d5eff"
# Semantic token: meaning-carrying
component.action.commitment.primary:
visual: color.action.primary
semantics:
intent: "user-commitment"
reversibility: "irreversible"
domain: "application"
risk_level: "high"
A component consuming this token does not only render the right colour. It also carries metadata that a consuming system can read and reason about.
2. Interface contracts: the design-owned semantic agreement
A semantic interface contract is the machine-readable agreement between a product model and a UI surface. It states what the interface means, what actions do, what data can be trusted, and how human or AI actors should reason about it.
component: ApplyButton
contract:
domain_role: "commitment-action"
outcome: "initiates_application_flow"
preconditions:
- user: "authenticated"
- job: "accepting_applications"
postconditions:
- creates: "draft_application"
- notifies: "employer"
reversibility: "partially-reversible"
reversibility_window: "before_application_submitted"
ai_instruction: |
This action is consequential. Confirm intent before invoking.
Do not trigger as part of bulk operations without explicit user consent.
accessibility_role: "button"
schema_type: "schema.org/ApplyAction"
schema.org/ApplyAction already exists. The interface contract does not replace it; it governs it.
The designer decides this component should express an ApplyAction with these specific preconditions and this level of risk. Engineering encodes that decision. The schema.org vocabulary is the implementation; the interface contract is the design specification.
Designers should define the contract. Design systems should govern it. Engineering should encode and expose it.
3. Metadata as design material
Metadata is not decoration around the interface. It is part of the interface.
Today, metadata often enters a product at the engineering layer: aria-label, data-* attributes, schema microdata, Open Graph tags, and other structured hints added as implementation detail. In an AI-native product, this is backwards.
The designer's job is not only to show distinctions visually. It is to encode them semantically so that any intelligent system can reason about them correctly.
Metadata as design material means:
- information architecture includes semantic annotation, not just visual hierarchy
- component design includes semantic property design alongside visual property design
- content strategy includes machine-readable meaning, not just human-readable copy
- design reviews include semantic correctness checks, not just visual quality checks
The design tool ecosystem is not ready for this. Figma does not have a semantic annotation layer. Storybook does not systematically expose semantic contracts. This is the gap that practitioners must lead, and that toolmakers will follow.
The semantic interface contract framework
Product model
│
▼
Semantic interface contract
├── Domain ontology
│ └── What entities and relationships exist
├── Component semantics
│ └── What each component means in this domain
├── Action semantics
│ └── What outcomes actions produce
├── Content provenance
│ └── Who or what generated this content, with what confidence
├── Access semantics
│ └── Who can see or do what, and under what conditions
└── Agent instructions
└── How AI actors should reason about this surface
│
▼
Visual and functional interface
The semantic interface contract is authored at design time, encoded at build time, and read at runtime by humans, machines, agents, and AI systems.
Where the semantic layer lives
The designer's semantic layer does not live in one tool. It is expressed across the product system:
- in design tokens, where intent and risk can be attached to visual decisions
- in component contracts, where roles, states, and outcomes are specified
- in content models, where entities and relationships are defined
- in structured data, where public meaning is exposed
- in accessibility semantics, where interaction meaning is made available to assistive technology
- in agent instructions, where AI actors are told how to treat sensitive actions
The point is not to create another isolated artefact. The point is to govern meaning across the whole design-to-code pipeline.
What this changes in practice
Design system governance
Components require a semantic specification alongside their visual specification. A new button variant is not shipped until its domain role, reversibility semantics, and agent-facing contract are defined.
Information architecture
IA deliverables include a semantic data model — entities, relationships, and properties — not just site maps and flows.
Design reviews
Review criteria include:
- Does this surface correctly encode the business rules it represents?
- Could an agent make the wrong decision based on its structure?
- Are role, state, provenance, and risk clear to both humans and machines?
Design-to-code handoff
Semantic annotations are part of the design specification, not implementation detail. Engineering does not infer the semantic layer from visual mocks; it implements the semantic decisions made by design, product, content, data, and accessibility partners.
Why design must own this
A visual design system can standardise how a product looks. A semantic design system standardises how a product expresses meaning.
That difference matters because AI agents do not only need access to screens. They need access to intent, state, trust, risk, and consequence.
Engineers can encode a schema. Designers, working with product, content, data, and accessibility partners, must decide whether that schema reflects the meaning users rely on.
Semantic correctness is inseparable from domain correctness.
The next design system will not only describe how an interface looks. It will describe what the interface means.
Terminology reference
Designer's semantic layer
The design-owned system of machine-readable meaning that connects product models to interface components. Defines roles, states, intent, provenance, permissions, risk, and outcomes independently of visual rendering.
Semantic interface contract
The machine-readable agreement between a product model and a UI surface. States what the interface means, what actions do, what data can be trusted, and how human or AI actors should reason about it.
Semantic token
A design token that carries domain meaning and machine-readable metadata alongside its visual properties.
Metadata as design material
The principle that structured metadata is a first-class design artefact, authored by designers rather than added later as engineering detail, because metadata is not decoration around the interface; it is part of the interface.
Agent reading problem
The challenge that AI agents cannot reliably infer domain semantics from visual presentation alone, creating a structural gap between what interfaces mean to designers and what they communicate to machine actors.
Interface contract
A formal specification of what a UI surface promises to any consuming system about its meaning, behaviour, and affordances. Distinct from an API contract in that it describes interface meaning, not only system behaviour.
FAQ
What is semantic design?
Semantic design is the practice of encoding machine-readable meaning into product interfaces and design systems, not just visual presentation and functional behaviour, so that any consuming system can understand what an interface means, not just what it looks like.
What is an AI-native design system?
An AI-native design system treats machine-readable semantic meaning as a first-class output alongside visual tokens and component APIs. It defines interface contracts, semantic tokens, and structured metadata as part of the design specification.
What is a semantic interface contract?
A semantic interface contract is the machine-readable agreement between a product model and a UI surface. It defines domain role, action outcomes, preconditions, reversibility, content provenance, and agent-facing instructions for a component or surface.
It is authored by designers, governed by design systems, and encoded by engineering.
How is a semantic layer different from design tokens?
Design tokens encode design decisions. Today, that usually means visual decisions such as colour, spacing, typography, elevation, and motion.
Semantic tokens extend this to encode domain meaning: intent, reversibility, risk level, and domain role. The semantic layer uses tokens as one mechanism, but also includes interface contracts, content provenance models, access semantics, and agent instructions that go beyond current token specifications.
How should product designers design for AI agents?
Product designers should treat metadata as a design material alongside visual and functional properties. They should define interface contracts for high-stakes components, specify content provenance for AI-generated content, and include semantic correctness as a design review criterion.
About the author
Richard Simms is Principal Product Designer at SEEK, where he leads design for the Career Discovery Agent and Career Feed. He also builds Sentiuma, a personal AI knowledge infrastructure layer.