Why we asked candidates more questions
SEEK's recommendation engine was performing well, but it had a known blind spot. Recommendations take into account a user's profile and browsing behaviour — and that fails to capture the nuance of what someone wants next in their career.
A candidate's history says "product designer in Melbourne". It cannot say "ready for a step up, open to contract work, done with agencies". Generative AI looked like the way to close that gap: ask candidates directly, in their own terms.
What we built
Instead of static forms, we designed a conversational GenAI interface. Candidates described their ideal role by choosing from dynamically generated answers; the AI generated questions based on user segmentation and previous responses, adapting to different candidate needs.
It felt more like a helpful career advisor than a form. And by the numbers that describe the interaction itself, it worked: candidates engaged deeply, spent more time, gave richer context. Roughly 85% of those who started completed all eight questions.
The numbers that disagreed
Then the experiment data came in, and the two halves of the story refused to line up.
People will answer short, structured questions — the information gain was real. But the search-based refinement flow lowered apply conversion and job-detail views per visitor. The funnel said what the engagement metrics could not: we had made the product worse at its job.
Friction timing beats information gain
The post-mortem insight was about when, not what.
We had placed a thoughtful, well-crafted conversation in the middle of a high-intent moment. A candidate arriving at a job feed wants jobs, and we answered with questions. Every signal we gained was paid for with a delay at the point where delays are most expensive.
Worse, the loop never closed in the moment. Candidates answered eight questions, then saw recommendations that still didn't feel right — with no simple way to adjust what they were seeing. That gap is corrosive: it teaches users that giving the system information doesn't change what the system does. Trust drops, and engagement with the feed drops with it.
What we did with the lesson
The experiment killed the standalone flow and set the direction that followed: micro-questions asked inline, in context, one at a time — each answer visibly changing the feed in real time.
The transferable rule for anyone building AI products: measure your feature by the user's goal, not the model's appetite. Signal capture is a cost the user pays. If the product cannot pay them back in the same session, you are borrowing trust you will have to return with interest.
