Changed the survey, not the sample
One week outdrew five weeks of the original
CIBT processes visas and passports for corporate clients and individual travelers. People were reaching the requirements page and the application funnel and leaving, and nobody could say why. I ran on-site polls across both pages to find out. When the first round returned thin, low-quality answers, I rebuilt the instrument rather than the sample.
Also in this role
- Designed and built responsive landing pages for corporate clients including Microsoft, Apple and IBM
- Maintained and redesigned components across multilingual company sites
What I was trying to find out
Two pages were losing people: the requirements page, where travelers check what a visa needs, and the application funnel itself. Analytics could say they left. It couldn’t say why.
Round one
I put open-ended polls on both pages and let them run for six weeks. They returned 420 responses, 291 on requirements and 129 in the funnel.
The volume was workable but the answers were vague. Opening with an open question asks someone to compose a thought before they’ve agreed to participate at all, and most people simply don’t.
Changing the instrument, not the sample
For round two I inverted the structure: lead with a binary yes/no, then ask the open question as a follow-up to whoever answered.
The effect was immediate. One week of round two drew more than double what five weeks of round one had. Roughly half of the people who answered the first question went on to answer the second, which meant the qualitative pool grew alongside the quantitative one instead of competing with it.
The answers also changed character. Round one produced procedural questions from people asking how to use the service. Round two produced declarative ones from people who had already found what they needed and hit a specific wall. More useful, and more specific.
What it found
Coding the free-text into themes put the blockers in order:
Mentions, coded from open-text responses on the requirements page
- Requirements not clear. The largest single category.
- Cost not clear. People couldn’t tell what the total would be, service fees included.
- Processing time not clear.
- Can’t find the documents. Letters of invitation and business cover letters especially.
- Too time-consuming. The theme both rounds agreed on.
Underneath those sat a repeated request the site had no answer for: let me save this and come back. People were gathering passports, photos and employer letters over days, and the funnel assumed a single sitting.
What I’d do differently
The round-one design was the mistake, and it cost five weeks to find out. I’d now treat the first question as a participation decision rather than a research question. Earn the response first, ask for the thinking second.