this post was submitted on 06 Aug 2024
332 points (98.5% liked)
Firefox
17302 readers
115 users here now
A place to discuss the news and latest developments on the open-source browser Firefox
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
@neme loaded questions are loaded.
The "Want most" to "Want least" scale is loaded AF.
Where is the option for "I don't want any of these things"?
Edit: Yeah, fuck that. That survey is bullshit. I stopped bothering to give answers due to the multi-choice questions seeming like a way for Mozilla to have a wank about itself.
This is fairly standard survey design, I believe. They're not looking to know which features are wanted in general; they want to know their relative popularity. The sets you're presented are randomised (i.e. we don't all get to see the same sets), which allows them to get a ranked list of lots of potential features, while only having to run ten survey questions per participant.
If you get a set with three features that everyone likes or dislikes at about the same level, then it doesn't really matter want you answer: they'll all end up at the top or bottom of the list, respectively. Because each of those options also get presented as part of different sets to different users, where different answers can win out.
You're bang on. It's called MaxDiff. I use it frequently in my line of work to prioritise product or service messaging with panel data. It's better in some cases to use Inferred preference rather than stated, but generally good to keep the options comparable in "size" of offer.
I would never interpret a MaxDiff model low end result as "wow, 5% of people want slower browsers." Instead I'm focusing on the top cluster. As with any model, they're only ever so accurate. Don't read into the questions too much.