I remember the first time I set up an online account for Xbox Live to play Halo and selected all of the options for "Prefer players who use voice chat" and other similar options to find people to actually play with.
The next day, I'm having an awful experience and my friend told me, "You know that's not like.. an 'in general' kind of thing, right? Do you really like listening to Halo players on Xbox live?"
Turned them all off. Never looked back.