OK Cupid admits running experiments manipulating user matches: ok or not ok?

Unhappy couple upset with marital problems
Last month it came out that Facebook had conducted a secret study in 2012 of 689,000 random users in which they manipulated news feeds to be either positive or negative. The study found that moods were transmittable in facebook and that users did become happier or sadder as a result of the filtered posts. This experiment, only disclosed after the fact, brought up all sort of ethical and digital privacy issues. The public was outraged and facebook COO Sheryl Sandberg offered a weak half-apology citing research purposes.

Now OKCupid president Christian Rudder has admitted to manipulating user match data in an effort to fine tune the service. That seems like a legit reason, but the methods he mentions are incredibly questionable. If I was an OKCupid user I would be pissed. I’m going to quote People here because the original post they’re reporting on is a little TL;DR and technical, and you can read that here. Here’s People’s version:

OkCupid, the popular and free dating site, has been manipulating users’ compatibility in order to study and monitor dating behavior, the site revealed.

In a blog post that ran on Monday, company president Christian Rudder admits: “We might be popular, but OkCupid doesn’t really know what it’s doing … Experiments are how you sort all this out.”

Those experiments involved twiddling with the site’s compatibility data to make users think they’re a better match than they really are – and vice versa.

Case in point: In one experiment, the website told pairs who scored a 30 percent match that they were 90 percent compatible. OkCupid claims the ploy worked: “When we tell people they are a good match, they act as if they are. Even when they should be wrong for each other,” Rudder’s blog post says.

The site also flipped the experiment, fooling strong matches into thinking they weren’t right for each other. The results? Only a tenth of pairs moved past a single message to each other.

In another experiment, the site hid the profile text of a group of users to see how people would rate them based on their looks and photos alone. “Essentially, the text is less than 10 percent of what people think of you,” Rudder writes. “So, your picture is worth that fabled thousand words, but your actual words are worth … almost nothing.”

OkCupid’s mea culpa comes on the heels of controversy surrounding Facebook’s recent manipulative mood study, but the dating site has so far offered no apologies for tweaking those compatibility ratings. In fact, it’s digging in its heels.

“If you use the Internet, you’re the subject of hundreds of experiments at any given time,” says Rudder. “That’s how websites work.”

[From People]

If you read the original post that People is excerpting, some of the data is interesting and probably relevant to OKCupid’s matching service. They wanted to know if their matching algorithm works and how much match percentage, text and photos count when gauging compatibility.

Rudder’s post was incredibly arrogant though. These aren’t fruit flies he’s studying, these are people who are looking for relationships. When you’re offering people a service (and OK Cupid does have paid subscribers) you shouldn’t mess with the service without informing them, that’s dishonest. What’s worse is that this is something that’s highly personal and which many people feel vulnerable about. They’re putting themselves out there to find love. This isn’t a music matching service trying to get them to listen to an artist that’s not in their selected genre, they’re actually corresponding with and sometimes meeting the people they’re matched with. At best this was a waste of people’s time and at worst there are probably missed connections from people who trust OKCupid to help them find mates. Rudder just admitted that the experiments worked and that people who were actually compatible were less likely to correspond when they were lied to about their level of compatibility. That’s not something to brag about.

This statement lacked an apology or real numbers on how many people were affected and how long this experiment lasted. (There was “Love is Blind” day but Rudder didn’t indicate when and how the other experiments took place.) Just because other sites do it behind the scenes doesn’t make it ok. Also, Rudder has a book coming out in September. Many people think he’s revealing this now to promote his book. Hell he may have set up these experiments to have material for the book.

I asked a former OKCupid user to comment on this story, and she found it disturbing. She said that she doesn’t rely on the percentage matches but that it was wrong for OKCupid to do this. “I think it’s messed up they are playing with the data like that. To me what [people] write is important so that part of OKCupid hiding it is just wrong.”

Note: Header and frontpage images are not of Rudder. They are stock photos. A photo of Rudder is here.

Header photo credit: © Maridav – Fotolia.com. Photo below credit: MFer Photography on Flickr

Screen Shot 2014-07-30 at 8.07.42 AM

You can follow any responses to this entry through the RSS 2.0 feed.

46 Responses to “OK Cupid admits running experiments manipulating user matches: ok or not ok?”

Comments are Closed

We close comments on older posts to fight comment spam.

  1. I dunno, I’m having a hard time working up any outrage on this one. Kind of an extension of market research, and in the case of online dating, you’re basically going to end up on a blind date no matter what, so the outcome isn’t really changing. You still have to do the legwork and get to know the person. That is not different.

    Ostensibly, the “matching” parameters are changing, but the company itself has just admitted not really knowing what they are doing, so The scientific rigour of the so called “data” they are manipulating is likely going to not be very statistically significant, or even pertinent anyway.

    I’m a science nerd.

    • bbvnq says:

      Not okay. You don’t screw with people’s lives.

      • hunter says:

        You’re still totally making your own decision with your life. This isn’t like breaking our health care system or anything.

      • Jarredsgirl says:

        Hunter, it is setting precedent. If people don’t care that they are being secretly manipulated when it comes to love and facebook, then what is to stop people from doing more dangerous experiments? it is highly unethical that OK cupid are offering a service and then using it to do secret experiments that actually affect outcomes within people’s lives. I don’t believe it is ever acceptable to conduct experiments on people without their informed consent… It’s just not fair. They were not briefed on the way it would affect them and given a choice. I’m actually surprised that this is not illegal and that I have not heard about it. I’m not on facebook, and I always knew that it was a little bit more than just a social networking site.
        I think the thing with facebook is that if they were deliberately making people feel sad, and they don’t know whether that person has depression or is suicidal, then they could have pushed someone into very dangerous territory. I think they need to be sued, ASAP, actually.

      • Jarredsgirl says:

        Just wanted to add in one more thing: Manipulating people’s emotions is intrusive because it is so personal. Facebook can gauge responses to their experiment while a person is logged on to facebook, but what happens when that person logs off? Their emotions will impact the rest of their environment as well, and plays a part in relationships and day to life.

    • hunter says:

      Agreed. If they don’t know much about their algorithms in the first place, why not futz with what works?

      As for hiding the profile info, as this was only for a limited period of time and we can assume it has been restored, big deal. Now it’s populated. Take another look.

  2. TX says:

    There is a very simple philosophy that internet users need to understand: If you are not paying for a service, you are the product being sold. Save yourself some grief and accept this reality, and you will never be surprised by stuff like this. Sad but true.

    • Ellie says:

      Only there are a lot of people who do pay for OkCupid. It’s a small fee compared to some of the big names in online dating like Match and eHarmony, but it’s still a monthly fee. Maybe OkCupid only did their experiments on free users, in which case I’d agree with you, but if not, then it’s not ok.

      • Tx says:

        I guess it depends on what was in the fine print. People get really up in arms about what websites do with our info- forgetting that often we agree in the fine print (that nobody reads) that the website can use it. I just wanted to remind everyone that you have to be so careful and maintain a realistic view of internet services. caveat emptor and all that.

      • Wren says:

        Yes, I think that is the key point here. If they jacked with paying customers, that’s not okay, but if they only manipulated the profiles and percentages of the free users I don’t really care. Like facebook, if it’s a FREE service WTF do you expect?? Nothing in life is free and if you’re not paying money for something, you’re certainly providing something else whether you notice it or not.

  3. Kiddo says:

    I don’t think I would trust a computer’s algorithms to determine compatibility, even if they were acting on the up and up, without manipulating results. Those are orchestrated by humans, in the first place, are not infallible, and are based on what, exactly? Someone’s intuitive sense? Or calculating similarities like demographics: proximity, age, income, and so on?

    • GoodNamesAllTaken says:

      I agree. The things that seem to make you compatible may not mean anything at all when you meet, and the thing that creates the “spark” is probably not something you expected or could articulate on a form you filled out. I have dated plenty of men who were perfect for me “on paper,” but nothing came of it in after we went out. And I fell in love with things about my husband that are still hard to express. Decency, for example. Who would answer “no” on a form if asked “are you a decent person?” Lol

    • Azurea says:

      I was on eharmony for a bit. When I was frustrated by the crappy matches they provided, I wrote to them asking what the “29 dimensions of compatibility” were that they were using. Did this mean they were matching me with people who were as much like me as possible?
      That’s not very attractive to me. I want someone who is somewhat different, not EXACTLY like me. In any case, nobody answered me! Dating sites really are a crapshoot. After a while I wrote as little as possible, & was surprised that some interesting, thoughtful men responded.
      What does that tell you? 😀

      • hunter says:

        Yes I find I’m best with my opposite to a fair degree. Not enough to drive me insane but to provide a good balance.

  4. mystified says:

    Boy does this smell creepy. Aren’t they supposed to get permission for this sort of thing, or is it buried in 300 pages of fine print? The ultimate bait and switch.

    Those poor users who were looking for a connection and were instead used as fodder for some sloppy experiment.

    • Kiddo says:

      Isn’t it an experiment to begin with, to try to a select a mate based on one dimensional photos and whatever data that person decides to put up, true or false?

      I still think the best way to meet someone is through other people you know. It’s not a perfect system, but there is more realistic screening.

      • Jarredsgirl says:

        It is not an experiment in the first place, if they act like their service is legitimate. If they brand it as an experiment, then fine… but if not, It requires some critical thinking to see what it really is, and it becomes manipulation.

      • Jarredsgirl says:

        Sorry I just re-read my comment and want to rephrase it.
        What i mean is that people do not have a reasonable expectation of being experimented on, when they are using facebook or OK cupid. These sites are marketed as a social website / dating website, not places where you might be subject to blind experiments by private companies who have no ethical guidelines.

  5. aenflex says:

    It’s a free site. When you use a free service, you are not the customer, but rather you are the product. Now if an expensive dating site did this, I’d want my money back.

    • megs283 says:

      I’d agree with you, except that there are apparently people who pay for OKCupid. (News to me too.)

  6. starrywonder says:

    I think it sucks that they did this since they were treating people like lab rats. OK Cupid is free but some people actually do pay for the extras on the site like meeting up with people the site calls A+ matches and allowing you to actually filter for more things. OK Cupid is like Match.com though, just a site people use to hook up since none of my friends who have used it so far like OK Cupid. Also there are several people on there a friend of mine now realizes are fakes due to the fact when she initially signed up she had a model looking guy as one of her matches, she emailed and favorited him and crickets. She deleted her account after six months. She recently got back on a year later, the same guy popped for her. And once again she contacted and favorited him and crickets. She had another one of our friends sign up just to see if he would pop for her results and the guy did too. So that could have been an experiment that OK Cupid was doing.

    • Badirene says:

      The model type liking your friends was a fake profile, ex-employees have admitted to setting up these profiles to keep people interested and keep them using the site.

      An employee of one of the sites sued cause she got carpel tunnel from typing up the fake profiles for employers. A write up was in the Toronto Star about it. She had to do 1,000 profiles in 3 weeks

      • starrywonder says:

        Yikes. That’s awful. And yeah I guessed it was a fake profile too once she told me her story.

  7. chia says:

    Eh, I’m also a former OKC user and it doesn’t bother me. As said above, you still have to put in the legwork to find out if someone is actually compatible with you.

  8. Snazzy says:

    Not sure about the science of these sites anyway. When I was on the free part of the site, it sucked. Completely useless, and all were the same. They only got interesting and useful when you paid for the service.

    I met Mr Snazzy on one of these kinds of sites, and even as a paying customer in the past on a dating site I can honestly say I often ignored the results and looked more at what the people wrote in their profiles to find similarities or interests vs the % matches alone. In the end, like someone said above, you still have to do the legwork, meet the people and decide for yourself if this is what you want.

  9. Miss M says:

    No wonder it is considered a bad dating site. Many of my friends said it is a hot or miss and most of the time miss

  10. annaloo. says:

    My opinion of this and any other type of social networking online: if you volunteer to give your info to any of them, that’s your choice. No one holds a gun to your head for any of this and I’d naturally expect for any company to analyze data that they collect from users.

    • GoodNamesAllTaken says:

      Good point. I hadn’t thought of that.

    • Ms. Lib says:

      Analyze vs. manipulate — that is the issue.

      • Jarredsgirl says:

        Analysing is observing and making sense of data in it’s original form, manipulating is changing said data to produce a particular effect. They were not simply analysing data! I really can’t believe people are not shocked about this.

  11. hindulovegod says:

    I met my partner on okcupid, and he and I didn’t have a high match percentage. (He was also filtered out of my results, since he smoked at the time.) The “good” matches were often odd, so I think testing to improve them makes sense. The methods used were poor, though.

  12. QQ says:

    Doesnt surprise me in the Least, between those “you are hotter, so we are gonna show you the quality people” emails and Rudder’s Interview/advice in Savage Love a few months back where he was a flat out douchebag to a woman that wasnt getting dates and called her “probably more unnatractive than she thought she was so she should aim lower” and was all manner of condescending… That’s the type to run these games on people, so sure why not?

    (Tho in the end I met my Babyest there? And it wasnt someone I’d normally go for or whatever but it’s been in most aspects the funnest easiest relationship ive ever had?)

  13. Ciria says:

    The Placebo Effect

  14. OTHER RENEE says:

    I think any kind of data manipulation is wrong if people are relying on a service, paid or otherwise. I met Mr. Other Renee on a site that offered no matches, it was up to you to sort through and choose. However, it is a religion based site so the pool is much smaller than say Match or Cupid. However, users could elect to see new members daily. Mr. Other found me that way, my profile popped up for him on the day I joined.

  15. Marianne says:

    I think it might have been better if they had told people ahead of time they would be experimenting. You know, telling people some of the matches would be correct and some of them would “wrong” or whatever, and see what happens. I think what has people upset is that they feel lied to. I mean, maybe “the one” slipped away because the site told them they weren’t a good match.

  16. eliza says:

    More like OkStupid.

  17. boodiba says:

    Personally I don’t think “free service” justifies shady behavior. My friends and I all call it “OKstupid” anyway.

    Internet dating just doesn’t work for me personally, and it only got worse with time and aging. At least when I was in my mid 30s I could find a good fling online. Now that I’m 47 I’ve decided (fortunately) that sex is disgusting and human entanglements on the 99.99% unfortunate side.

    • Mel says:

      “Now that I’m 47 I’ve decided (fortunately) that sex is disgusting”

      Huh? I trust that’s a joke. 😉

  18. LouLou says:

    I used OkCupid. It’s hit or miss and would be difficult even without the manipulation. I always looked when the site suggested someone had a high percentage of a match with me, but I often did not agree, so I stopped paying attention to all those other features. I can’t remember what some of the extra features were in which OkCupid seemed more involved in the matching process, but they all struck me as fake and based on nothing. I did, however, meet my partner on the site. I didn’t even pay attention to whether or not the site thought we were a good match. I just liked the message he sent me, and that overrode everything.

  19. suzz54 says:

    Not a fan of being a subject fit science without my knowledge. Maybe Rudder should stick to making music with his cute little band, Bishop Allen.

  20. astra says:

    Haha OKCupid! This one site I always read has a huge multi-page thread dedicated to the freaks on that site. Those people are seriously scary and it seems to be the absolute bottom of the barrel, perhaps even under the barrel. Sure, consenting adults and all of that, but they give me the shudders and worry excessively about the future of humanity.

    • hunter says:

      Woah woah woah. Maybe it depends on where you live? I’m an OKC user in NYC and I have found my matches from there (once filtered & vetted) to be roughly 90% as advertised and very appropriate and quality men.

      Just haven’t found the right one yet. Lots of “meh” experiences too, but the guys are not crazy or low class or anything. I’ve heard Plenty Of Fish can be full of white trash but I’ve never been on it and again, throughout all of the USA if you think about it, this can be a very regional thing.

      • greenmonster says:

        I agree with you. I live in Germany and met my partner on Okcupid. I can also say, that Okcupid is way better than a lot of other dating websites. My boyfriend was the only one I’ve met with over Okcupid, but I had a couple of nice email-contacts – no freaks, creeps or anything like that. A friend of mine was dating a couple of guys before meeting her boyfriend (all from Okcupid) and neither of us had any weird experiences. Nice, regular and interesting guys.

  21. Chicagogurl says:

    I met my husband via okcupid. We had a 98% match – the highest we had both seen. It was accurate for us. We’re pretty much always on the same wavelength and we analyze and view situations and people eerily similarly a vast majority of the time. Additionally, we have a very biting/sarcastic sense if humor and spend most of our time together cracking each other up and we bring out the best in one another. Regardless of the specifics, in my marriage, okcupid absolutely was right. Algorithms are typically hypothesis at best but I couldn’t have found a better match myself. I tried a few dating sites prior – match, eharmony, meetup, zoosk and hated most of the “matches” but I found 2 friends and a husband on okcupid so they’re doing something right.

  22. Jolene says:

    As a regular user, a lot of so-called 90% matches were awful. From my experience, enemy% is much more accurate on okcupid. I think their experiments are wrong. I wonder if the absolutely awful man I met recently came into my matches because of the experiments.

  23. Jade says:

    Actually, even if you paid or did not pay for an online service, or whether you like it or not, it is most likely that this type of data experimentations are covered in the lengthy terms of agreement if you clicked OK for one. It may be data manipulation but they or their lawyers can also twist it easily as data research. I understand though that it’s disappointing to expect a scientific match but being given otherwise. Some people just do not want to waste that kind of time and get to know non-matches better.

    Regardless, I’ve come to accept that no online activity of mine is sacred, but I’d still use the Internet.