Hot or Not, Part Deux

Today’s guest post is by Simon Galbraith. Simon is co-founder and joint CEO of Red Gate Software.

I’ve been a long-time believer that, correctly chosen, professional photography is a key element in marketing. I’ve backed this belief with the money of my company and have approved countless campaigns that involve professionally taken photographs.

A couple of weeks ago I blogged about an experiment where four of us (including two world experts in marketing) guessed how two photos of me posted under aliases on would be ranked. I also asked readers of this blog to comment on how they thought the rankings would turn out.  As expected almost everyone thought the professionally taken photo would score higher.  The only differences were the actual numbers.
Here are the hotornot results based on the professionally taken photo of “Jeremy” (left), and the point-and-click amateur photograph of “Aden” (right). Click on the images to view larger versions.


Surprise! The professionally taken photograph scored substantially lower (8.1, 79%) than the amateur photo (8.7, 86%), both in absolute terms and in the percentage ranking of attractiveness.

I’ve a quick confession to make: had the professional photograph scored higher than the amateur photograph, I would have immediately stopped thinking about this issue and have considered my case totally proven. But when the results belied a premise I’ve supported with huge sums of money, I was compelled to find a way to prove the experiment wrong; to start looking at the sample size, the details of the photo and so on.

Hotornot is a compelling site and it has a whole heap of great features enabling people to show increasing levels of interest that might ultimately lead to a relationship. I thought I might be able to use some of the more anecdotal data on hotornot to demonstrate that although the score for professionally photographed Jeremy was lower, it actually was better by some other measure.

Although less statistically rigorous, the results from my further study were the same. Aden has had one woman approach him to meet, whereas Jeremy hasn’t had that honor. Aden has been “favorited” twice by women, compared to Jeremy’s one solitary bit of interest. In every way I could think of to analyze it, Jeremy did worse than Aden.

Looking at the outliers in the histogram is interesting. Aden is more polarizing – more women ranked him at the extreme ends of the scale.  This might be valuable from a marketing point of view – perhaps people who are prone to polarization are more likely to take notice and act (another theory I’m now propounding that I need to prove).

I asked the opinion of some experts who work for the best marketers in the world, and their opinion was worthless compared to an experiment that took less than an hour to set up and cost me nothing. Given the eye-popping sums I’ve spent so far on marketing with professional photographs, it seems crazy that I haven’t tested this before. I’m still not sure that I want to use amateurish photos in our marketing, but now I have to admit that my past decisions have been based on blind prejudice rather than wise insight. It leads me to wonder: how many other of my opinions are as fact-free as my views on professionally taken photographs?

On a personal note, I’m amazed and flattered that I scored so much better than when I was a shy 16-year-old, but I suspect that might come down to the rather unusual way that hotornot averages the numbers in the histograms rather than a change in my attractiveness.  But, in this case, it might not be worth analyzing too closely.

10 responses to “Hot or Not, Part Deux”

  1. Interesting. Have you tried swapping the names over?
    Aden feels different to Jeremy to me.

  2. One of my last tasks at the last job was to sort out a directory page for members of staff. The Head of Department was keen to ensure that this included photographs. Being a cash-strapped academic department, we couldn’t afford a pro photographer. As I just about know one end of an SLR from the other, the plan was for me to take the photos too – but I ran out of time and ended up just doing the shots of Professors Haynes and Hawking. All of the rest were user-submitted – a mishmash of everything from high-end pro work to mobile phone snapshots. This was the result:
    This conveys something very different from a page of consistently-lit, consistently-processed images, but I’m surprised at how well it works. Perhaps it’s because variation implies individuality?
    One of my favourite corporate mugshot web pages is here:
    The photographer is a friend of mine – a serious amateur like me, but it’s not his day job. The best bit? Mouse-over the photographs.

  3. Michael says:

    The reason is simple – as “Jeremy” you look like you’re worried about having your photo taken; as “Aden” you’re standing with a confident, dominant posture – guess which one women prefer.
    Which one people would like to do business with is a different question.

  4. Michael says:

    BTW, posting comments doesn’t seem to work in FireFox, I’m posting this via IE

  5. Hot or Not works by showing the photo, the name probably isn’t something that is prominent enough to have a significant impact. I’d suggest checking it out, first warning your partner(s?) that you are doing important research…
    There are lots of variables between the two photos (including “mood”) but the biggest by far is the quality of the photo. None of the thousands of readers of this blog nor any of the people I asked individually predicted the actual outcome before it was shown by experiment.
    We can try to justify it afterwards but that is just being wise after the event.
    We need to face up to the truth of the result of this experiment – the quality of the image is unimportant (which is the bit that costs the money) compared to other more subtle factors in how we respond to the image of a person.

  6. Benjamin says:

    But doesn’t it seem that on HotOrNot, you are primarily asking women if they find you attractive as a prospective romantic partner? In contrast, in corporate marketting aren’t you asking both men and women if whether the photo inspires trust and confidence in your qualities as a potential business partner? That seems to be a substantially different question. While the answers might be related (perhaps a woman who finds a man personally attractive *might* be more likely to want to do business with him) I’m not sure I see that it conclusively rebuts your original theory. It seems like you would need a study that asks a question more relevant to business marketting and the frame of mind people are in when being confronted with business marketting material versus the frame of mind when browsing a site like HotOrNot.

  7. Benjamin,
    To really get to the bottom of using a photo in an ad, we should do some split A B testing on the same advert with different images. It is pretty easy to do this using the Google image advert service so I’ll definitely give it a go one of these days.
    I agree with your more subtle interpretation of the question we should be asking about the photo viewing audience and what their response really means. However I can assure you that I’d never asked the question before when commissioning a photo campaign and I suspect few others had too.
    My guess is that the reason you are coming up with this in depth reasoning is that your mind is repelled by the results of the experiment. If I’d told you that the results were the other way around you would have nodded and moved on.

  8. Simon Sabin says:

    I read your first article and wasn’t sure which would win. However on looking at the photos again I realise you are smiling and look relaxed as Aden but look unformfortable with a forced smile/more serious look as Jeremy.
    It would be interesting to distinguish between a smiling photo and a non smiling one.
    People always like things that make them happy.

  9. Jason Cohen says:

    Besides the other problems with the test that others have pointed out, I’ve run the stats and IMHO the test isn’t statistically significant.
    In particular: The null hypothesis is that “the pictures are equally handsome,” and you’re trying to determine whether the data are extreme enough to reject this hypothesis.
    The correct test is Student’s t, Two-tailed ('s_t-test).
    We have the mean and number of samples from the pictures; I had to guess at the variance by recreating the charts you give in Excel.
    My result was that the samples have a variance of 5.82, and running the appropriate test gives a t value of 1.52, which doesn’t hit the 95% confidence interval on the distribution.
    So…. I think it would be fun to change things up as suggested in other comment and try again!
    Anyway it’s a great experiment and lots of fun. Thanks for posting it.

  10. joe says:

    One comment on this…
    People can sniff out a professional level photo. On a rating site like HotOrNot, an amateur candid probably will score better since people see that person as real, vs a “model” with ideal lighting, make-up, etc. Have you seen some models or actors in real life? They’re not as “hot” in person. In addition to the above, the photos are not similar enough, one is a head shot, the other is 1/2 body. I’d like to see results of another “model” experiment with photos that are both 1/2 body.