Interpreting Beauty Product Studies & Their Results

While I'm not a professional researcher, the topic of studies has interested me since I was taught to discuss and dissect them for Psychology in college. Obviously there's a difference between behavioural studies and those created to test skincare, haircare or make up products, but the basic questions we need to ask in order to determine just how reliable and trustworthy a study is remain the same. I'll be the first to admit that I'm drawn in by amazing trial results just as much as good reviews or recommendations for a product, but are these results always creditworthy? And should we be looking deeper than just percentages on a page?

The first thing to think about when presented with a study for a cosmetic item is- why was the study commissioned? In the commercial world where the results are readily posted onto publicity campaigns the answer will almost always be to sell more products, and this is why we need to think about whether the study is scientifically reliable or just part of the advertising.

Another thing to think about is who conducted the study? Was it the brand themselves using an in-house team (least reliable), a third party paid by the brand (more reliable) or an unrelated group conducting a study for example to compare similar products (most reliable). It is almost a guarantee that for the first two the results will be completely positive, and that moves us on to thinking about how it was conducted.

One of the most important things when looking at results is how many participants were included? The main place we see high positive percentages are in mascara adverts, like the one above. The fact that 94% saw visible lift is an amazing statistic, but follow the asterisk and we find out that these results are part of a 'consumer panel survey'- which could have only included 5 people. Even when trials have what seems like a large amount of participants- say 300, is this a good representation of the ~54 million people that live in England alone? Would the participants include a good mix of ages, skin types, hair types etc and does it indeed actually include the people to whom the product is targeted (eg mature skin for an antiwrinkle product)?

Next we move on to the study itself- how long was the study, how was it conducted and results collected? This can be tricky because unless you can find the full published study there will only be a limited amount of details available, but there are important points to take from the information we can find.
Obviously a trial of a longer duration is going to give more reliable results- I'm sure there have been many examples of when you tried a product and loved it on the first go, but as time went on you realised you didn't get on with it as well as you thought.

How the study was conducted can also make a big difference to the results- it could've been a survey that people received on a postcard to fill in and send back to be entered into a prize draw, maybe a scientific trial carried out by researchers, or even experiments on cell samples under a microscope. 'In Vivo' clinical trials performed on a whole living organism such as a person will generally give more helpful results than those studies performed on samples in an artificial environment for example 'Ex Vivo' or 'In Vitro' performed in a test tube or petri dish. For extra In Vivo study credibility points, look for phrases like 'placebo-controlled' (meaning the product was tested against another product that was meant to provide no real effects), 'blind/single blind' (meaning the participants didn't know what the product was or what it was claiming to do) and the gold standard 'double blind' (meaning neither the participants nor the researchers knew who was using a placebo and who was using the studied product). 
In terms of how the results were collected, physical measurements will always be more accurate and credible than visual observations or self assessment: someone says your hair looks shiny today, but does that mean its actually healthier? There is a wide range of scientific equipment that can measure everything from how hydrated your skin is to how much breakage your hair has undergone and can determine whether important results are just cosmetic.

Finally, we need to consider if there are any factors that might have added to the results, such as if the item was used in a particular way or other products were used at the same time. For this we need to follow the asterisks, small arrows or little numbers next to the big results to find the small print. A popular French brand claims one of it's products provides results equal to prescription skincare, on studying the fine print we find out that this claim is only true when using the product plus a 0.025% topical retinoid, something not mentioned any where else. The small print is where you can find extra information, or disclaimers, for trial results appearing in advertising and they can often help you in deciding whether the study is reliable or just a ruse.

We need not though be too cynical when it comes to interpreting results, a lot of studies do highlight just how great a product really is and I think actually what's more worrying than a sneaky study is a company who doesn't give any source at all for their claims or statistics. 

Do you go on studies or recommendations for your purchases?