Better email marketing applying the scientific method

Email marketing should be every marketer’s dream. As well as requiring creative thought and strategic planning, you instantly know whether it is working or not. Every aspect of it is trackable, testable and reportable.

Rather than approaching making changes an ad-hoc or subjective basis, you can improve your email marketing performance consistently and iteratively by using the scientific method.

[su_note note_color=”#7fff66″ radius=”5″]What is the scientific method?
The scientific method is a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge (source).[/su_note]

The steps of the scientific method

scientific method

Taken from

For the purpose of this article, I will use the example an online portfolio creator for designers, photographers etc. to showcase their work, that offers a 14-day free trial. I will name the company ‘Pyxels’ (note: this is a totally made up company to illustrate my points).

Purpose: State the problem

Before you start making changes to your email marketing, take a step back and think about the end goal i.e. the problem you want to solve.

Email marketing is a means to an end, serving your business goals. It is not a goal in its own right.

For example, even if you want to to increase click-through rates, those clicks serve the purpose of increasing traffic to your website via email.

Your metrics should be serving a bigger purpose e.g. We want our email marketing to…

  • Reduce customer support phone calls
  • Increase basket size
  • increase referrals
  • Etc.

After reading this SaaS conversions benchmark study, Pyxel are unhappy with the number of customers converting from the 14-day free trial to a paid account. This currently stands at 2%.

Research: Find out about the topic

To make any changes, we need to work out what differentiates successful customers from ‘unsuccessful’ customers, and how our email marketing can help solve the problem.  

Try to use data to identify the characteristics of customers rather than anecdotal evidence. We want to know who…

  1. Converts into paying customers
  2. Spends a lot
  3. Buys frequently

Once you know who they are, you can start building mechanisms for new customers to perform those actions quickly and easily.

For example…

Free trial sign-ups

We have identified an area that splits active users and inactive users.

The next step is to review how we are currently addressing this problem (if at all), and  the best way to achieve that outcome.

Based on these figures above, Pyxels need to get more of their trial sign-ups to customise their default portfolio theme.

The logical place to start is how Pyxels are currently communicating the customisable portfolio feature to new sign-ups.

Here is their free account sign-up welcome email:

[su_box title=”Welcome To Pyxel”]

Hi [Name],

Thanks for signing up to Pyxels, it is great to have you on board.

Pyxel makes it easy for you to showcase your amazing work and attract new clients, so let’s get started.

[Button] Login to your account [/Button links to account home page]

Kind regards,

The Pyxel Team


The email is short, friendly and comes with a very clear call to action to login.

However, based on our data we now know that it is not directing people to perform the action we want them to do.

With this in mind, we want to try a new welcome email that achieves that goal.

[su_box title=”Welcome To Pyxel”]

Hi [Name],

Thanks for signing up to Pyxels, it is great to have you on board.

Pyxel makes it easy for you to showcase your amazing work and attract new clients, so let’s get started.

Getting started

Your first step to success is making your portfolio your own. Add your own unique style quickly and easily with our editor.

[Button] Customise your portfolio [/Button links to editor]

Kind regards,

The Pyxel Team


Hypothesis: Predict the outcome to the problem

What change are we expecting based on this change? By building a hypothesis before you start you can judge whether the change has been a success or not.

Additionally, although my example focuses on one change, it is more normal to have multiple areas/problem you want to improve upon.

Your hypothesis should also include the uplift you expect from your change, based on on quantifiable numbers such as

  • Revenue
  • Basket size
  • NPS
  • Support queries
  • Number of subscribers
  • Open rate
  • Click through rate
  • Social sharing
  • Etc.

This then allows you to prioritise your resources to focus on what you expect to have the biggest returns.

Our hypothesis is…

“Our new welcome email will make it easier for new customers to customise their portfolio, increasing our conversion rate from 2% to 3%”.

Experiment: Develop a procedure to test the hypothesis

The simplest way to test anything is run an A/B test and email marketing is perfect for this.

All we have to do is send 50% of new sign-ups the old version (the control group) and 50% the new version (the test group).

We can then see if there is an uplift in our key metric of free to paid conversions amongst the test group.

Analysis: Record the results of the experiment

This is the easiest part of the process. Your email marketing software will do all this for you. Tools such as MailChimp have this built in, and are very easy to set up.


Conclusion: Compare the hypothesis to the experiment’s conclusion

Now is the moment of truth. Has our new test version performed better than the control version?

For Pyxels, we identified a problem (low conversion rate), we stated what we wanted to achieve (free to paid conversions of 3%) and we researched the best way to do this (direct new users to customise their portfolio).

We can easily directly compare the data for both versions of emails.



Emails sent



Paid conversions



% conversion



These stats are all illustrative, and show a positive uplift. However, even if the change you makes has a negative impact, it is still a test worth running, because know you now.

Additionally, you only exposed a test sample to this version which means you can now roll back, and test a new idea/version.

What next?

We move on to the next test!

Assuming we have a list of goals we want to achieve, along with supporting hypothesise to test, we can now begin to systematically improve our email marketing.

By using the scientific method and applying values to each hypothesis to create your list of priorities, you will quickly see improvements you can measure and build upon.

Leave a Reply

Your email address will not be published. Required fields are marked *