CATEGORY: Deliverabilityemail-marketing

Rise of the machines - A definitive guide to bots

We are under threat from robots and artificially intelligent programs designed to stop us. Yes, like some dystopian sci-fi thriller, the world of email marketing is under attack!In a recent analysis of 1 million emails sent from our platform, we identified over 256,000 interactions from 17,613 suspected bots. What the analysis also revealed was large variance in the amount of bot activity experienced by each sender. In some extreme cases, over 90% of the tracked activity on a campaign was suspected bots. There are patterns that we can identify as to why some senders experience more bots than others, but let’s start with a bit more context.

What is a bot? 

A lot of internet activity is done by automated (and sometimes artificially intelligent) programmes carrying out a variety of tasks. Distil Networks publish an annual report on bot activity and the most recent of these reports claim that just under 38% of all web traffic is non-human. 

Bot activity varies greatly from legitimate activities such as indexing websites for search through to malicious ones used to carry out cyber-attacks. In between those two extremes, there is a wide gamut of bot activity that can distort website analytics and lead to other technical issues. It is in this grey middle ground that we find a number of bots that affect email marketing. Here we explain the usual bot suspects in email marketing and what you should be doing to deal with each type: 

Email security bots 

These bots are actually the good guys. In the same way that fraudulent bot activity is very prevalent, so is fraudulent email activity and scams involving phishing and impersonation happen continually in extremely large volumes. To combat this, many businesses have implemented cyber security technologies to minimise the risk of users falling victim to these attempts to glean information and access.The headache that this creates for email marketers is that protection bots will intercept emails before they reach the intended human recipient. The bots will click on links in the emails and visit the destination URLs to look for potential security threats. This does not happen universally and in our analysis, we see a correlation between certain sender traits and increased bot activity. These are: 

Complaints 

Most email clients contain a mark as spam function – if this happens in significant quantities then your emails are identified as suspicious and start getting intercepted.

What to do about it – ensure that you send emails that are expected and desired by the recipients and even then, make unsubscribe options clear. e-shot™ also places a compliant unsubscribe link in the email header so that if recipients use the native email client unsubscribe function, it will automatically get processed by e-shot™.

Enterprise contacts 

Some companies have very robust cybersecurity routines that treat everything with a degree of suspicion and send the lot via a bot – this will typically be true for larger organisations in industries such as finance. 

What to do about it – accept that your campaigns will be subject to bot activity and use measures such as Lead Score to identify true engagement. 

Content

If your emails contain links to known sites that are associated with cybersecurity threats or spam. 

What to do about it – Check campaigns thoroughly using email forensics tools and do not use shortening services that may also be used by spammers or hackers. 

Identity verification failure 

Most email fraud relies on recipients being duped into interacting with emails that falsely appear to be from a known source. This is why ISPs and spam filters look for a number of forms of verification in your email marketing. We explain each of those in more detail here. 

What to do about it – To make things simple for our clients, we set up, maintain and check these verification methods continuously as part of our sending domain management services. If you use your own domains via e-shot^TM, then our email forensics will flag any issues. If you are not a customer, but have concerns about this, our free Healthcheck may be of use.

The impact of email security bots 

The main impact of email security bots is inaccurate reporting. Your campaigns show inflated click behaviour and your website analytics show similar spikes of activity. It can also lead to problems with marketing automations that use clicks as a trigger for subsequent communications. 

What to do about it – Review results and take action if you suspect that your campaign has been targeted by bots. e-shot™ automatically identifies suspected email protection bots by using… bots! Yes, bots to detect bots. Suspected bot activity within email campaigns is then flagged in our reports. Many clients would prefer that these activities are prevented entirely, but it is a constantly evolving game of Catbot and Mousebot. Some email protection bots can be associated with IP addresses that we know belong to cybersecurity and spam filtration companies, but others will cycle through large pools of IP addresses where human and bot activity is mixed. The other common way to spot bots is click behaviours – some bots will click every link in an email quicker than a human would ever be capable of doing. We do see human clicks following bot clicks though – as intended, the bot is checking the email is safe and then subsequently handing to its human recipient.For these reasons, we identify and flag potential email security bots, but we do not suppress them.

Form spam bots 

If you have a form on a web page, it is a foregone conclusion that a bot will try to submit data to that form. Your form technology and processes need to protect you as well as welcome new subscribers. One client that we assisted recently had over 7,000 fraudulent submissions to a third-party website form and then carried out email marketing activity to those 7,000 dodgy email addresses. Not only did the emails not go through to the 7,000, but it caused wider deliverability issues with the client’s emails to their regular subscribers. We were able to solve this in a matter of days, but prevention is better than cure. 

What to do about it - To prevent form spam polluting your database, there are various technologies that you can use to protect your forms and your database:

CAPTCHA 

Many forms contain functions that bots cannot fulfil, however there are now numerous examples of the bot technology beating the CAPTCHA technology. Google are continually updating their reCAPTCHA to stay ahead of the bots and the latest iteration of this uses a scoring system to identify unusual website behaviour.

Double opt-in 

Verify form submissions by sending back emails to the submitted addresses asking for verification via another link click in the email. Treat non-verified addresses as potential hazards. This is also best practice and standard functionality in e-shot™. 

Data scraping bots 

There remain a number of questionable data sources available to companies for marketing purposes. The origins of data can include information that has automatically been harvested from websites and other online sources. Not only can this information contain large amounts of erroneous data, but it can also (and most probably will) contain spam trap addresses designed to get you into enormous trouble. Always ensure that data is provided by reliable and reputable sources.In B2C email marketing, you have no choice in the matter – all of your data should be opted-in and properly verified. In B2B email marketing, if you are still relying on third party data sources, you need to ensure that they are reputable and safe. This is not about legitimate interest or any legal principle - although these are equally important. From a technical perspective, hitting bad or out of date lists is highly likely to land you in trouble with spam filters and subject to bot activity. 

What to do about it - If potentially bad addresses have got through your forms or been imported from other sources, then it is important to detect and remove them. e-shot^TM maintains a global suppression list of known bad addresses. This is continuously updated and will stop emails being sent to bad addresses before any damage can be done. We can also validate data with third parties who will identify suspicious or bad addresses. This is useful for those who have not previously had a robust audit trail for data acquisition and verification. 

In the more general sense, it is prudent to regularly review campaign reports and take further action to remove suspicious contacts. Comprehensive bounce reporting can be used to identify which addresses in your database are causing problems.

Conclusion 

There is a simple and uncomfortable truth to why some email marketers are targets for bot activity – data quality. Cyber security is a very real concern for everyone and email marketing by its very nature (sending large quantities of email simultaneously) is always going to attract the attention of security countermeasures. 

Bot activity is increasing, globally and the sophistication of these bots is also increasing with some now mimicking human behaviour. Legitimate email senders with good data health processes have less to fear but may still suffer the annoyance of email security bots. There is sophisticated technology on both sides of this battle and e-shot™ continues to develop bot detection methodologies and Deliverability tools in the more general sense. You may find our recent webinar on deliverability to be of additional help.