Thanks to the community for their input over the last few days, and over the years. Hands-on experiences with fraud vendors from real practitioners were key to this article. I’ve been asked to write an article like this for many years. Now is the time, given the virus crisis and current economic pressures. Advertisers are re-evaluating their digital ad spend, programmatic supply chain costs, and agency partners and vendors like fraud detection tech companies. They are looking more closely and need insights and context around which vendor to choose for which task, and which vendor is good at what.

Who Am I?

I’m an outsider. I don’t work with any ad tech companies in any paid capacity. I do not participate in any ad tech or industry trade association committees or working groups. I’m a digital marketer of 25 years, with both client side and agency side experience. I am a scientist and a data person who looks at analytics continuously, because I enjoy it and it fascinates me, especially the “crime scene investigation” aspect of it. I do not buy or sell digital ads (manage media) for anyone else; but I have run my own digital campaigns for more than 20 years. I run experimental programmatic campaigns with my own money to gather data for ad fraud research — I’ve purchased 1 cent CPM inventory continuously for the last 3 years, so I know what the bottom of the barrel looks like.

I audit marketers’ campaigns with my fraud analytics technology to show them what their current fraud vendors caught and what they missed. I built the platform and have been refining and updating the detection algorithms over the last eight years. My technology is not MRC-accredited and will never seek to be MRC-accredited, because any user can review the supporting data themselves and see why something is marked as fraud or not fraud — i.e. it is not “black box” tech that someone else has to verify or “accredit.” I am not paid by any fraud verification companies and have never “looked under the hood” of their technologies.

The #FouAnalytics chart below compares tightly managed programmatic campaigns to ones focused on getting as much reach and frequency as possible. In both cases, fraud detection tech was used. Red means bots, and blue means humans. As you can see fraud still gets through despite fraud detection technologies. The quality of the campaigns depend on how closely the campaigns are managed, of course using analytics to keep an eye on the data.

Where Am I Coming From?

I am independent and I have fraud analytics technology. My tech has been used to audit campaigns side-by-side against the fraud detection tech already in use by marketers. So I have a good sense of what they can catch, what they miss, and what they don’t look for. It is from this vantage point that I am discussing the fraud verification companies below and providing context around their specialties.

I will not go into detailed examples of incorrect measurement in this article. I’ve already previously written about the capabilities and limits of fraud detection tech:

1. Why Fraud Detection Doesn’t Work

2. Documented Examples of Failures of Fraud Detection Tech

Some of the companies below are widely known and used while others are lesser known but have great expertise and do excellent work. The intent is to arm advertisers with more insight when selecting a fraud verification vendor, or deciding not to pay for one any more, particularly in scenarios where buyers are reviewing their costs and upcoming renewals, or taking parts of programmatic in-house for more control and detailed analytics.

This is not a numerical rank order of fraud detection companies, because 1) I don’t have 100% of their data to review, 2) I have never looked under the hood at their tech, 3) most of them are “black box” and don’t provide any means of verifying the accuracy of their measurement, and 4) a numeric rank like 40 might suggest a vendor is 2X better than another ranked 20, which would not be accurate. What follows is a combination of descriptions, context and recommendations for where to use various vendors most optimally. It is based on audits, experiments, and observations from over the years; a key aspect of this is the willingness and ability of the vendor to explain the details behind specific measurement outcomes and input from other practitioners who have used these fraud detection vendors.

A Word on Accreditations and Certifications

MRC (Media Ratings Council) accreditations are useful. It means an informed third party (a PwC accountant) rigorously interviewed the fraud vendor to confirm they are measuring what they said they would measure (can check off every row of the spreadsheet listing the detections the vendor plans to do). But note that this is different than measuring ad fraud correctly. The MRC does not have tech of their own, has no experience measuring real campaigns, and has no “truth set” (data used to verify if bots are detected correctly or not). “MRC accredited” does not mean the vendor detected the fraud correctly or completely; there may be a lot of fraud it didn’t even know to look for. MRC accreditation only means the vendor is measuring what they said they would measure.

TAG (Trustworthy Accountability Group) Certified Against Fraud is mostly a self-attested certification that vendors pay for. Self-attested means the vendors commit to doing certain things (e.g. designate a “TAG Compliance Officer” etc.) and make representations that they meet a certain minimum threshold (e.g. use the IAB Bots and Spiders list in their detection). Further the vendors marked “Independent Validation” means yet another third party reviewed the paperwork that was submitted — in this case BPA Worldwide, historically an auditor of publishers with no fraud detection tech of their own, no campaign analysis experience, and no truth set. “TAG Certified” does not mean the vendor detects fraud correctly or completely.

Related: TAG, you’re s*!t: Internet advertising industry bods admit self-policing approach is a sham

FRAUD DETECTION AND VERIFICATION COMPANIES

Note that this article is focused on companies that sell fraud detection technologies and services as their primary business. The article is not focused on companies where fraud detection is not their primary business, but may be a feature in their offering. I will also give a shout out to companies that have experienced fraud hunting practitioners on staff who are proactively protecting customers from exposure to fraud – Camelot Strategic Marketing and Media, The Trade Desk, Rubicon, and SpotX. There will always be fraud. Fraudsters will always find ways to trick the detection. That’s why it is critical to have real humans with real expertise keeping a close eye on new forms of fraud, not yet detected and documented.

Auditors

Media Optimise, Method Media Intelligence – The core specialty of these companies is that they have have performed numerous audits of digital campaigns over the years, with a focus on reconciliation between different data sets (e.g. DSP vs ad server, payment flows, etc.) Their expertise outweighs tech-based detection.

Malware-Based Fraud Specialist

White Ops is a specialist in malware-based ad fraud. This is ad fraud done by malware installed on real devices. This is the most advanced form of bots that can mimic human behaviors or record and replay human interactive events to circumvent other fraud detection tech. White Ops also properly discloses the limitations of fraud measurement (applies to all other vendors). For example, in the 2019 Bot Baseline Report, on page 21, they included a table which showed that certain types of digital ads like video were only 30% “validatable at the highest level.” That means “impressions could not support a dynamic fraud detection tag via JavaScript. Those impressions are vulnerable to the most sophisticated fraud operations.”

White Ops is especially useful in industry verticals that pay exceptionally high CPMs and CPCs, like pharma and financial services, and are therefore the targets of the most advanced types of bots (made from malware on real devices).

Affiliate and Lead Gen Fraud

mFilterIt, Oxford Biochron – The technology leads and analytics practitioners at these companies have a lot of hands-on experience finding well-disguised fraud, targeting affiliate and lead generation advertising campaigns. They focus on the activity after the click leading up to the purchase event; they show you the supporting data and explain why something is fraudulent so you can understand as well. 

Invalid Traffic Detection

TrafficGuard, CHEQ, Excelaca, Imperva – Website scrapers steal valuable content from sites (pricing, listings, photos, etc.) or commit fraud by purchasing tickets in bulk for the purpose of reselling them at much higher prices later (scalping, now subject to the B.O.T.S Act of 2016). Companies like Imperva specialize in detecting bot traffic coming to sites for the above, and also for DDoS (distributed denial of service).

The other companies in this group have sound technologies. They detect a substantial amount of fraudulent and harmful bot traffic. They show proof-of-work. The combination of their tech and the expertise of their analysts is key. Practitioners can corroborate their effectiveness. And clients are well served by these companies.

CTV Fraud Specialist

App Science – App Science has the tools to collect data on CTV ad impressions and the in-house expertise to do the analysis required for finding CTV fraud. The most amateur CTV fraud has been caught by observing misspellings in the faked device names, super old versions of devices, abnormal numbers of devices per household (IP address), teleporting devices (devices that appear to jump long distances geographically), etc. But CTV fraud that is slightly more advanced will get by technology detection that depends on IP addresses and declared variables. Any variable that can be declared can be faked. And fraudsters fake everything so they can remain hidden. That is why expertise in analysis is critical to real fraud detection and prevention in CTV; again not black box detection tech.

Geolocation Fraud Specialists

GeoGuard, Location Sciences – These companies are specialists in location fraud. Specifically gambling apps are subject to state and local laws. The accuracy of geolocation detection and the detection of falsified locations is important for enforcement. Similarly, video streaming (like YouTubeTV) requires enforcement of state and market boundaries.

Mobile App Install Fraud

Kochava, Secure-D, Scalarr, Machine, Interceptd, Appsflyer, Adjust – These companies’ technologies and SDKs are used by advertisers to monitor their mobile app install campaigns, for install fraud. Note that this is a different specialty than measuring the ads themselves or the quality of traffic to websites. Their technologies analyze the install itself, the devices that installed the app, and post-install events to determine tell-tale signs of fraud.  I have reviewed their work and their resident experts provide value to their clients.

Malvertising Specialists

Confiant, clean.io, The Media Trust, GeoEdge – These companies’ technologies are focused on detecting and stopping malicious code inserted into programmatic ads by bad actors. These ads get served into mainstream websites (at no fault of the site owners) and may be disguised with a legitimate advertiser’s ad creative (at no fault of the advertiser). Hackers steal any advertiser’s ad creative and alter it with malicious javascript that can break out of iframes and take over the page or redirect the user to malicious sweepstakes or tech support sites, intent on tricking the user to click and approve the installation of malware on their own devices. 

Viewability and Traffic Counting

Moat, comScore – Moat is a viewability specialist that has since added IVT detection capabilities. comScore counts traffic to websites and purchased a fraud detection tech company called MdotLabs in 2014 to add fraud detection capabilities. 

General Purpose IVT Detection

DoubleVerify, Integral Ad Science – These companies are private-equity owned and their detection is “black box” so there is no way to verify whether the detection is correct or not. You only get excel spreadsheets with IVT numbers, and often the only recourse for the marketer is to try to get refunds for campaigns that have already ended months ago. It’s like selling spreadsheets to marketers, who pay for black box fraud detection but never ask if the detection works, is correct, or how things were measured. Their pre-bid blocking/filtering does not work as well as they claim, because bid requests contain fields of data that can be all faked (and bad guys always fake such information). These companies are highly skilled at writing press releases, some misleading, and getting press coverage for themselves.

And the “404bot” was not a bot at all; it was the most basic form of domain spoofing where non-existent webpages (”404 error” in server speak) made fraudulent bid requests for ads. These were let through by pre-bid filtering (because of the faked data fields); these were caught when analysts tried to load the non-existent webpages and got 404 errors.

Fake Fraud Detection Companies (4) 

Their names have been obscured. I have said over the years that if the fraud detection tech companies’ primary skill is in writing press releases and tricking reporters into covering them, their skill and focus is not in detecting fraud. Over the years, I have met so called fraud detection companies where 1) the founder didn’t understand ad fraud and the parties that were responsible for committing it, 2) companies with good data scientists who saw fraud in the data but didn’t understand how the tech worked; so they thought there as a bot, when there was no bot at all, 3) one that runs an extortion racket with lists ranking sellers and exchanges. And comically there’s even a 4th fraud detection company in this group founded by a fraudster to make money on “fraud detection” too, because why not? None of the customers will ask for any details anyway, or know the difference between correct measurement vs not. Beautiful-looking dashboards is what they are paying for and the peace-of-mind that they are paying for fraud detection (note, I didn’t say actual detection of fraud).

Where Do We Go From Here?

When it comes to ad fraud, use some common sense and look into the analytics more closely yourself. You can find a lot of the fraud yourself, because it’s still getting through, despite the use of black box detection technologies. When you spot something strange, dig deeper and find the reason behind the strangeness – like these 2 small businesses did themselves. Once you find the fraud and where it is coming from, it is straightforward to turn off those domains and apps from your campaigns, while the campaign is still running. This is much better than waiting till after the end of the campaign to get an IVT report spreadsheet from vendors, because your campaign can run more cleanly and more ads can be shown to humans.

Just like I’ve said about ad fraud itself, also use some common sense when assessing fraud detection vendors. You can easily tell if their sales people and account people actually know anything about ad fraud or the tech they are selling if they consistently they hide behind “trust us, our number’s right, because we’re MRC accredited or TAG certified.” (see MRC/TAG section above). If the vendor is “black box” and you have no way to tell if the measurements are correct or not, how do you know if the 1.43% IVT they tell you is right or not? Are they detecting everything? Do they even know to look?

Also, don’t blindly believe the hype around AI and ML (artificial intelligence and machine learning). The AI/ML is only as good as the data you feed it. If the data scientists don’t understand bots and don’t understand ad tech, their training of these algorithms is also suspect. There are countless examples of false positives and false negatives coming out of this more black, black box. If there are experienced analysts who understood the data science as well as programmatic ad tech, that’s how fraud can be detected, not just the AI/ML alone. Experienced human analysts still have to check the answers and those answers should still make sense to you.

Still not sure what to do? Run some experiments. You can use my fraud analytics platform at no cost (www.fouanalytics.com) to audit campaigns where other detection tech vendors area already in use. Look for whether they can explain what you see in the data, so you can understand why something is marked fraudulent or not fraudulent. Assess if the mitigation is easy (turn off domains and apps) or if they require you to buy even more services from them (e.g. pre-bid filtering) in order to mitigate the fraud. And ask me for help, if you need it (just google my name).

FURTHER INPUT FROM COMMUNITY

Ian Mc Grath, Marketing & Media Consultant, provided the following written statement:

“It’s been a tough year for most advertisers and agencies. Tackling leakage in the digital supply chain is an easy way to improve the effectiveness of their advertising and increasing its revenue return. With particular reference to adfraud, this also means stopping marketing spend from seeping from trusted environment to criminals. Some areas to note:

• Many clients and agencies have been using detection companies as a tick-the-box, pushing the responsibility to mitigate against adfraud from themselves back into the supply-chain. This is somewhat understandable, given the proliferation of martech needed to just deliver an online campaign in today’s market. While not fully absolving themselves of accountability, clients and agencies have outsourced this, trusting it to a specialist. However, control is difficult because as campaigns increasingly target lower CPMs to drive ROI / ROAS, buying slips into more murky environments with a higher propensity for fraud.
• Most agencies do not want to rock-the-boat, as programmatic has represented an important revenue stream among other tight margins. In many cases, detection companies are also directly appointed by the client and the agency simply works with them as a third party.
• Detection companies have inflated their importance with very little proof of their effectiveness. Ad fraud is still rife. They often operate as reporters, rather than solution providers. Targets should be verified financially.
• It is a difficult and a hugely time consuming task to perform a digital audit.
• Dependent on the market, there is little to no shared data, significantly robust, at an industry level. Collectively more information should be shared to raise the profile and scale of this issue.
• It is worth questioning how these companies work with police and criminal task forces. Policing bodies need a greater understanding of this environment, how and where fraudsters are operating, in order to better penalize bot-farm criminals.”

Anonymous

“Real ad fraud verification companies work with law enforcement to prosecute and take down scammers; they don’t issue press releases

There’s so much talk about SIVT when most can’t even detect and stop amateur unsophisticated fraud.

Industry certification bodies that allow for self certification serve no purpose, they are brand safety theater.”

“MRC accreditation review committees are made up of marketers, trade association representatives, and ad tech and agency representatives. They accredit fraud vendors that they like or that they already have deals with and lock others out. Considering that agencies get kickbacks from detection vendors disguised as discounts, this kind of self-dealing should be called out.”

“TAG Certified Against Fraud is fraud. It lets companies self-attest. Even the independently verified ones are bull**** because the verification is done by BPA and the don’t know anything about ad fraud.”