Press release

First-of-its-Kind Study Ranks Spam Detection Providers’ Ability to Accurately Detect Unwanted Robocalls

businesswire,
0
Sponsored by Businesswire

Global
Wireless Solutions, Inc. (GWS) and Hiya
today shared findings from a study that assessed how well wireless
carrier spam detection solution providers (Hiya, First Orion and TNS)
detect unwanted robocalls. This study introduces a first-of-its-kind
methodology designed by Hiya to set a new industry standard, focusing
specifically on spam “identification rate” and spam “error rate,” and
assigning an overall “detection score” based on the combination of these
two metrics.

The results of this
study revealed the following:

  • Identification Rate (portion of all confirmed spam phone
    numbers in the dataset tested that the provider correctly classified
    as spam):

    • For this study, Hiya’s identification rate was 87%, First Orion’s
      was 55% and TNS’ was 34%.
  • Error Rate (portion of all calls that a provider classified as
    spam, that were not actually spam, i.e. false positives):

    • For this study, Hiya’s error rate was 0.25%, First Orion’s was
      1.26% and TNS’ was 1.48%.
  • Detection Score (a calculation that takes the Identification
    Rate and the Error Rate and applies a penalty for incorrectly
    classifying a number as spam):

    • The overall Detection Score for Hiya was 85 versus First Orion’s
      at 48 and TNS’ at 26.

New Industry Standard for Evaluating Spam Detection Capabilities

In order to provide the most scientific way of testing each spam
detection provider’s ability to accurately detect spam calls, Hiya
developed a unique and first-of-its-kind approach to the testing
methodology. Instead of testing static “lists” of known spam numbers,
they examined actual phone traffic over the course of 24 hours and
leveraged the resulting set of numbers to test and determine how the
providers classified each number. GWS worked with Hiya by manually
verifying the number classifications that did not correlate across all
three of the solution providers. A detailed explanation of the
methodology and why Hiya is suggesting it as the new industry standard
for testing can be found here.

The Rise in Robocalls and the Importance of Accurately Detecting Spam

The rise in robocalls continues to be a major issue affecting American
consumers and regulators. FCC Chairman Ajit Pai recently threatened
regulation if US carriers don’t work faster to adopt a robust call
authentication system (STIR/SHAKEN) to combat illegal caller ID
spoofing. A bipartisan group of senators also recently re-introduced the TRACED
Act, a bill that would dramatically increase the fines faced by
robocallers. The issue of accuracy and correctly identifying a call –
whether wanted or unwanted – is important both to protect consumers from
annoying spam and potentially harmful scams, but also so they don’t miss
important calls from numbers they might not otherwise recognize
(schools, doctor’s office, emergency services, etc.).

Methodology Overview:

The study was conducted in December 2018 by GWS and Hiya to compare the
spam filtering capabilities of Hiya, First Orion and TNS. This study was
based on an extensive list of phone numbers gathered by Hiya from a
sampling of neutral network traffic instead of using a static list of
known spam numbers. The preliminary tests performed by Hiya evaluated
over 5,000 phone numbers to determine the caller ID classification
assigned by each spam detection provider, ultimately categorizing the
numbers as some form of spam or not. This “call down” process involved
synchronized dialing through an automated system to multiple mobile
phones with spam detection software installed and documenting the caller
ID label. From the initial numbers tested, a subset of 751 numbers was
identified where the three providers’ caller ID label disagreed on
whether the call was spam or not. Those numbers were provided to GWS to
apply manual verification via online research and verification calls if
needed to determine the correct classification (spam or not spam).

About Global Wireless Solutions, Inc.:

Founded in 1996 and headquartered in Dulles, Virginia, Global Wireless
Solutions has set the industry standard for network benchmarking,
analysis and testing. Its proprietary OneScore network ranking combines
high-level network testing data with real-world network usage scenarios
to make the results of testing relatable and easy to understand. Working
with some of the world’s largest wireless network providers, GWS offers
high-quality network data and engineering analysis through a suite of
benchmarking products, services and OneMeasure™ diagnostic apps that
include drive, venue and in-building testing. At last count, GWS has
driven 12 million data collection miles for its customers. For more
information, visit gwsolutions.com
and follow us on Twitter at @gwsolutionsinc.