Expand my Community achievements bar.

Improve IP-to-Domain Resolution and bring in Organization name from WHOIS

Avatar

Level 5

6/25/15

We have found that 628,000 IP addresses, or 23% of IPs that have accessed our site in the last two months, do not resolve to a domain in the Domain report. This is despite the fact that the first 20 no-domain IPs we tested are listed in APNIC and commercial databases like IP2Location with a domain.

 

Additionally, domain is only so useful, as it's often the ISP's domain and not the registering organization. We're looking into using a third party data source to bring this information in, but surely other Adobe customers want to know which companies and organizations are visiting their sites. Is there any chance of integrating with one of these databases to provide whois organization?

5 Comments

Avatar

Level 1

7/5/16

Our research is similar, with over 20% listed as "None" in the Domains report. 

 

This has significant impact on segments that use "Domains", and it's not just because over 20% of the Domains report consistently contains "None". It's the fact that over a given timespan some IP address have multiple "domain" values, primarily "None" and the ISP domain. And we all know how "Some, but not all, data" is often worse than "No data", and here "Some data" means "Just enough data to see a particular desired company's domain in a report and build a segment based off of it, not knowing that only a small % of traffic from applicable IP addresses were correctly associated with that domain."

 

Here is a good example:

  • Using Data Warehouse, from January 1st through May 31st this year an example IP address in our reports only had 1.7% of the page views associated with the correct domain, 7.7% of the page views had the domain of that company's internet service provider, and 90.6% have a blank domain ("None").
  • So, say there was a segment with "Visitor domain equals somecompanydomain.com" (i.e. "Show me all reporting data from all visits where at any time that visitor had domain=somecompanydomain.com"). If the domain did not equal "somecompanydomain.com" at the time of any visits during the selected reporting timeframe, reporting data from that customer IP address wouldn't be in any report with that segment applied. It doesn't matter if the domain did equal "somecompanydomain.com" at some time before and/or after the selected reporting timeframe.

Suggestions:

  1. At the time of this writing, Adobe uses Digital Envoy for their Geolocation reports and the Domains report (likely the NetAcuity product from Digital Element), but Digital Envoy's emphasis seems to be on Geolocation data not "Domain name". At a minimum, I would ask that Adobe push Digital Envoy to improve the quality of their "Domain name" data, and strongly consider either supplementing or replacing the "Domain Name" portion of that service with another company (obviously after comparing a company's domain data files OVER A SPAN OF TIME with each other, and also comparing the quality of the domain name data with Digital Envoy's domain name data.)
  2. I understand the reality that IP addresses are reassigned to different domain names over time, however I believe that aside from improving the quality of the source IP/domain lookup data, there should also be some reasonable logic applied by Adobe Analytics to backfill the "None" domains and also to replace the ISP/Rackspace provider domain name with the deeper domain name of the ISP's business customer (i.e. "somecompanydomain.com") if/when it becomes available.
  3. Another option would be to allow us to upload IP/Domain mapping tables to supersede and supplement the data in the Domains report.

Avatar

Level 5

7/5/16

Hi Philip,

 

Regarding this point:

  1. Another option would be to allow us to upload IP/Domain mapping tables to supersede and supplement the data in the Domains report.

We copy IP Address to an eVar using a processing rule, and then utilize SAINT Classifications to upload IP2Location data in. Unfortunately, you can't really hide the "Domains" report in all interfaces, so along with other out-of-the-box reports that we've copied and classified (e.g. referring domain), it adds to the learning curve for new org users who need to know what's redundant and which version is better.

 

Additionally, on our high-volume site, IP addresses have a high enough cardinality each month that many get folded under (Low Traffic), so you need to be careful about partial coverage of your classification. Best to use Data Warehouse.

Avatar

Level 1

7/6/16

I considered that. The "max allowed monthly unique eVar values" is another reason that particular suggestion was for the Domains report. This is my least desired suggestion anyway because Adobe Analytics customers likely either do not have the skills or time for it.

 

My primary suggestion is that Adobe improve/replace the Domain data they currently get along with the Geolocation data from Digital Envoy. (I'm not suggesting they improve/replace the Geolocation data they also get from Digital Envoy)

 

Side note: The IP2Location home page says my location is New York, NY. Most IP Geolocation services (including Digital Envoy) get close and resolve my company's IP to Littleton, CO but some correctly resolve it to Englewood, CO.