Site logo
Luxxle Web Crawler
Introduction
Luxxle uses web crawlers to improve its index and to provide the best possible service to our users.
"Crawlers" are programs, also known as robots or spiders, that are used to scan websites and by finding and following links from one webpage to another. Luxxle's primary crawler is called Luxxlebot, and this page contains information about what our crwalers do and how they interact on other websites.
User-agent
When our crawlers visit a website, they identify themselves with various user-agents which contain information about the crawler and its specific version. Our crawlers are always identifiable in a website's referrer logs by the names Luxxlebot or Luxxlebot-news. They should be specified in robots.txt, the robots meta tags, and the X-Robots-Tag HTTP directives with these names.
Crawler Verification

How to check to see if a web crawler from Luxxle is accessing your server:

1. Do a DNS lookup on the IP address

2. Check to see if the domain name contains "luxxle.com"

3. Do a forward lookup on the previous domain name

4. Check to see if the IP address matches the original one
Questions?

If there was a problem when our crawlers visisted your site, we're extremely sorry for any inconvenience.
Please contact us with any problem caused by the crawlers through our contact page.