- Parses URLs from a TXT file with your custom priority & frequency settings
- Crawls the site like any SE would do (http,https,javascript,window,frame,iframe and more!)
- Limits crawling to specified indexing depth
- Filters out over 40 different file extensions not supported by SE
- Filters out duplicate URLs or URLs pointing to the same content
- Filters out all pages disallowed in robots.txt
- Filters out all links with nofollow tag
- Filters out all outbound links
- Filters out all links to subdomains
- Filters out all non text or html files (option to allow pdf,doc,xls,ppt)
- Generates XML sitemap code - obviously!
- Zips XML file in SE's preferred format gzip
- Pings Google, Yahoo! & MSN with sitemap's location
- Allows only SE bots to view XML's zip, returns '404 Not Found' for web browser
- Notifies you when sitemap is generated and everytime it is crawled with all crawler's details
- Outputs full detailed report of all what got done everytime it generates a sitemap