Class ApacheLogRobotsProcessor


  • public class ApacheLogRobotsProcessor
    extends Object
    Command line utility to create a file of spider addresses from an Apache log file.
    Author:
    Mark Diggory (mdiggory at atmire.com), kevinvandevelde at atmire.com, ben at atmire.com
    • Method Detail

      • main

        public static void main​(String[] args)
                         throws Exception
        Creates a file containing spiders based on an Apache log file by analyzing users of the robots.txt file
        Parameters:
        args - the command line arguments given
        Throws:
        Exception - if error