Sign In/My Account | View Cart  

advertisement

AddThis Social Bookmark Button

Article:
  A New Visualization for Web Server Logs
Subject:   Works like a charm
Date:   2007-02-09 07:20:51
From:   iconara
Very good idea, and an excellent article. I adapted your scripts to work with my IIS-generated log files:




while ( <> ) {

	# IIS log files have comments which start with "#", skip those

	next if /^#/;

		

        chomp;


	# this is the IIS log format, it may vary, but it prints the format as a comment at the beginning of the file

	my (undef, $time, $ip, undef, undef, undef, undef, undef, undef, $url, undef, $sc) = split;

        

	next if ($url =~ /(gif|jpg|png|js|css)$/);


	// no need to reformat the time so I removed that line


        print TEMP "$time $ip $url $sc\n";


        $ipList{$ip}++;

        $urlList{$url}++;

}


And I also added this to the gnuplot-commands to make it create a png (add before "splot ..."):




set output "plot.png"

set terminal png
Full Threads Oldest First

Showing messages 1 through 5 of 5.

  • Forgot the time format
    2007-02-10 03:28:03  iconara [View]

    I forgot that I also changed the time format (since IIS has date and time as two separate items in the log file). You need to change the line begining with "set timefmt" to:


    
set timefmt "%H:%M:%S"
  • Works like a charm
    2007-02-09 11:01:13  rajuvarghese [View]

    Good work extending the script for IIS logs. The output to png files was planned for the next part of the article. The plot is going to be colorful as well. Gnuplot 4.2 has reached RC4 and release is imminent. Glad you liked the article.
    • Works like a charm
      2007-02-09 12:58:44  bjelkeman [View]

      Interesting. I have been working with web traffic analysis tools more or less since they came out and I have been saying that the current tools are very poor at giving good overviews of traffic. Mapping the traffic in 3d, there are several ways to do this of course, is a great way to increase the amount of information you can show in a graph. Not a lot of people think like me apperently, as essentially every developer I have talked to have essentially shrugged their shoulders at the suggesions.
      • I agree: sysadmins have no graphical tools
        2007-02-09 15:20:25  rajuvarghese [View]

        As I mentioned in the article, standard analysis tools seem to cater more to publishers/editors than to the sysadmins. Though web servers have been around for over 13 years, one does not see much progress on that front. Further, none of the standard tools that I am familiar with have 3D views of data. I believe that interactive 3D charts could make the data flood less intimidating for both publishers and sysadmins.
        • I agree: sysadmins have no graphical tools
          2007-02-15 13:36:31  ksfiles [View]

          If you like 3D visualization of network traffic, you should definitely check out Merit's Flamingo tool. It provides lots of interesting ideas.

          http://flamingo.merit.edu/gallery.html

          Thanks,
          --kirby