2 Replies - 8755 Views - Last Post: 19 June 2012 - 10:33 AM

#1 prescott2006   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 12
  • Joined: 13-November 07

Compare multiple files and output differences

Posted 07 March 2012 - 08:10 PM

I am writing a script to read a text file, say a.txt, which contains a list of text file to be analyzed, say b.txt, c.txt and d.txt. Each of the a, b and c will be opened and traced to capture a certain string using pattern matching. The result will be stored in corresponding log file. After that b.log will be treat as a reference, and c.log and d.log will be compared with b.log and the missing and extra entries of them will be output to result.list. This is the code I had written so far.

#!/usr/bin/perl

print "Enter filename:";
chomp($fname = <STDIN>);

# Open the file which contains list of files to be opened.  If unsuccessful, print an error message and quit.

open (file1,$fname) || die "Can't Open File: $fname\n";

while (<file1>)
{
  chomp;
  open (file2, $_);   # Open file in the list one by one
  $temp = $_;         # Store the file name to name the corresponding log file
  print "$temp\n";    # Print the file name to indicate the different output
  while (<file2>)
  {
    chomp;
    sub grep_pattern	# Print strings which contain the pattern
    {foreach ($_)
    {if (/$pattern/)
     {  print "$1\n";      
        open (file3, ">>$temp.log");    
        print file3 "$1\n";     # Print result in log file in append mode
        close (file3);    
     }
    }
    }
    $pattern = 'hello '; # Find the pattern
    grep_pattern;
  }
  print "\n";   # Print a blank line to seperate the results from each file
}
close (file1);


But at best the log file should not be used and probably the intermediate result should be stored in some arrays or hashes and directly be compared. But I don't know how to create different array for each of the b.txt, c.txt and d.txt automatically. Anyone can help?

Is This A Good Question/Topic? 0
  • +

Replies To: Compare multiple files and output differences

#2 spek_   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 1
  • Joined: 12-March 12

Re: Compare multiple files and output differences

Posted 12 March 2012 - 10:22 AM

As far as I could tell, you are trying to do something like the following. It reads a.txt, which contains a list of other filenames like this:

b.txt
c.txt
d.txt



...then b.txt, c.txt and d.txt have entries something like this:

hello from b.txt
this line shouldn't match
this line hello should match
shouldn't
hello should match



...then you want to grab all of the lines from b, c and d, and save all those lines that match $pattern, removing duplicates:

#!/usr/bin/perl

use warnings;
use strict;

print "Enter filename:";
chomp( my $list_filename = <STDIN> );

open my $list_fh, '<', $list_filename 
  or die "Can't open the file that contains the other filenames!: $!";

my @storage; # store the lines in here
my $pattern = qr/hello /;

while ( my $inner_filename = <$list_fh> ){

  chomp $inner_filename;

  # ensure the files actually exist (-e)

    if ( -e $inner_filename ){

        open my $inner_file_fh, '<', $inner_filename
          or die "Can't open $inner_filename!: $!";

        print "$inner_filename\n";    # Print the file name to indicate the different output

        while ( my $inner_line = <$inner_file_fh>) {

            chomp $inner_line;

            if ( $inner_line =~ /$pattern/ ){

                print "$inner_line\n";
                push @storage, $inner_line; 
            }
        }
        close $inner_file_fh;
    }
}
close $list_fh;

# combine all lines removing duplicates
# see: perldoc -q duplicate

my ( @unique, %seen );

foreach my $elem ( @storage ){
    next if $seen{ $elem }++;
    push @unique, $elem;
}

print "\n\nResults:\n\n";
print join( "\n", @unique );



I'll leave it up to you do do any further output to other files :)
Was This Post Helpful? 0
  • +
  • -

#3 TheBlueDalek   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 1
  • Joined: 19-June 12

Re: Compare multiple files and output differences

Posted 19 June 2012 - 10:33 AM

Hello

Just found this forum because I am looking for something like this script. On the surface it looks exactly what I want it to do. However, I am wondering if there is anyway that the script can look at another file as a template...and report back any differences?

I am not very familiar with Perl, so any help would be appreciated!

I have between 50 & 70 log files, that in theory should all be the same. however, I am looking for a discrepancy in one or more than one of the files.

The files can exceed 100k lines in length.

I have one file that I know is as it should be.. lets call is timestamp01.log
I have a txt file that lists all of the log files to be checked.. we'll call is list.txt

Using the script as is, if I manually modify one of the files at random and insert the word 'not' into it, then run the script with 'not' entered as the item to search for... it returns all of the logs, with the offending entry displayed.

Is there anyway to use this script and have it call a file as reference instead of a keyword?

Hope this made sense?

Many thanks!
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1