[atraub]

Hey guys. I've been trying to do some stuff that shouldn't be too terribly hard, but I'm sure I've somehow managed to make it more difficult than it has to be

I'm trying to find and replace text across multiple files. I'll need to do this for a lot of text (ugh) and I found this beautiful one liner

perl -pi -w -e 's/SEARCH_FOR/REPLACE_WITH/g;' *.txt

This works great for what I'm doing, so I wanted to make it a little simpler to run. First, I thought I could just make a bash function, something like this:

function replace{
    perl -pi -w -e 's/$1/$2/g;' *$3
}

But that gives me plenty of errors

bash: .bashrc: line 110: syntax error near unexpected token `perl'
bash: .bashrc: line 110: `    perl -pi -w -e 's/$1/$2/g;' *$3'

Soo, I thought it might be smarter to make a perl script... but how do I do that??? I tried this

#!/usr/bin/perl -w

's/$ARGV[0]/$ARGV[1]/g;'

but obviously only met failure. Admittedly, it's been a LONG time since I've used perl, and I've never done anything like this. I definitely could use some help!

This post has been edited by atraub: 02 October 2012 - 01:56 PM

[sepp2k]

Disclaimer: It's been a while since I've done Perl as well (and I don't think I was ever good at it), so I might make mistakes here or recommend things that are bad practice. I hope this helps anyway.

The problem with your shell script is that the shell is more space-sensitive than you might think. Since there was no space between your function name and the opening brace, the shell's lexer took the brace to be part of the function name. So when the parser encounters the token "perl", it's still looking for an opening brace and gives up in confusion. Thus the error.

tl;dr: Add a space between the function name and the opening brace to make the error go away.

Another issue with your shell script is that the single quotes will prevent variables from being expanded inside them, so $1 and $2 won't be expanded. To fix this, use double quotes instead.

Now to your Perl script:

First of all the quotes around the Perl code were part of the shell syntax - not of the Perl code. So remove those.

Now another problem are the command line flags that you used in the shell, but not in your Perl script - specifically -p and -i.

The -p command line flag causes Perl to loop over the available input (either from the filenames in ARGV or, if ARGV is empty, from stdin), execute your code on each iteration and then print the value stored in $_ (which will contain the result of the regex substitution in your case).

Without that flag, no input will be read unless you read it (using the diamond operator for example) and nothing will be printed unless you print it (using print). So your code could look like this:

while(<>) {
    s/$ARGV[0]/$ARGV[1]/g;
    print;
}

But actually that won't work either. Why not? Well, in the original version of your one-liner the command line arguments were used to signify which files to read the input from. Now we're using the command line arguments to signify which string to replace with what. So where do we want to read input from? Either we could take the input file(s) as another argument(s) or we could simply read from stdin.

To read from stdin, use <STDIN> instead of <>. To read from a file, open it with open and then use <your_file_handle>.

Now the only problem is that the script writes its output to stdout. Your original code modified the files in-place because of the -i command line flag. To get the same behavior without -i, look into opening files in write mode and writing to them.

As an alternative to doing all that you might also just add the -p and -i flags to your shebang line. If you go that way, you'll need to remove the source and target strings from ARGV (using shift possibly), so that they won't be interpreted as filenames.

This post has been edited by sepp2k: 02 October 2012 - 02:34 PM

[dsherohman]

atraub, on 02 October 2012 - 09:56 PM, said:

I tried this

#!/usr/bin/perl -w

's/$ARGV[0]/$ARGV[1]/g;'

You're so close... You already took the -w command-line argument and added it to the shebang ($#) line, so how about putting the other command-line arguments there, too?

#$/usr/bin/perl -piw

s/SEARCH_FOR/REPLACE_WITH/g;

I'm pretty sure that should work. Note, though, that it has to hard-code the search/replace terms because -pi uses @ARGV as a list of filenames to process, so you can't use it to also pass in parameters.

If you need to pass in the search/replace terms on the command line (which is certainly more convenient!), you'll need to use plain Perl code instead of the -pi switches. Fortunately, perldoc perlrun tells us what regular Perl code they're equivalent to - search the page for perl -p -i.orig -e "s/foo/bar/; ... " to find it.

Using that as a basis and adjusting for passing the search/replace terms on the command line and your not needing a backup, I came up with this:

#!/usr/bin/perl

use strict;
use warnings;

my $search_for = shift @ARGV;
my $replace_with = shift @ARGV;

my $oldargv;
LINE: while (<>) {
  if ($ARGV ne $oldargv) {
    unlink($ARGV);
    open(ARGVOUT, ">$ARGV");
    select(ARGVOUT);
    $oldargv = $ARGV;
  }
  s/$search_for/$replace_with/;
}
continue {
  print;	# this prints to original filename
}
select(STDOUT);

(Untested, but should be correct, at least in a Unix/Linux environment. Under Windows, you'll probably have to write to a new file, then delete the old one after you're done with it instead of immediately deleting the file.)

[atraub]

Thanks guys. I really appreciate the work you guys put in, but ultimately, I had to go with what was quickest and easiest so I could keep moving with my work:

#bash function
function replace {
    perl -pi -w -e "s/$1/$2/g;" *$3
}

[atraub]

So, I've used my perl script for a lot of simple things so far, but now I have a tougher one that it's not working for.

Here's my 3 attempts with it:

replace '"degradation"  :"^3^",' '"degradation"  :"^4^",'
replace "\"degradation\"  :\"^3^\"," "\"degradation\"  :\"^4^\","
replace "\\\"degradation\\\"  :\\\"^3^\\\"," "\\\"degradation\\\"  :\\\"^4^\\\","

As you can see, I'm trying to replace "degradation" :"^3^" with "degradation" :"^4^".

The first two attempts are pretty straight-forward, I tried to pass the values into the function either by using single quotes to hold the string or by escaping the double quote. The third attempt want to double-escape everything so that when it gets passed into the perl function, the strings would be sent in with escaped quotes. Nothing seems to work though, any advice?

This post has been edited by atraub: 12 October 2012 - 05:47 AM

[sepp2k]

The problems aren't the quotes. There's no need to escape the quotes for Perl - your first version will work fine as far as the quotes are concerned.

The problem is that ^ is a special character in regular expressions, so you need to escape that one.

[atraub]

I tried this:

replace '"degradation"  :"\^3\^",' '"degradation"  :"\^4\^",'

but I'm still not having any success.

EDIT:
While you probably would assume this, no luck with this either

replace "\"degradation\"  :\"\^3\^\"," "\"degradation\"  :\"\^4\^\","

EDIT 2:
and for good measure

replace '"degradation"  :"\\\^3\\\^",' '"degradation"  :"\\\^4\\\^",'

also did not work

This post has been edited by atraub: 12 October 2012 - 06:02 AM

[sepp2k]

Are you sure that your files contain what you think they do?

If I create a file bla.txt that contains only

"degradation"  :"^3^",

and then run

replace '"degradation"  :"\^3\^",' '"degradation"  :"\^4\^",' bla.txt

, the file is changed as expected.

This post has been edited by sepp2k: 12 October 2012 - 06:14 AM

[atraub]

Doh! copy and paste issue. When I copy the stuff to change from the terminal, it removed the tab. adding in a \t fixed the issue... damn tabs lol

Thanks man

[dsherohman]

atraub, on 12 October 2012 - 03:28 PM, said:

Doh! copy and paste issue. When I copy the stuff to change from the terminal, it removed the tab. adding in a \t fixed the issue... damn tabs lol

If you use \s+ in your regex, that will match either spaces or tabs (and any number of either).