7 Replies - 6388 Views - Last Post: 26 July 2012 - 08:07 PM

#1 apaddobs  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 49
  • Joined: 16-May 12

Compress file while sending over the socket

Posted 21 May 2012 - 05:29 PM

    my $self = shift;
    my $filename = shift;
    my $fh = new FileHandle ($filename);
    if (not defined ($fh)) {
        $self->message ("$filename: $!");
        return 0;
    }
    my $filesize = (stat ($filename)) [7];
    binmode ($fh);
    $self->_w2 (START);
    my $buf;
    my $written = 0;
    while ($filesize > 0) {
        my $bufsiz = $fh->read ($buf, 512);
        $self->_write ($buf, $bufsiz);
        $filesize -= $bufsiz;
    }
    $self->_w2 (EOF);
    $fh->close;
}

sub _w2 {
    my $self = shift;
    my $val = shift;
    $self->_write (pack ("n", $val), 2);
}

sub _write {
    my $self = shift;
    my $bytes = shift;
    my $nbytes = shift;
    $sock->syswrite ($bytes, $nbytes);
}



Sending a file of 2G takes quite sometime to go through. If I want to compress it over the socket and send, how do i do it? i.e., i don't want to gzip,tar and send the tar.gz over the socket. Coz that will put the overhead of untarring(zipping) at the other end.

--Apaddobs.

Is This A Good Question/Topic? 0
  • +

Replies To: Compress file while sending over the socket

#2 GunnerInc  Icon User is offline

  • "Hurry up and wait"
  • member icon




Reputation: 858
  • View blog
  • Posts: 2,284
  • Joined: 28-March 11

Re: Compress file while sending over the socket

Posted 21 May 2012 - 05:33 PM

Well any type of "compression" that you do on your data even as simple a s RLE, the client still has to undo your compression. Overhead in todays computers is a mute point, either have them sit there and wait for 2gb to download or compress it and let their computer uncompress it... most will vote for the later.

But the question is also, what is in the 2gb of data? Not all data can be compressed as well as your hoping.
Was This Post Helpful? 0
  • +
  • -

#3 turboscrew  Icon User is offline

  • D.I.C Addict

Reputation: 100
  • View blog
  • Posts: 624
  • Joined: 03-April 12

Re: Compress file while sending over the socket

Posted 21 May 2012 - 09:07 PM

Isn't there a Zlib or something that lets you compress blockwise?

This post has been edited by turboscrew: 21 May 2012 - 09:08 PM

Was This Post Helpful? 0
  • +
  • -

#4 JackOfAllTrades  Icon User is offline

  • Saucy!
  • member icon

Reputation: 6074
  • View blog
  • Posts: 23,540
  • Joined: 23-August 08

Re: Compress file while sending over the socket

Posted 22 May 2012 - 02:13 PM

Moved to Perl
Was This Post Helpful? 0
  • +
  • -

#5 dsherohman  Icon User is offline

  • Perl Parson
  • member icon

Reputation: 226
  • View blog
  • Posts: 654
  • Joined: 29-March 09

Re: Compress file while sending over the socket

Posted 23 May 2012 - 05:27 AM

There are several compression-related modules available for Perl, it's just a matter of choosing one that uses a compression format that the receiving software will know how to decompress. As GunnerInc said, any compression you do will require the receiving computer to decompress the data, but the computational overhead of doing so is negligible these days.
Was This Post Helpful? 1
  • +
  • -

#6 apaddobs  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 49
  • Joined: 16-May 12

Re: Compress file while sending over the socket

Posted 24 July 2012 - 08:51 AM

View Postdsherohman, on 23 May 2012 - 05:27 AM, said:

There are several compression-related modules available for Perl, it's just a matter of choosing one that uses a compression format that the receiving software will know how to decompress. As GunnerInc said, any compression you do will require the receiving computer to decompress the data, but the computational overhead of doing so is negligible these days.

Ok so what I am saying is this. If I have a.jar, b.jar and c.jar (each being say close to 1G each), would creating an total.tar.gz (consisting a.jar, b.jar and c.jar) and sending the total.tar.gz over the socket (like i would send a normal file) make sense at all?
I'm looking for something like tar -zxvf a.tar | more (wherein the |more starts doesn't wait for the entire tarring to happen and I can have the "more" part ready to be sent over the socket). Did i explain it correctly?
Was This Post Helpful? 0
  • +
  • -

#7 GWatt  Icon User is offline

  • member icon

Reputation: 278
  • View blog
  • Posts: 3,078
  • Joined: 01-December 05

Re: Compress file while sending over the socket

Posted 24 July 2012 - 10:21 AM

You can pipe to ssh and have that write data over the network.
tar cz folder/ | ssh host cat \> /file/on/remote/computer


Was This Post Helpful? 0
  • +
  • -

#8 apaddobs  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 49
  • Joined: 16-May 12

Re: Compress file while sending over the socket

Posted 26 July 2012 - 08:07 PM

View PostGWatt, on 24 July 2012 - 10:21 AM, said:

You can pipe to ssh and have that write data over the network.
tar cz folder/ | ssh host cat \> /file/on/remote/computer



Correct. The question was how to achieve this in my Perl code wherein I open a socket. That is how to incorporate this in the $fh->read and $socket->write.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1