10 Replies - 1003 Views - Last Post: 03 April 2012 - 08:34 PM Rate Topic: -----

#1 aklo  Icon User is offline

  • D.I.C Head

Reputation: 18
  • View blog
  • Posts: 229
  • Joined: 23-January 09

byte array size 1 or 2048 or whatever

Posted 02 April 2012 - 06:19 AM

I'm trying out this code and wondered
byte[] b = new byte[1];
and
byte[] b = new byte[2048];

From javadoc: Reads some number of bytes from the input stream and stores them into the buffer array b

It means that I am able to store more data per read (inputstream.read(B))?
URL img = new URL("http://fc03.deviantart.net/fs71/i/2012/092/9/9/environment_study_02_by_real_sonkes-d4uu8fs.jpg");
    	
InputStream is = img.openStream();
OutputStream os = new FileOutputStream("test.jpg");
    	
byte[] b = new byte[1]; 
    	
int length;

while ((length = is.read(B)/>) != -1) {
    os.write(b, 0, length);
}

os.flush();
os.close();



Is inputstream.read() a 1 way only thing?
An example and some analogy.

inputstream in = abcdefghijklmnop //lets take this as stream of bytes...
byte[] b = new byte[2];

while ((length = is.read(B)) != -1) {
os.write(b, 0, length);
}

each loop of the while loop,
1st loop : os = ab
2nd loop : os = abcd
3rd lopp : os = abcdef

So 2 by 2 it pops out because I did new byte[2]?

I know this post is a mess hope you guys can understand what i'm trying to say.

Is This A Good Question/Topic? 0
  • +

Replies To: byte array size 1 or 2048 or whatever

#2 baavgai  Icon User is offline

  • Dreaming Coder
  • member icon

Reputation: 5800
  • View blog
  • Posts: 12,636
  • Joined: 16-October 07

Re: byte array size 1 or 2048 or whatever

Posted 02 April 2012 - 09:24 AM

JavaDoc

You'd set up a "buffer" of X size. The read command tells you how many bytes it actually got. Each read overwrites the buffer.

In practice, the buffer will always be completely full until maybe the last read, when it will have some fraction of the capacity read. The read after that will have zero, because there's nothing left. Then you're done.

This post has been edited by baavgai: 02 April 2012 - 09:25 AM

Was This Post Helpful? 1
  • +
  • -

#3 r.stiltskin  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1833
  • View blog
  • Posts: 4,927
  • Joined: 27-December 05

Re: byte array size 1 or 2048 or whatever

Posted 02 April 2012 - 06:22 PM

Huh. That's what I expected too, but it's not what I'm seeing.

I ran this code with a 5000 byte array expecting reads of 5000 bytes at a time
import java.io.*;
import java.net.*;

class test1 {

    public static void main(String[] args)  throws MalformedURLException, FileNotFoundException, IOException
    {
        URL img = new URL("http://fc03.deviantart.net/fs71/i/2012/092/9/9/" +
                          "environment_study_02_by_real_sonkes-d4uu8fs.jpg");

        InputStream is = img.openStream();
        OutputStream os = new FileOutputStream("test.jpg");

        byte[] b = new byte[5000];

        int length = 0;
        int next = 0;

        while ((length = is.read(B)/>) != -1) {
            os.write(b, 0, length);
            System.out.println(length);
        }

        os.flush();
        os.close();
        is.close();
    }
}

but this is the output I got:
Spoiler

Then I wanted to see if it was breaking on a particular character, so I modified the code to this:
import java.io.*;
import java.net.*;

class test1 {

    public static void main(String[] args)  throws MalformedURLException, FileNotFoundException, IOException
    {
        URL img = new URL("http://fc03.deviantart.net/fs71/i/2012/092/9/9/" +
                          "environment_study_02_by_real_sonkes-d4uu8fs.jpg");

        InputStream is = img.openStream();
        OutputStream os = new FileOutputStream("test.jpg");

        byte[] b = new byte[5000];

        int length = 0;
        int next = 0;
        /*
                while ((length = is.read(B)/>) != -1) {
                    os.write(b, 0, length);
                    System.out.println(length);
                }
        */

        length = is.read(b, 0, 2616 );
        os.write(b, 0, length);
        System.out.println(length);
        next = is.read();
        os.write((char)next);
        System.out.println( "next char is: " + next );
        next = is.read();
        os.write((char)next);
        System.out.println( "next char is: " + next );
        while ((length = is.read(b, 0, 1451 )) != -1) {
            os.write(b, 0, length);
            System.out.println(length);
            next = is.read();
            os.write((char)next);
            System.out.println( "next char is: " + next );
            next = is.read();
            os.write((char)next);
            System.out.println( "next char is: " + next );
        }

        os.flush();
        os.close();
        is.close();
    }
}


and this time the (surprising) output was:
Spoiler

The only thought that comes to mind is that maybe the read sizes are being limited by TCP or HTTP or something like that. Anybody have any ideas about this?
Was This Post Helpful? 0
  • +
  • -

#4 pbl  Icon User is offline

  • There is nothing you can't do with a JTable
  • member icon

Reputation: 8328
  • View blog
  • Posts: 31,857
  • Joined: 06-March 08

Re: byte array size 1 or 2048 or whatever

Posted 02 April 2012 - 06:32 PM

It is fully explained in the API


public int read(byte[] B)
throws IOException Reads some number of bytes from the input stream and stores them into the buffer array b. The number of bytes actually read is returned as an integer. This method blocks until input data is available, end of file is detected, or an exception is thrown.
If the length of b is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at the end of the file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

The first byte read is stored into element b[0], the next one into b[1], and so on. The number of bytes read is, at most, equal to the length of b. Let k be the number of bytes actually read; these bytes will be stored in elements b[0] through b[k-1], leaving elements b[k] through b[b.length-1] unaffected.

Was This Post Helpful? 0
  • +
  • -

#5 r.stiltskin  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1833
  • View blog
  • Posts: 4,927
  • Joined: 27-December 05

Re: byte array size 1 or 2048 or whatever

Posted 02 April 2012 - 06:40 PM

View Postpbl, on 02 April 2012 - 09:32 PM, said:

It is fully explained in the API

Well, not fully -- unless I'm overlooking something.

The API says "The number of bytes read is, at most, equal to the length of b." It doesn't fully explain why, or when, it will be less than b. In this case, where I set up a 5000 byte buffer, why is it reading one chunk of 2617 bytes, followed by a series of 1452 byte reads?

And why, when I ask it to read 1451 bytes, does it read mostly 1450 byte chunks, but occasionally 1449 and occasionally 1451?
Was This Post Helpful? 0
  • +
  • -

#6 pbl  Icon User is offline

  • There is nothing you can't do with a JTable
  • member icon

Reputation: 8328
  • View blog
  • Posts: 31,857
  • Joined: 06-March 08

Re: byte array size 1 or 2048 or whatever

Posted 02 April 2012 - 06:56 PM

Nothing said that reading from a socket you'll get all the byte at once
If you read from a modem line you might receive 512 bytes frames.
You don;t know from the sender, it can send a packet of a buffer size you ignore, then it may proceed with other requests, and come back later to continue the job. At time time, your client will have read the first buffer.

When you go with your browser on an heavily used site, like your bank site, never seen that the browser can change the layout of the displayed page a few times ? It is because it does not have yet all the information to display the full page. So it starts with what it has, and then reformat the page, as required, when new frames are accepted an available until the whole page is transmitted. Some sites, to let beleive the users that they have an immediate response time, trasnmit a small frame to all clients (so your browser can start to react), then another frame to all clients,... and so on

Hope it is clearer
Was This Post Helpful? 1
  • +
  • -

#7 baavgai  Icon User is offline

  • Dreaming Coder
  • member icon

Reputation: 5800
  • View blog
  • Posts: 12,636
  • Joined: 16-October 07

Re: byte array size 1 or 2048 or whatever

Posted 03 April 2012 - 05:50 AM

InputStream is a top level abstraction.

I was thinking of files, so I'll try to be more clear. Read will try to get data and fill its buffer. If it gets some data on request, it may come back with just that. The reason is that not all streams are created equal.

For fun, here's a test class:

import java.io.*;
import java.net.URL;

public class ReadTest {
	class ProgressMeter {
		private int lastRead, packetCount, totalBytes;
		
		public ProgressMeter() { clear(); }
		
		public int getTotalBytes() { return totalBytes; }
		
		public void clear() {
			lastRead = -1;
			totalBytes = packetCount = 0;
		}
		
		public void display() {
			if (packetCount>0) {
				System.out.println(lastRead + " bytes read " + packetCount + " times");
			}
		}
		public void update(int bytesRead) {
			totalBytes += bytesRead;
			if (bytesRead==lastRead) {
				packetCount++;
			} else {
				display();
				packetCount = 1;
			}
			lastRead = bytesRead;
		}
	}
	
	public final int BufferSize;
	public ReadTest(int BufferSize) { this.BufferSize = BufferSize; }
	public ReadTest() { this.BufferSize = 2048; }
	
	public void copy(InputStream is, OutputStream os) throws IOException {
		ProgressMeter pm = new ProgressMeter();
		
		try {
			byte[] buffer = new byte[BufferSize];
			int bytesRead;

			while ((bytesRead = is.read(buffer))>0) {
				os.write(buffer, 0, bytesRead);
				pm.update(bytesRead);
			}
			pm.update(bytesRead);
		} finally {
			is.close();
			os.close();
		}
		System.out.println(pm.getTotalBytes() + " total bytes read");
	}

	public void copy(InputStream is, File file) throws IOException { copy(is, new FileOutputStream(file)); }
	public void copy(URL url, File file) throws IOException { copy(url.openStream(), file); }
	public void copy(File src, File dst) throws IOException { copy(new FileInputStream(src), dst); }
	
	public void test(String urlPath) throws IOException {
		File file1 = new File("test1.dat");
		File file2 = new File("test2.dat");
		
		System.out.println("Test Buffer Size: " + BufferSize);
		
		System.out.println("Test URL Download");
		copy(new URL(urlPath), file1);
		System.out.println();
		
		System.out.println("Test File Copy");
		copy(file1, file2);
		System.out.println();
	}
	
}



And here's the results of running test:
Test Buffer Size: 2048
Test URL Download
2048 bytes read 1291 times
536 bytes read 1 times
2048 bytes read 254 times
1512 bytes read 1 times
2048 bytes read 33 times
848 bytes read 1 times
2048 bytes read 1 times
600 bytes read 1 times
2048 bytes read 93 times
600 bytes read 1 times
2048 bytes read 1313 times
646 bytes read 1 times
6118021 total bytes read

Test File Copy
2048 bytes read 2987 times
646 bytes read 1 times
6118021 total bytes read



It's clear I get some stutters on network traffic. And this is on a LAN, not even journeying out to the internet. My file copy behaves as I described it. Sorry for the confusion.
Was This Post Helpful? 1
  • +
  • -

#8 r.stiltskin  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1833
  • View blog
  • Posts: 4,927
  • Joined: 27-December 05

Re: byte array size 1 or 2048 or whatever

Posted 03 April 2012 - 07:06 AM

So would it be accurate to say that read will be "satisfied" (i.e. stop blocking and extract the data) whenever it finds anything in the stream?

And the read amounts that we're seeing are being limited by the amounts of data that the OS found in the packets it received, and the relative timing of when the OS actually placed that data in the stream vs. when the application gets to access the stream?
Was This Post Helpful? 0
  • +
  • -

#9 baavgai  Icon User is offline

  • Dreaming Coder
  • member icon

Reputation: 5800
  • View blog
  • Posts: 12,636
  • Joined: 16-October 07

Re: byte array size 1 or 2048 or whatever

Posted 03 April 2012 - 07:39 AM

That appears to be case.

Network IO is a little hairy. Retries actually change the size of the pay load, etc.

I don't think I've seen the behavior in C/C++. It usually goes for a full bucket. But a socket library might do something like this.
Was This Post Helpful? 0
  • +
  • -

#10 pbl  Icon User is offline

  • There is nothing you can't do with a JTable
  • member icon

Reputation: 8328
  • View blog
  • Posts: 31,857
  • Joined: 06-March 08

Re: byte array size 1 or 2048 or whatever

Posted 03 April 2012 - 06:47 PM

Your initial code will work


while ((length = is.read(octet)) != -1) {
        os.write(octet, 0, length);


if length returned is 0, the write will simply do nothing
and you back reading the stream until -1 is returned
Was This Post Helpful? 0
  • +
  • -

#11 pbl  Icon User is offline

  • There is nothing you can't do with a JTable
  • member icon

Reputation: 8328
  • View blog
  • Posts: 31,857
  • Joined: 06-March 08

Re: byte array size 1 or 2048 or whatever

Posted 03 April 2012 - 08:34 PM

View Postr.stiltskin, on 03 April 2012 - 10:06 AM, said:

So would it be accurate to say that read will be "satisfied" (i.e. stop blocking and extract the data) whenever it finds anything in the stream?

And the read amounts that we're seeing are being limited by the amounts of data that the OS found in the packets it received, and the relative timing of when the OS actually placed that data in the stream vs. when the application gets to access the stream?

Yes. You've got it :^:
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1