14 Replies - 5197 Views - Last Post: 03 August 2011 - 02:24 PM Rate Topic: -----

#1 jsam84  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 4
  • Joined: 09-January 09

Java file binary read for statistical analysis

Posted 20 July 2011 - 07:47 AM

I am working on a project where I need to read in a file in binary mode (which I can accomplish) and be able to manipulate it on the binary level for statistical analysis. I have tried using a bitset but it has not worked for me. The default way for java to manipulate binary data is in bytes but I need bits. Does anyone have experience breaking data out into bits? I have thought about just using bools but they seem to waste space.
Is This A Good Question/Topic? 1
  • +

Replies To: Java file binary read for statistical analysis

#2 cfoley  Icon User is offline

  • Cabbage
  • member icon

Reputation: 1940
  • View blog
  • Posts: 4,027
  • Joined: 11-December 07

Re: Java file binary read for statistical analysis

Posted 20 July 2011 - 08:27 AM

BitSet would have been my first choice too. What happened for you to write it off? Maybe it's a problem we can solve.

Other than that, the bit operators might be of interest to you: http://download.orac...dbolts/op3.html
Was This Post Helpful? 0
  • +
  • -

#3 g00se  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2652
  • View blog
  • Posts: 11,189
  • Joined: 20-September 08

Re: Java file binary read for statistical analysis

Posted 20 July 2011 - 03:24 PM

Quote

Does anyone have experience breaking data out into bits?


What do you need to do?
Was This Post Helpful? 0
  • +
  • -

#4 jsam84  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 4
  • Joined: 09-January 09

Re: Java file binary read for statistical analysis

Posted 20 July 2011 - 05:03 PM

View Postcfoley, on 20 July 2011 - 08:27 AM, said:

BitSet would have been my first choice too. What happened for you to write it off? Maybe it's a problem we can solve.

Other than that, the bit operators might be of interest to you: http://download.orac...dbolts/op3.html


bitsets after I initialized to 0 then loaded data in only 17 were filled not 1024 maybe I am not understanding how they work

View Postg00se, on 20 July 2011 - 03:24 PM, said:

Quote

Does anyone have experience breaking data out into bits?


What do you need to do?



Basically for the first phase is I need to load a file in binary format into some sort of array type mechanism and iterate through counting 1's and 0's as they happen.
Was This Post Helpful? 0
  • +
  • -

#5 cfoley  Icon User is offline

  • Cabbage
  • member icon

Reputation: 1940
  • View blog
  • Posts: 4,027
  • Joined: 11-December 07

Re: Java file binary read for statistical analysis

Posted 20 July 2011 - 05:58 PM

Quote

bitsets after I initialized to 0 then loaded data in only 17 were filled not 1024 maybe I am not understanding how they work


Can you post your code to load them so we can see what's up?
Was This Post Helpful? 0
  • +
  • -

#6 pbl  Icon User is offline

  • There is nothing you can't do with a JTable
  • member icon

Reputation: 8324
  • View blog
  • Posts: 31,857
  • Joined: 06-March 08

Re: Java file binary read for statistical analysis

Posted 20 July 2011 - 08:18 PM

View Postjsam84, on 20 July 2011 - 10:47 AM, said:

I have thought about just using bools but they seem to waste space.

Actually, in Java, boolean in a class, and in methods, are implemented as a bitset. So an array of 8 boolean takes a byte :)
But you can't read 0 and 1 and make them boolean as in C/C++

So better to go with a BitSet, just fill it in byte per byte
Was This Post Helpful? 1
  • +
  • -

#7 g00se  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2652
  • View blog
  • Posts: 11,189
  • Joined: 20-September 08

Re: Java file binary read for statistical analysis

Posted 21 July 2011 - 02:59 AM

Personally, while the BitSet class is a great idea, it misses some tricks and i've never found it tobe of much (any?) practical use.

Quote

Basically for the first phase is I need to load a file in binary format into some sort of array type mechanism and iterate through counting 1's and 0's as they happen.


BitSet problem number one: is it just me or is there no way to derive a BitSet from a numerical primitive? The most obvious ctor would be BitSet(int) where the int is represented as a bit set. Their own BitSet(int) ctor is a lot less useful.

The following is the sort of thing you need:

import java.io.*;

public class Bits {
    public static void main(String[] args) throws IOException {
	InputStream in = new FileInputStream(args[0]);
	int buf = -1;

	while((buf = in.read()) > -1) {
	    // Visualize
	    buf &= 0xFF;
	    int cursor = 0x80;
	    for(int i = 0;i < 8;i++) {	
		System.out.print((buf & cursor) > 0? '1' : '0');
		cursor >>= 1;
	    }
	}
	in.close();
    }
}


Was This Post Helpful? 1
  • +
  • -

#8 cfoley  Icon User is offline

  • Cabbage
  • member icon

Reputation: 1940
  • View blog
  • Posts: 4,027
  • Joined: 11-December 07

Re: Java file binary read for statistical analysis

Posted 21 July 2011 - 03:22 AM

To be fair, you could alter that loop to put the bits in a BitSet instead of dumping them to the screen. Then you've got easy access to each bit through BitSet's methods.
Was This Post Helpful? 1
  • +
  • -

#9 g00se  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2652
  • View blog
  • Posts: 11,189
  • Joined: 20-September 08

Re: Java file binary read for statistical analysis

Posted 21 July 2011 - 03:46 AM

Yes, that's true (although getting them into the BitSet in the first place is unnecessarily difficult - see above) but that's just an example. Further use cases would be needed to provide a more reusable solution
Was This Post Helpful? 0
  • +
  • -

#10 g00se  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2652
  • View blog
  • Posts: 11,189
  • Joined: 20-September 08

Re: Java file binary read for statistical analysis

Posted 21 July 2011 - 04:57 AM

It's actually much easier to use a BigInteger to fill and hold a bitset. e.g. as you read the file, you can call the following (where 'bitSet' is a BigInteger initialized with "0"):


    
private void addToBitSet(int n) {
	bitSet = bitSet.shiftLeft(8);
	bitSet = bitSet.or(BigInteger.valueOf(n));
}


This post has been edited by g00se: 21 July 2011 - 04:58 AM

Was This Post Helpful? 0
  • +
  • -

#11 jsam84  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 4
  • Joined: 09-January 09

Re: Java file binary read for statistical analysis

Posted 02 August 2011 - 01:22 PM

With the release of Java 7 and the binary literals is there a way to read a file in binary mode into an array or some other mechanism of binary literals aka:

file read in binary(file){
       array of binary literals each one bit long
       for each bit of file read into array of binary literals
}



Thanks for the suggestions
Was This Post Helpful? 0
  • +
  • -

#12 pbl  Icon User is offline

  • There is nothing you can't do with a JTable
  • member icon

Reputation: 8324
  • View blog
  • Posts: 31,857
  • Joined: 06-March 08

Re: Java file binary read for statistical analysis

Posted 02 August 2011 - 05:29 PM

Just check the API of JRE 7
There is no method to read binary litteral from a FileInputStream just the regular

read() that returns a byte into an int

Don't see what is so complicated about the BitSet solution or the array of boolean that will be implemented as a BitSet by the compiler anyhow (or even an array of byte[] set to 0 or 1)

Anyhow using JRE 7 for production code in the next few months might not be a good idea
Was This Post Helpful? 0
  • +
  • -

#13 jsam84  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 4
  • Joined: 09-January 09

Re: Java file binary read for statistical analysis

Posted 03 August 2011 - 11:53 AM

View Postpbl, on 02 August 2011 - 06:29 PM, said:

Just check the API of JRE 7
There is no method to read binary litteral from a FileInputStream just the regular

read() that returns a byte into an int

Don't see what is so complicated about the BitSet solution or the array of boolean that will be implemented as a BitSet by the compiler anyhow (or even an array of byte[] set to 0 or 1)

Anyhow using JRE 7 for production code in the next few months might not be a good idea


I must be missing something with bitset because when I tried to load in values into a bitset instead of expected bool values I got 0, 17, 32, 0, 4.
I have looked at the pages on bitset and thought I had it correct but must be missing something.
Was This Post Helpful? 0
  • +
  • -

#14 g00se  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2652
  • View blog
  • Posts: 11,189
  • Joined: 20-September 08

Re: Java file binary read for statistical analysis

Posted 03 August 2011 - 12:03 PM

Quote

I must be missing something with bitset because when I tried to load in values into a bitset instead of expected bool values I got 0, 17, 32, 0, 4.


Please post code you used
Was This Post Helpful? 0
  • +
  • -

#15 pbl  Icon User is offline

  • There is nothing you can't do with a JTable
  • member icon

Reputation: 8324
  • View blog
  • Posts: 31,857
  • Joined: 06-March 08

Re: Java file binary read for statistical analysis

Posted 03 August 2011 - 02:24 PM

I would use an array of 8 masks (one for each bit), but g00se solution actually works


import java.io.*;
import java.util.*;
public class LoadBitSet extends BitSet {
	
    private final static int[] mask = {0x80, 0x40, 0x20, 0x10, 0x08, 0x04, 0x02, 0x01};
	LoadBitSet(String filename) {
		File file = new File(filename);
		int offset = 0;
		try {
			FileInputStream fis = new FileInputStream(file);
			int oneByte = fis.read();
			while(oneByte != -1) {
				for(int i = 0; i < mask.length; ++i) {
					if((oneByte & mask[i]) != 0)
						this.set(offset);
					++offset;
				}
				oneByte = fis.read();
			}
			fis.close();
		}
		catch(IOException e) {
		}
	}
	
	public static void main(String[] args) {
		BitSet bs = new LoadBitSet("LoadBitSet.java");
	}
}



Happy coding
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1