[kingAmin]

Hi! I'm working on two data files one has missing data, I'm interested in writing a python code to read the complete file put zeros where the other file has missing values.Can any one help me with a code? Thank you

[atraub]

Sure, we can help you, show us what you have so far.

[kingAmin]

Hi! I'm working on two data files one has missing data, I'm interested in writing a python code to read the complete file put zeros where the other file has missing values.Can any one help me with a code? Thank you

from numpy import*
fp = open("beth.txt","r")
fs = open("bftn.txt","r")
f = open("newdata.out","w")
a = []
b = []
c = []
n = 0
for line in fp:
    a.append( int(line.split()[0]))
    b.append( int(line.split()[1]))
    c.append( float(line.split()[2]))
    n += 1
fp.close()

eps = 1.0e-5
for line in fs:
  a1 = int(line.split()[0])
  b1 = int(line.split()[1])
  c1 = float(line.split()[2])
#  Test for match of all three numbers
  ok = 0
  for i in range(n):
    if a1 == a[i] and b1 == b[i]:
      if abs(c1 -c[i]) < eps:
        ok = 1
# If there is a match ok = 1, otherwise ok = 0
  if ok == 1:
    f.write("%d\t%d\t%f\n" %(a[i],b[i],c[i]))
  else:
    f.write("%d\t%d\t%s\n" %(a1,b1,"nan"))

fs.close()
f.close()

Edited by Dogstopper:

This post has been edited by atraub: 20 June 2012 - 07:50 PM

[atraub]

Posts merged. It's not necessary, or appropriate, to create a duplicate topic just to add more information.

Could you give us an example of the inputs and an example of the output?

This post has been edited by atraub: 21 June 2012 - 06:38 PM

[kingAmin]

Hi! I'm working on two data files one has missing data, I'm interested in writing a python code to read the complete file put zeros where the other file has missing values.Can any one help me with a code? This the code i have written

from numpy import*


fp = open("beth.txt","r")


fs = open("bftn.txt","r")


f = open("newdata.out","w")

a = []

b = []

c = []

n = 0

for line in fp:
    
    a.append( int(line.split()[0]))
    
    b.append( int(line.split()[1]))
    
    c.append( float(line.split()[2]))
    
    n += 1
fp.close()
for i in range(n):

 
    for line in fs:
    
        a1 = int(line.split()[0])
    
        b1 = int(line.split()[1])

    
    
        if a1 == a[i] or b1 == b[i]:

       
           f.write("%d\t%d\t%f\n" %(a[i],b[i],c[i]))

    
        else:
       
           f.write("%d\t%d\t%s\n" %(a1,b1,"nan"))



fs.close()

f.close()

and this is what i'm getting
 result.txt (606K)
Number of downloads: 21

[atraub]

Stop copying and pasting the same damn introduction. I was very clear:

1.) Show us what you would input to the program
2.) Show us what the program would then output.

If this request is beyond your abilities, then helping you is beyond mine.

This post has been edited by atraub: 21 June 2012 - 07:58 AM

[kingAmin]

FOR EXAMPLE THESE ARE THE TWO INPUT DATA FILES AND ONE OUTPUT DATA FILE

   
INPUT DATA FILES                                                OUTPUT DATA FILE

     A                                   B                         IWANT MAAKE "B" START FROM FIRST DAY AS "A"
 DAY NO. TIME IN HRS.  TEC/U     DAY NO. TIME IN HRS..  TEC/U      DAY NO. TIME IN HRS.  TEC/U
 1       1.0           4.5       2        1.00          5.3        1           1.0        NAN
 1       2.0           3.5       2        2.00          4.5        1           2.0        NAN
 1       23.59         5.6       2        3.0           4.2        1           23.59      NAN
 2       1.0           3.5       3        1.0           2.5        2           1.0        5.3
 2       2.0           2.5       3        2.0           3.9        2           2.0        4.5
 2       3.0           3.2       3        23.59         5.3        2           3.0        4.2
 3       1.0           1.5       4        1.0           9.4        3           1.0        2.5
 3       2.0           5.3       .          .             .        3           2.0        3.9
 3       23.59         7.2       .          .             .        3           23.59      5.3
 4       1.0           3.7       .          .             .        4           1.0        9.4
 4       2.0           6.2       29       23.59          3.5       4           2.0        NAN
 4       23.59         4.3                                         4           23.59      NAN
 .       .              .                                          .             .         .
 .       .              .                                          .             .         .
 .       .              .                                          .             .         .
 29       23.59        4.2                                         29           23.59     3.5
 30      1.0           6.2                                         30           1.0       NAN

This post has been edited by atraub: 21 June 2012 - 10:00 AM
Reason for edit:: added code tags to try to improve readability

[atraub]

looking at your example, I can't figure out what you're looking for.

[kingAmin]

FOR EXAMPLE THESE ARE THE TWO INPUT DATA FILES AND ONE OUTPUT DATA FILE

[/ INPUT DATA FILE

A
DAY NO. TIME IN HRS. TEC/U
1 1.0 4.5
1 2.0 3.5
1 23.59 5.6
2 1.0 3.5
2 2.0 2.5
2 3.0 3.2
3 1.0 1.5
3 2.0 5.3
3 23.59 7.2
4 1.0 3.7
4 2.0 6.2
4 23.59 4.3
. . .
. . .
. . .
29 23.59 4.2
30 1.0 6.2]





[/ INPUT DATA FILE
B
DAY NO. TIME IN HRS.. TEC/U
2 1.00 5.3
2 2.00 4.5
2 3.0 4.2
3 1.0 2.5
3 2.0 3.9
3 23.59 5.3
4 1.0 9.4
. . .
. .
. . .
29 23.59 3.5]

[/ OUTPUT DATA FILE

DAY NO. TIME IN HRS. TEC/U
1 1.0 NAN
1 2.0 NAN
1 23.59 NAN
2 1.0 5.3
2 2.0 4.5
2 3.0 4.2
3 1.0 2.5
3 2.0 3.9
3 23.59 5.3
4 1.0 9.4
4 2.0 NAN
4 23.59 NAN
. . .
. . .
. . .
29 23.59 3.5
30 1.0 NAN]

[baavgai]

You do not want to screw with files. Read both entire files into memory. Use readlines, it will get you a list. Write a parse function for the line. You have three pieces of data. However, the first two form a key. You can use that key to compare both data sets and put them in order. A dictionary would help here.

Write a parse function that returns either valid line data or None.

[kingAmin]

I DID SOMETHING LIKE THIS

fp = open("beth.txt","r")


fs = open("bftn.txt","r")


f = open("newdata.out","w")

a = []

b = []

c = []

n = 0

for line in fp:

a.append( int(line.split()[0]))

b.append( int(line.split()[1]))

c.append( float(line.split()[2]))

n += 1
fp.close()
for i in range(n):


for line in fs:

a1 = int(line.split()[0])

b1 = int(line.split()[1])



if a1 == a[i] or b1 == b[i]:


f.write("%d\t%d\t%f\n" %(a[i],b[i],c[i]))


else:

f.write("%d\t%d\t%s\n" %(a1,b1,"nan"))



fs.close()

f.close()

BELOW IS A PORTION OF THE RESULT I'M GETTING. WHICH IS PICKING ONLY ONE VALUE FROM THE THIRD COLUMN

233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
233 4 5.280000
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan
234 0 nan

This post has been edited by atraub: 21 June 2012 - 12:16 PM

[atraub]

1.)
2.) Stop writing in caps
3.) Was that a question?

This post has been edited by atraub: 21 June 2012 - 12:17 PM

[baavgai]

So... a, b, and c are parallel arrays? And you're casing values like 23.59 into int? Doing a split three times, never doing a validation check? The n variable is meaningless. Without indents, the code is crap. Your methodology is unfortunate.

Your second pass considers the second file, but ignores the first.

# pointless
# for i in range(n):
# might as well just say
i = 0
for line in fs:
	# when this loop is finished, you're out of file

Read them both first, then mess with them. Use functions. The same code should be used to read both files. You needn't do any casting to fit the files togehter.

Good luck.