Faster way to execute my loop

  • (4 Pages)
  • +
  • 1
  • 2
  • 3
  • Last »

46 Replies - 1768 Views - Last Post: 28 June 2013 - 10:32 AM Rate Topic: -----

#1 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Faster way to execute my loop

Posted 24 June 2013 - 12:13 PM

Hello everyone, I have a problem similar to one I worked on a few weeks ago where I need to compare some genomic data in very large input files. We're looking at the allele present at a number of loci on the genome of a chicken, with three possible alleles for each locus, represented by a 0, 1, or 2.

Currently, I'm reading in the entire sequence of allele ids as one long string, ~33,400 characters. Next, I look at the value for two birds at a particular locus and if one is a 2 and the other is a 0, I'm adding +1 to a counter. No other combination of 0, 1, or 2 gets counted.

I'm accomplishing that by looping through my 33,400 character string character by character and evaluating each locus pair for every possible combination of birds in my data file. The loop looks like this:

        For i As Integer = 0 To AnimalCount - 2
            For j = (i + 1) To AnimalCount - 1
                For k As Integer = 0 To GeneCount - 1
                    strCurrent = aryGenome(i)
                    strContrast = aryGenome(j)
                    If Math.Abs(CInt(AscW(strCurrent(k))) - CInt(AscW(strContrast(k)))) > 1 Then intDiffCount += 1
                Next

                aryResults.Add(intDiffCount.ToString())
                intDiffCount = 0
            Next
        Next



To speed up the process, I'm splitting the data file into 8 pieces and running the above loop in 8 different tasks, then adding up the results at the end. I can process one of our data files with 5,151 records (13,263,825) in about 48 minutes (13 minutes on my gaming PC at home), but I'm thinking there's probably plenty of opportunity to improve on that time.

When I was asking about the related problem before, there were a lot of suggestions revolving around my datatypes and how I'm representing the data, but I found that it didn't seem to matter how I represented the data; I couldn't get any better performance doing bit-wise compares or anything else. It could be a just did it wrong or misunderstood the suggestion.

Anyway, that's my question: how can I get the loop above to execute faster?

Is This A Good Question/Topic? 0
  • +

Replies To: Faster way to execute my loop

#2 Martyr2  Icon User is online

  • Programming Theoretician
  • member icon

Reputation: 4424
  • View blog
  • Posts: 12,293
  • Joined: 18-April 07

Re: Faster way to execute my loop

Posted 24 June 2013 - 12:53 PM

You can cut this down by applying a filter. If I am understanding you correctly this should work and save you a bit of time. You said that the only combination you are looking for is when the first value is 2 and the second value is 0. No other combination. So don't bother with comparing each value (k). Loop through strCurrent looking for a 2 and only when one is found do you check K in strContrast. And even then you will check if strContrast(k) is a zero because if is not, no use counting it.

Let's run through an example using something simple... I have two sequences and the only one I am worried about is when the first one is "2" and the second is "0". No other combination is counted. I have the two sequences below. The first is animal 1 and the second is animal 2.

{1, 1, 0, 2, 1, 0, 2}
{1, 0, 1, 0, 1, 1, 2}



Right now you are taking animal 1 and animal 2 and looping through K one by one looking for a difference between the two being greater than 1. That is fine, but why bother comparing 1 and 1, 1 and 0, 0 and 1 etc?

Loop through animal 1 and when you find a 2, you will know what "k" is so you can check animal 2 at location "k" to see if it is zero. Not the difference of, just check the value. This is going to save you conversions, save you an operation and save you from having to do comparisons of difference at each element.

So with our example we loop through animal one and land on a "2" at location index 3 (assuming we start at zero). Is index 3 of animal 2 a zero? Yes, count it. Continue with animal 1. we get to the second "2" at index 6. Is animal 2, at index 6, a zero? no, don't count it.

Hope you get that idea. Now, an additional optimization would be to move the lines strCurrent = aryGenome(i) and strContrast = aryGenome(j) out of the for loop for k. Move them to before the loop...

For i As Integer = 0 To AnimalCount - 2
    For j = (i + 1) To AnimalCount - 1

        ' Move them here, this prevents them being set each time through k when they don't change.
        strCurrent = aryGenome(i)
        strContrast = aryGenome(j)

        For k As Integer = 0 To GeneCount - 1
            If Math.Abs(CInt(AscW(strCurrent(k))) - CInt(AscW(strContrast(k)))) > 1 Then intDiffCount += 1
        Next

        aryResults.Add(intDiffCount.ToString())
        intDiffCount = 0
    Next
Next



As my comment above suggests, but moving these two comments out of the for loop for "k" you avoid the operation of setting the values strCurrent and strContrast each time. Those values only change when "i" and "j" change. No use resetting them. That will also save you two assignment operations.

Another different optimization would be that while looping through strContrast that you don't even find a 2 (or a zero), kick the whole animal out and set strCurrent to the one after strContrast. No use processing an animal if none of the values are 2 or 0.

Hopefully I gave you some food for thought. The only way you are really going to save time is looking for ways to filter animals as you go along. You might even be able to save the filtered list later and use it to apply to other comparisons.

:)

P.S. CInt and AscW are old school VB6 syntax. Might want to update that to VB.NET. Just a side note.

This post has been edited by Martyr2: 24 June 2013 - 12:59 PM

Was This Post Helpful? 1
  • +
  • -

#3 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Re: Faster way to execute my loop

Posted 24 June 2013 - 01:14 PM

First off: thanks for your detailed and indepth look at my problem. I really appreciate your analysis.

View PostMartyr2, on 24 June 2013 - 07:53 PM, said:

Loop through animal 1 and when you find a 2, you will know what "k" is so you can check animal 2 at location "k" to see if it is zero. Not the difference of, just check the value. This is going to save you conversions, save you an operation and save you from having to do comparisons of difference at each element.

So with our example we loop through animal one and land on a "2" at location index 3 (assuming we start at zero). Is index 3 of animal 2 a zero? Yes, count it. Continue with animal 1. we get to the second "2" at index 6. Is animal 2, at index 6, a zero? no, don't count it.


This seems like a great idea on the surface, however I feel like I'm going to end up losing time doing more comparisons this way than I would just comparing once per character. First, I'll have to check if the current character is a 0 or a 2 (not just a two, because 0,2 and 2,0 are both valid differences), and if it is, I have to check and see if the same index in the contrast animal is a 0 or a 2. Overhead-wise, wouldn't just making one decision be faster than making two?

I'll test it and let you know.

Quote

Another different optimization would be that while looping through strContrast that you don't even find a 2 (or a zero), kick the whole animal out and set strCurrent to the one after strContrast. No use processing an animal if none of the values are 2 or 0.


This will never happen with my data; remember that we're looking at 33,400 loci, each of which can only have a value of 0, 1, or 2. Good thought, though for a less complex dataset.

Quote

As my comment above suggests, but moving these two comments out of the for loop for "k" you avoid the operation of setting the values strCurrent and strContrast each time. Those values only change when "i" and "j" change. No use resetting them. That will also save you two assignment operations.


I can't believe I somehow missed having my value assignment for strCurrent and strContrast in the wrong loop! I wonder if the optimizations in the compiler fixed that for me. I'll fix it and see.
Was This Post Helpful? 0
  • +
  • -

#4 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Re: Faster way to execute my loop

Posted 24 June 2013 - 01:33 PM

Just moving the unnecessary assignment of my two variables out of the inner loop bought a performance improvement of 34.4% (from 6.8 µs per animal pair to 2.3 µs per animal pair). My god, man. I can't wait to see how fast this goes at home on my burly computer. Let me test your filtering idea, now.

This post has been edited by C.Andrews: 25 June 2013 - 06:45 AM

Was This Post Helpful? 0
  • +
  • -

#5 lar3ry  Icon User is offline

  • Coding Geezer
  • member icon

Reputation: 310
  • View blog
  • Posts: 1,290
  • Joined: 12-September 12

Re: Faster way to execute my loop

Posted 24 June 2013 - 01:49 PM

View PostC.Andrews, on 24 June 2013 - 02:33 PM, said:

Just moving the unnecessary assignment of my two variables out of the inner loop bought a performance improvement of 34.4% (from 68 ns per animal pair to 23 ns per animal pair). My god, man. I can't wait to see how fast this goes at home on my burly computer. Let me test your filtering idea, now.

Would it be faster to use .IndexOf() to find a '2' value, then check the other string for the presence of a 0? I'm wondering if the .IndexOf would be faster than a loop looking for a 2. Each time you find a 2 and check it, you would do the next .IndexOf() using a Substring specifying the rest of the string. If you are looking for either 2,0 or 0,2 you could do each pair twice, once looking for 2s in animal1 and once looking in animal2. If I had a few examples of your strings, I'd give it a test.
Was This Post Helpful? 0
  • +
  • -

#6 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Re: Faster way to execute my loop

Posted 24 June 2013 - 02:01 PM

View Postlar3ry, on 24 June 2013 - 08:49 PM, said:

Would it be faster to use .IndexOf() to find a '2' value, then check the other string for the presence of a 0? I'm wondering if the .IndexOf would be faster than a loop looking for a 2. Each time you find a 2 and check it, you would do the next .IndexOf() using a Substring specifying the rest of the string. If you are looking for either 2,0 or 0,2 you could do each pair twice, once looking for 2s in animal1 and once looking in animal2. If I had a few examples of your strings, I'd give it a test.


Here is a test file I'm using with just 50 records in it instead of the full 5,151. The first set of numbers is the animal ID, the second set is the genome. Enjoy!

Attached File(s)


Was This Post Helpful? 0
  • +
  • -

#7 Martyr2  Icon User is online

  • Programming Theoretician
  • member icon

Reputation: 4424
  • View blog
  • Posts: 12,293
  • Joined: 18-April 07

Re: Faster way to execute my loop

Posted 24 June 2013 - 03:16 PM

Oh ok, so 2 and 0 and also 0 and 2. The first optimization should still help if you just compare the characters without converting to ints and comparing the difference. Why convert the char "2" to an int, then "0" to an int, subtract and compare when you can just do...

if (strCurrent(k) = "2"c and strContrast(k) = "0"c) or (strCurrent(k) = "0"c and strContrast(k) = "2"c ) then
   ' Increment count
end if



The boolean logic here should be faster than the two AscW calls, the two CInt conversions, the subtraction operation, the Math.abs call and comparison to 1. At least I would think. I couldn't say for sure. We are removing the overhead of function calls and conversions.

:)

This post has been edited by Martyr2: 24 June 2013 - 03:25 PM
Reason for edit:: Corrected Char literals

Was This Post Helpful? 0
  • +
  • -

#8 lar3ry  Icon User is offline

  • Coding Geezer
  • member icon

Reputation: 310
  • View blog
  • Posts: 1,290
  • Joined: 12-September 12

Re: Faster way to execute my loop

Posted 24 June 2013 - 06:06 PM

View PostMartyr2, on 24 June 2013 - 04:16 PM, said:

if (strCurrent(k) = "2"c and strContrast(k) = "0"c) or (strCurrent(k) = "0"c and strContrast(k) = "2"c ) then
   ' Increment count
end if


Perhaps:

if (strCurrent(k) = "2"c AndAlso strContrast(k) = "0"c) or (strCurrent(k) = "0"c AndAlso strContrast(k) = "2"c ) then
   ' Increment count
end if


might reduce the time by a little, if the first conditions are not met.

Oh! And here's something else I tried.

        Dim x As String = "1202010"
        Dim y As String = "1022001"
        For i = 0 To 6
            Debug.Print(x.Substring(i, 1) Xor y.Substring(i, 1))
        Next


Was This Post Helpful? 0
  • +
  • -

#9 AdamSpeight2008  Icon User is offline

  • MrCupOfT
  • member icon


Reputation: 2271
  • View blog
  • Posts: 9,499
  • Joined: 29-May 08

Re: Faster way to execute my loop

Posted 24 June 2013 - 06:39 PM

I wrote this one using PLINQ don't know if it produce the desire result or any faster.


Dim r1 = seq1.AsParallel.Select(Function(g,i) If g="0"c OrElse g="2"c Then
                           Return New Tuple(Of Char,Integer)(g,i) 
                         Else
                           Return Nothing).Where(Function(g) g IsNot Nothing)

Dim r2 = seq2.AsParallel.Select(Function(g,i) If g="0"c OrElse g="2"c Then
                           Return New Tuple(Of Char,Integer)(g,i) 
                         Else
                           Return Nothing).Where(Function(g) g IsNot Nothing)
Dim r3 = From s1 In r1.AsParallel()
         From s2 in r2.AsParallel()
         Where s1.Item2 = s2.Item2
Dim c  = r3.AsParallel.Count



Was This Post Helpful? 0
  • +
  • -

#10 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Re: Faster way to execute my loop

Posted 25 June 2013 - 06:15 AM

View PostMartyr2, on 24 June 2013 - 10:16 PM, said:

Oh ok, so 2 and 0 and also 0 and 2. The first optimization should still help if you just compare the characters without converting to ints and comparing the difference. Why convert the char "2" to an int, then "0" to an int, subtract and compare when you can just do...

if (strCurrent(k) = "2"c and strContrast(k) = "0"c) or (strCurrent(k) = "0"c and strContrast(k) = "2"c ) then
   ' Increment count
end if



The boolean logic here should be faster than the two AscW calls, the two CInt conversions, the subtraction operation, the Math.abs call and comparison to 1. At least I would think. I couldn't say for sure. We are removing the overhead of function calls and conversions.

:)/>/>



You're right, this saves a lot of overhead and executes quite a bit faster. Using my mathematical method I get about 5.25 µs per pair of birds; with your boolean method I'm getting 3.43 µs per pair.

Not bad at all.

Edit: Corrected my math

This post has been edited by C.Andrews: 25 June 2013 - 06:44 AM

Was This Post Helpful? 0
  • +
  • -

#11 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Re: Faster way to execute my loop

Posted 25 June 2013 - 06:21 AM

View Postlar3ry, on 25 June 2013 - 01:06 AM, said:

Perhaps:

if (strCurrent(k) = "2"c AndAlso strContrast(k) = "0"c) or (strCurrent(k) = "0"c AndAlso strContrast(k) = "2"c ) then
   ' Increment count
end if


might reduce the time by a little, if the first conditions are not met.



I tried this and it actually comes in slightly slower than the original mathematical method, at 5.85 µs seconds.

Edit: Corrected my math

This post has been edited by C.Andrews: 25 June 2013 - 06:43 AM

Was This Post Helpful? 0
  • +
  • -

#12 dbasnett  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 110
  • View blog
  • Posts: 609
  • Joined: 01-October 08

Re: Faster way to execute my loop

Posted 25 June 2013 - 06:53 AM

If these files are simple ascii strings of 0,1,2 then read the file as bytes into byte arrays and change your comparisons. So if the strCurrent and strContrast are byte arrays then your comparison would be:

        Const zero As Byte = 48 'ascii 0
        Const one As Byte = 49 'ascii 1
        Const two As Byte = 50 'ascii 2



        'the comparison
        If (strCurrent(k) = two AndAlso strContrast(k) = zero) OrElse (strCurrent(k) = zero AndAlso strContrast(k) = two) Then
            ' Increment count
        End If




Not working with strings should help some.
Read File as Byte Array

This post has been edited by dbasnett: 25 June 2013 - 06:57 AM

Was This Post Helpful? 0
  • +
  • -

#13 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Re: Faster way to execute my loop

Posted 25 June 2013 - 06:57 AM

View Postdbasnett, on 25 June 2013 - 01:53 PM, said:

If these files are simple ascii strings of 0,1,2 then read the file as bytes into byte arrays and change your comparisons. So if the strCurrent and strContrast are byte arrays then your comparison would be:

        Const zero As Byte = 48 'ascii 0
        Const one As Byte = 49 'ascii 1
        Const two As Byte = 50 'ascii 2



        'the comparison
        If (strCurrent(k) = two AndAlso strContrast(k) = zero) OrElse (strCurrent(k) = zero AndAlso strContrast(k) = two) Then
            ' Increment count
        End If




Not working with strings should help some.



This is one of the things I tried with my original project (suggested by many people), but it turned out to have no effect on performance. I think maybe the compiler is doing some kind of optimization that equates to this, but that's just wild supposition on my part.
Was This Post Helpful? 0
  • +
  • -

#14 dbasnett  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 110
  • View blog
  • Posts: 609
  • Joined: 01-October 08

Re: Faster way to execute my loop

Posted 25 June 2013 - 07:12 AM

I kludged together this test and there is a savings, but maybe not in your case.


    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Const tries As Integer = 100
        Dim stpw As New Stopwatch
        Dim s As String = "2100212000000122110001111000200220212020200112201121000000122202211222102202101111110220020200000100201021100011000121111110021111002222202212121111202211112120002111210101122101102022111021011121101111120102011112010011202022021010100201000211010210112121010102000221020102010121020202111112001102011100110021212000111211111110111010201022002021102100100202120200012111101122020202002001102200202110120121112001011102002021111111021110100220000200020022020200210010021111112110111111111111121111121102121012210220202220111200222000020220020202222000211011110011110200222200222202021202201221211212220011111121110111201000101010011111111100110012011121110211121210122110020110220111101112121112010111001200120111102011122111111102222202202002220220020020220220200020222202202222020000020002022020020202200220000200210111111022012221000220201001111111211111211120110021001201011222111000110021102102110011022221111012111112002011111110101120220111111112022220022220000002021111111110011110111012000010111102201122121101110111211200100211122222222202202002000220202000002020200000222002002201211111211202222000111111120211110202022022220200200220011120111120221111210111111120111111111111111111110221022001101111211201111112101111101111111111112111111111120111111211102220022000220202211111011002211102112211211111012111111111110221111111111211020222101111220112001120101112200222021110001112022112110220111122220011102200211022111101120002221122120112210111221021012211010011100111001211210121111111021211000122122111110102110111210110112221011111201111121110011202111111111022020202201202200211120022002001102001200021012111121101120220202000220202210121211110022211221021221121011120212111210211011111111111011111112210111100010112000220202200200000200000222222020002002222022022200022020022002220022000020000020200222220202002211112111110121211012111110121120110022210111111210201220012220002200111111111111111120110021001111100112000122200110002022021120112110200020002011000111110221200112011122120012001101011101011112110211211111112200012110211220111021111111111222121210101012001000011211021100112101011111011110101212101211100112111102210111211102010112111101012001110102112112120101121111201112121102001212220101022020112010111021021111110111111111010001222000202202222002020000212021211111111101011111110110121102111111011011001020111000022202022001221100211012211210112111120110212011211111111121102120011002111210112111121111101121012001010101111102220020222021020210201121111122202211111221011021111101022012011212000020020111110021110020011001122201211110100111000020211002111112220120002200202011111121011102212111111211102111000111102111200000020222002022002202200022022220000211012111221010121111011000111021010202002020000220000002220022222200222020222200200002222200020020220020202202020120221112122002200120112211100211112011102111120121111101211112011111121011102110012111111111022011111111021111012111101000200022200002222000220200020020200020220200202220020222002202222000222020200200200002200200020222222002220002222002000000022022000022002202002002020200202202211111122222002022220202222222222200000222220022000202200022000202002000202020001121021111120221121001121011011212111111111211111002111202011122211111111112020220020022000202011202110220020000002020020200220200021011111111110111212201110110202111101221102111001210212112110211012111122010201112112012121120211210121221210101211011200102111121211111111110101112022020200200022220012022002101101100101100112101112102011221202101101021101020122111111011111111100112111011010112211021102122112220202121212020210111111111111111111112012111121111111111211101022002212001111211102222220222002000002200200211211211121122001101202111101111111111002111111022201110121121112000221112001110011020020111212211201110121121101111120102202111112002121101010220112211202110021222022011111110022100212120011111112111012111111100220211001121112022020201102102111112111110011122111012111211122111121012122011121110012121101012212112110111111111111111121121111102211212111022000222201111210111011002011111000002111111111120000200222022002020222002000002022000002200020020200220000002000022220220020222202022220202201211000202110112101111220020211111121121111121211102210211111211211111011211211012111111121001212011111211000112220111010010221110211221102111111112020200112102001200120111112202012020120110100102011102112202000221022111111000222211111112210111111022202020002222200220202200022022002020002220202200202202222000200002020002200202020222202002020020202202200020202220222002001221102202222020221111112012120002112101221100000210202011122121101110211101011100101110200221002112212102120111211111011000002222202022000200020220022200220020220202220000200022020002222220022202011112111011101110220200002201011101111011112012211121220211220202110020211122212011211221101111122110000211101022112002101001112122111101121110201202222110111101100100110002212102200001101102111021100211101111000011111111111022011220112221010111011120010200021111111011220020112111110202222202200220200211110111111112000020200200220020220222202200202112111121112121111110110202202022202112102100010111210112021110012010221212220100202100010021211112112112201011001112122012120112110112221210120120100210211101102011012011210102112100101100011021011101111011101211202121111120111120111212012112111111111001110111111111101001211112111211102212020121001111110111200200220202002002020200000020021111110222200222000112112111111111011210222122010111102210101111100112100211112112111202122111022112122111101111110221111111120220202002020020002220202220200000202022022202222002002200020220202202020022000010111111121010201121102121221121112011111112112210022220200221021112121012201212202020010122021202121010120122200222000222220222020202020002000222002002020220200201120110111102000000202011011220111212101010211100000111011100210110212121010101101111011111212202220000002002102211111202222222202200200201111201101011111012201122021201122200220022002022202200220211111110000202002020010102111111110211101102210112111011111111101111111110121002121112112112011101112102210010010102111112211121112001112011011112212002220202200222202022202220000101111101011102200020201111110112212211210220002200002202202002200202022202000220000020220022020022002000020022200220000200200022002002022022202022200000002200220000200022200220022022220202202200200202020022020220221200101012111002120011201110201002201111101210120011100002110022021111111111021111022212122020212101120100112120112012011110111102101022111110112122111121002200220002020020222202020111101111111011212121111111110010211112011101210121112011111110112120112210200120000200200000002020011111111011111102022121011021111022110210111100210211111111111111111111111111112200220002220000222020020200020202222200220002020202002202222220002000200202220002022002222211011001111110101001011111111102121010111021020120202110100022000000001111121122222220002020220220200112111111110002020211020021100202220222200222000022010201200011111000001111111112100002022211101121012111111122002000202021211101121210011112112101000111222112211112211121121011222111221100011111111020200022020002201121110101221112011112021022001112222111111100122202021012011100111101020202002221212020210101001121212111211020210102211101100210202020002200200200020212121220012212111121112112022011210200010011201101002220200222202222220002022020002110010012110101111212020220000000020002222020202222222212202221112110101222202111110110001011111212022200011202011121111121112100100111100110110011211110111110121000111111111111011022002220202212121220210211112101112200002002220211011021110201011010111211120210211021102200210111101211121102201011220000002222021111111110222220220022002202020001122220002211001211020112011110111200020211101011121100211010111121020112111121111110110011112211012001201201101121120200110111211200111222212011212120012022200211111211112012110021001220221101112111111111120121222001021120121111011222102111111121121200212110010111000002121110111100110202101202111211121222121011111022120102002211200001011202102111122120011111121100221120122102200022002001101021102212021111111111201211011101100220122111111111011221112012011000202020202200200022020002002022200200020200202000200220220022200202220000222020002002202000220200002202000202202022200020220000222220220222202000020002002220002002022222200220022220011100111002112111011020222222020020020020220002000002202200022202022222222000022200220200021111201011111101222200211122011012111110201211100112111001120222022200012110101112012120020202010020221121110101101112111111010021111112201211111110110211112210012210011211111011222110111011210111111211110111010011110011110120102120212111002011010012221020011112101221111112211100000110022121110202002111121102201012201111211110110010120100020011002010010202011111110111111011011121010211101010022200220000000222011110112111202000000011111112200211121121111011111111121112111200211022010011121000222022002202002020000022222000020200210211112111111010020011111211000110111111201111211210111211011211111111101211011111111110020200202022002000200222200002002202022000020220000002011021111111211101111102112011101120211221111011110101020110212111201011011120011120000202200222222222022000200020202000022202022202020002201111001110001212100222212102221111111020121111021012001112021110212102100211211221001021102120020020200022000020202020002222201121112111111011012110220011221020011101111101111221111112122011220212021111021200221011002111112021121011102000220020020022002220020220020022220200020220200000220200000002200222000220002002002002000222022200002200202022002022002202220020200220200000220202220202022020222022220022020022002202202000222202220202022200022200002222002222020220200110100200221112111120121101110200222002220202012111202200200220220022022222020222002202020220200111100100101210221000111110011120110111211102111110220020002211121102111000000220021121212111101212210101210102112112111011121221101112122000020220002002222022002200022202200202020220222000020220100011221111011211101110111122022020200121202001121110002201122101101100101121111110121210111122022220002220022020220020002002000202222022222200022222022220220222220022022101011121111111110121122001111200211110101201021211022021022211020010111211010011112002200222220001111121121110111021110211120210100121111110112110112111110002211111120202020201011212111111111111121111002212220202020200220220222020200020220002022002202222022222201100112111011210110000002022200020220000222022222000200020200200222222220111111011111111111021111201011111111110200222111202101211110201210110011102020221110110100200201122011011110101200202202002202002202020200020020220022210021012112222110222000222020222000200020002022202202000220020002202200022022201111111121111110112111111112012122121121122121101101111111112111211111100111111211111110210101011010111111020022111100111211122111110211101111111110011112222112101210111111111111111110110101200202200200020220200011010111120222111111111120111101121110110110020221011110112112211010121002122012111212101121211112111011011111011112210111220111011220110111211102200201020010021112110002211001011102222002221120212202201212021111002022221110020110201121010110101221012121012200202111101212111010221122121201201101222002110112211221011110202220001112011100111111202111100111121012111120010111100110101111111001100201110202210102111011120002200111112202211211120220220000000202220222002220222222000220022002202000220000020022000220002000220000200022020222202200220200000022020002202002222220022202220020220200202020200022202202220222020000222021101222011122211110011110211101011110112010112010121222020000002222202000110110021112020112111111111112100011102110201111020012101121110000211100102111110200202021101021111022211211001021111112101020202222220000002220222222200222002202200222220000022000220020020022220200222021201100220200202201121020111110022200120111120211110022112000111102211111110010202201220200222121201011112021110211202220121021102200022010102002000022202020020002222020022002002011012212202120012011111201221000111212122101102121120211101012000202101211122101101212122211212111022112002011121100110102101112212112010110212111020002011101112111020012111210201112210112212022002202022200020002220202202200002020022022022020020220022200022002202200200000200202020201112121201221201111211110111010221120212020001011111110020201202221200211100121002200202200000200022002211111111211111111211111102201110110001111111121211021101102120211102120210210222120111121122012001110111010102122220111111110110020002221111112012020112211201000102020122100112101011112111101120110102121011121022002110021110020012111012020201111102001010202022002000022000202002101012111120022102111012101220000000222021120120121210101110022111012111122102111102010011121102200021122011011201110122110101100212002020222222222002122011002020210122011111111021111111201202211121111111111211222112201011101122220020022000020002220020011111011011122022222010220112102200022202020201010200200200220002020020202200200020012000220200220220000202221112111110110111010110121120001110112112100111210020001112110111221110010112120110201222220222222011111201020211020200102112022121111112000220001122211211021011112110002020121122001121111222111200200202010120112101122102201201110012200011110121121111110110220120222200200020202202202220002002000202002200020222100002021220122020020202001111102121010100011101200200021220202200222002022221010211120111121011011111222011122020022220200020002222220220022200022202200200222222111100002012201202012211221110211001120111111011111211000222022222222022000020020020220020202220000200022111112012022110110200110122200210111211202220022021111021101021011020021120220101211010112021011121200210211001110220122021010220021010010220221011012112121112210212110100022211001201022011001201111210110221011221200210011111200202210211001010221211102022210110200120221210221112011211112102101021011122201011112102102211111101000011111101202012012020002200200000200002000200002121111122110011110200121102000111100222202200200022001001111011111021122200012102202121211002111011210021111112211110002020000022022020222222202202022200020200022000002002020011111202122011211201220200200200222020022000000002020022220022202220222202020222000202022020202002002220222022220000220222220022222000000220220020211211111011022022112121120020022222202020022220202000020220011211112201101122111101111111111111111020122020120211010010010222101111111221112022201211111112111210111110111100111101100221011111111110012022111121110000212220021120222201122110111112202111112101220011121011120101111202220101202002201101011201121202111111211111011102111221110100001021111110121121101101020211221210111110121212002110102011111002120102220000111101022011212211221100022002001222100202002220222000022200202002020220222112111120101121111010112110111121201210121102120101110021100002022020202000200220220202202222002020002202200220102011110111211112121120212110102111101211110011102020120012011101110111002012211111111212101002110211112002000111211111111120200222020002020222220211112212001120200020110101201210110112211200222111201111112101122020200220022020020010221101222200201200110110220020221112121100201102201121112111121120011011101211010221221211011221011111102001020001201122100012010221020120022022020000220000220022202020020222200202202220002222021011111101112012102020011110101111001110220102001002021022212120210200002002202220222222220011102211112001111100110222111110111112010101211110011210211121010222102101210111111122211010220210121111202011201211112200020000022220202012010120020200011202122001121020220001101111111221002010011002000022002222222222220112101011211210110111100111122101112101120011220122111100220202010101021120111111022112222001011111101202101201020111011012121112121110111200100011000110121111111111011111011111112120221121221111111111111012111111222020001010102111101102102211211211012201121101111011221220221111021022010101011112211111211111021012201012112111212111011011102100011211100002200022222010211002112111100120101011120221212111122201101021212102211210012111122100102110222202111111110022022200020002220020020020212220202001012112021111011121112110102111011111120221212010212011100020121010102211101100111200220011220002111012001021210121200110121221012110200022111120111101211000002222020000222000020202011011200110201102201010100201120122110110101012010122110221111100100111021110121110112101201121111111010102022110011120222022222002201212112111122110110122002211111111011022010012111111112201011112011111111120111121011121111111101111111011201111111121211111201112101011111211122011111001222020002200221121111111211211121110211210210201011221012111111000200002200200020200020222220020202200222020202022022002020220222200000200002200202222220222200202000200202000200022200002000022002200202020000000202202200202000020022202002200200200020000022000202022200202020000220000020020002202020220202202121112211120120201011010212112111011121010201212120000201211010121001121121111010101111111111111002010222112111022010111102111210211210221122120022002002202022112202121122101212011111110111222110020222022202112112111111112201022120102210100011112101112020020202002002020201101120212110212220122220010220212100110210210121122110110111212111221110100200002002002002022000002002200101011222210222212101111211112110111121110210112100211020110122221111111111012112102101102200222112100121011001110011100101112110111101011110100201111110022000220002000202202002000222002220022220220011002110111211222020022000220022020200022022220101120112101021012111111120112020110122201000101222101210020011111201202211202020220021111111111111111111111111110111111111011011022011111111111011112102102120111111002022200002020202202212101101111020122111220001111111111101222000022200200020020022220000220000200202000002022202002000202020020001121011202200020022200202202200222022002020202222020202200211121021101221111201011120212211121110110202222211111100121202021211011121111101121220010001101010222111211121112101101220001111122112010221111111112202110111111001211112021111112111200110111111111211101111221000110101110202222020202020011022222011010201101011000202010001112010202022202112200112002202000202000211111111111120211111111221111222011110110112101111121210110210222202022022220220221012101111210220111011010212210110201020221111022111100221210111002222012111101222201220110201121100111011201001021011002220200020222022222200200002202220202200202222002022022002200202222002220202021002002022002002211012020020121222020100102000122111110202121000122111020120010111122021011100002222121011210022111110200222022220111111000202222002202022222220220020202211101201212111202201112021111021111011121002102000021110220112010200111211102122111201011121022120102110111100101100020011121211101111021211021111112011212111111111111000221011111101011120111101102101111111111022020220211221102202201010011011211211011101210122102202211012010120010100220111222001111211202120010201200100011221112012220211121020010121101002110121101110022202022011112010100011121100011111001021111021111020100100101212101111201111211111112121111020111010211112112112010111120110020120120011111111210110200112001021112011212022002112101022120222110111111001020110011111111112100112110101210101021101121110121221121111121111001110011111011111211111111112120001111112111111111102202221122011111111022100011111111102002022002222200222000202022002220022200111211111221011111101111000121020202111112020111200122111010101022102201021110011220002220220022202222110201110022112002012202111102111000002111200021022112101122022121101112011120121111122111110111111122221201002212110121200211111001110111020111111221111111111020201011110002000212101211211101020210011212100102201012211110210020112100021021101011211221201112111111011012111111101121111012010122101121010121120002011010202202002020022022020000002220200202000002202112211021111002201210111122011210012010010111111011111201010200011101211111011110102012101121112110022111021111021121101112111210012122212000202020102111110211111011120220101002022111120200220020220220020002022220020001111010012201211110021122121011111201100011202000222022011210110001111212010221211112211211121011111112102220020022020001110011011100212111111111200000020200200022022000001111000200001001010111211102021201111120011002212211020111022211211102102022111101021011110101202112111002020202002000000002220200202202022002022000202020002200202002222002220220200202222020020220222211002211111110122110100112011120110201111110200011202222002012200101010122101022002211021121011101112111111110111101110111121001110111111002020022001221201021111120021102212111112120022111001020110000002202222000000021200220220222000020002220020122100212202221010222111110211112200210121100022022020202000222011210120111211011211111101111022210112101111010122111110111110111110221001101110101110211002201021100201022200011211221021112200111211101022122101211011220020222000020220200000002200222020002022222221220101022100210212110210111010110221110021101110220022222002200002022000020222202220222022220220021101121112200100112120121001122101112110100102111111111111111111111111121010110210011200122121200010111211120021111121111001011111111111021111111111010211102112111121120100111212011121010202202220200202200002111120020200222000022202222022020200000000022002020200001111110221121011120212210002111111111121202210110111012110112102112211102011110021202100200022002002022020020200000112012210021101221110110110112211011221112110211102111022202021100112012220221101200020111212211000020022102011111110112111101010111111111111100101001001120202220002200000002111202111111112201111101200000011021012002111011121211111102002020200002220021100111222002000220020000022202000000202000210010110021022112011010111111101101221000211112210110211110111101022011111210211002220200221111021111111000200202220011101110202020020200111211010111011111022020200002010211011011102100120011110120212201102112122011121212111112211100012111001020100200101221112211102122211002000022020222020011001200112120001121011011001001110022222000020220111211201112111110102111120220111110121021121111111011221111111211211111100120210121121012111122011120121202001102220101122212012121011121211202020222120211112021111111111202211112011111001120111010211011110222111201121100112110121212111111011000202210111111110110210122201011220111112220201110201111111111110201101000122111111111210101121002121210120211111021111221101111121110112102111111121110010200202200020121211102111101111201110111111121021002122010111110101011221011111101022000222222002222121221201022011011111111011221012011221110000100211111010220112020021202000010101011111102111120112011101111111011120122111112000200112111012021101111200211101111011111212101111102121112022112112201222222202000010111121011111211112110111112111111121112011022200002220211010220122222202202001110201021102110011010220202020022002202020222020110211111111111111111111111120022222002022200222222020200022202220022000020220020211111111111101111111001022121000001121102201111101111120111111210111211221100121112011210210211002000222022200022200200002002020222200202111111020110210101111211211221111102011100111101010101210020120111111212121221210111110021122000011111210100011201121020211101101002220111111111211112111102110100210020011221100201101112101110210121200201220010122112121111012111122110122010001121011221020101001111121022020020021112100201211211201111111101120002111211012111111202101111111211212112112220012011111111121111111111111201012210110002000001012110212000110021101101110202201111011100012202200020111111001001221101012020002220011120111111210112012121110101111111211021111102101110120022202020200220200222020220002222200222110222020121111110111022222000020202002200200022010210202212021122202220102020221010212102111020111012201000111211111002000000022202021101211210101120002222002002211000022000202200121111011221112112001121201110211220201111101111111101111111111111102010111011011011010110201111110011111101101111111101010102200100010022211221101011202000011111111110111212100102111112200002022202202002022002220022020222020002002202202022202022020200000000020022220020220202222222200222020000022000020222202002000022222202222020022022002200220002202020220002222200002220000121111012222000011111120200220202200220022211211111212002111101102012102020211010110111111212120111002022011011121102100102111021120101012002000200121222200200001000121112111001001001112002012201011100222112212110021010121111111001111001201011111102102011122221222002000001122211022112011201101111111201121001021001122012011011111110011201202010120211111111102212222022022212011111112110100011112111111111111111111101111111111021111111202200000000020111111111112201120111002101011011102111001220101001221121110101020210001101121102202022022220000202200110110020011112112111020111121020220222000222110111111111111111101112110111210112112011210111211111021111111112020220022022222121211111000212021022201010200022211110101111111202022202001112111012101112111112022002220002200222000002200020000200022222202020022020222222002210111102211201010111111212011111120121112112111110201012201201120021212110110211111221102211000200021110211121111002100220111121111110111112112122112011212020220022022101102122111212111111122001010111122110111011201111021100002211111111110111121111111121120210122011111002222211122111111112011012110110111011110212202000022020220200200022222022000200202000022022111102101010010222222000000011012222011100011111111211200002222200202222222022200200001112022002000002202000121100121101101122102121001111112002120221200120100010200001120012022110102110122201110122100121111210000111021000122110222002120021002100220102110112021211010112201111001110100201101211010202222020020000220200200000111111111120002022220200022220201102201101101022211201000022120222201120200012110211220000112102211100110020121100111100210201112122111201112122120112022200010110112011211110001111111111111111211111211111201102210102101201011111102111110210211102112000110000010011012212000202000211110212110220022020200002000200202202220222022022200220002222220220002200202222020202022200200220220020020222022200020022002200200200020202220002020220220220202020220220212111122202100001210201021110201112111111112101111101111122111111011212110021011212222020002000002222112010101101111011121111110121201121112112221212210010111212001111111211210110122020011101111111201011111111112002200000222012211211100010020121011110210112011010010211100002120111201001101111112121022212210121110212110210101111111121012111101021111011111112201121220111111111222111100211222112110111101101101022000121010001111211111202110110111022211020120111220220200000000202022200000000012010111121012111002011122111111011021211110211111011011111002222202020020202000002002002202200002022022200022021110211001111012112021201011110100022121111120011100110201221221001020222002220222202202200022222200000000002020002000000020202020220202200222020020020021211100111202200202020202202000202221111200111011111211122101201111101210101211022022002020022220000000222022200222020022211010011112100111110100201001111111112121011012121112110102211112001211110010120021211100220111120110111112221101020200222122121102112110202200211100222102110111211121111111011011210111011011111000212122002100201120202222000202200000020020001112011021002002111011211110111120101200010001110110001110120210020111020101211121010001221110011211111010211122002020200201200221120102122001111200210101102112021111111111111111111111122000200022200211201100011101210010211021111111002111122220110110110200221010220102111111102220001111101110021010110020011100212022211111111111100022111012201100110111110010002211010112000002022200022011210211101111120102100022211122112120200002020002011200211001021122111111112021111112010122200120101200111100020211112222110112121012011202101120020222202200220002002002022200220022022020222022200200000200200020000200222022200000201111111110110002000220022222200112121020112101111011111110201012111122110101120002001101121101121210111112100110001011012112011020120011101120121102200022020200002022222202202200000202011111001121210012101211111110112111210101100202110202211121011112022012111111101010112110110121111222022101011210210002202221122002000002002020111211021001212211012201012012011100110020120112111112111111221211111111110102120120101211210112020110112120211101101101110000210010120012110002110220111200020110022000022200002200222020011000221110102012011122110102010000210121002112020100000222101111101212111110101011000220210220111202002220200002222022200002111001111000200102202020220022022200220200000020020201010020111112021012101121101111020200110102201200011202010111100002200200020020200022220220220000220222202220200020200212011121011112211211200200020220022101110111020201211121020220011111120202000202022002020220222000200002222200002202200200222002200200002020200202202222020222220220020220022010100022220022002022022222111101112111201110220010112200021101211111112201201011022222011111000021111211102121100121111100011012120202202211011111110120211110110121000002020001011100211111121111110020022202022020222202220020220202022002000220220020202020222202200202020002220002200122101222011112112121121011100100001101201202022110221012211011220112222011211001012221211121012202110110111101100200201110110102021202202211211121211211012111022001212011111101111112221111110111011111101110211112000211011202001101121202200120110201002111010111012120012120002111101101210100102002211220120101001021212200012101121111001210220200111111100121002020020220222222200200202202200200202000002220220000222000002110220110111112011002022000212112202011212111111111102211111201111101111002200220222220222000020200200201110111111021101111021110121111011111102221111110111002102211112112111110002111110111022111101110201101011111111210021212112102011010202102112010121110121101010121220122002100022202020210101220202100020001110121111101102112122112210011011211111100211002212201112120111010220011111111101201000200000022020202222000200222200020200202000002200202220222010011111111211010012010101222021111120112121011102202111111001110222200110110021111111222102221200122210021101111012100201112211210112111102211122000202011111121121211220000011211202102102012012120020021001011101202211122120111110022200222020020002002202000002002112111112100122002000201211120111022222100100111212112110102200021101212121110211100200021110012020211001000110121021210101121210202001112020100111101200111002111120200212101022020010220110010102110011111221002111102020120012110101112121001120100110002000002010111111202120220120212002101112200101001111120211001221011111101200012011100122021121121210111011210202012221100202020020001222022221011112001102111111101200111111200201210221012010111210202220222002211012021111100200211121011010111111111111002021010202022010111121111111110110010201201120100211120211110120210200011110111012021102101101101011020211101002110111220011101001201120202002222000222220002022222220220000001112121121111110002110111111110110012212220002010101102100101101111101022111201000120120010022020210111020110210001020101111101102111100211111011111101022002001111110111211121020121101110011202200001112120000011002111110010022201111111200022022111201111111000100222100220002111010202220222200002011110111101101211012201111111101110020002210220111022111111021010120211001011120010020002011120100100110110110202110012010111011011211222021110111112120212011211111111211001011010022021110210110010112201211111112111101111111221001002201211111002201022010120101100111110100220001100110110010210200012211120111211022002021111121100112201102000202200200002022220022022000200211112201112111010211121102211000111111120200020122111112001221121222012010101101021111110111122100020220022021011122000111111102111101000210102000000002100201002012111110112220122220122111011112110011101101222120101210010121111102222200012020202020020202200022202020022022202111111121100121111111111110101120120102122121210010021110201101221122121011112002021120021202001010111111110200011101002220211001110102122200200201200200101022110011201101201211111002211101000202000000200220220202020020022022202022202201200220002200200011101121121111021101101120011010211111111111112101010120001220111111100010210010110111011111201021101100002101001211100200121111111200002220202002222020222202022210202202001002000222200020020222022011211000020002222020020000202020202002022022202200211101211101121121101111012002002011121110012202020000020020002022220022222111210210221011122100110121021020111221111211201001111111102110201201201110111001120001111011111021012212111112212101121202220000020000210200111101211110001212111110110111220202002020000020222002002000022022021020212111121111211120110011012021000000202200200020201020220202022020110000020200021001120011111100111212111111101201121221110101111101111111101122211100110100020002222200220200200220220211020111111020011210211021111102111121111211120101021002100011211101121011200020020002002202020020202000222202220020022002220200000002000002002022002222000020022220222020200222000011112221011111020022002020200220112111201121021020112111111010021100101010011100121022111100201112001112211"

        s &= s

        Dim strCurrent() As Byte
        Dim strContrast() As Byte

        strCurrent = System.Text.Encoding.ASCII.GetBytes(s)
        strContrast = System.Text.Encoding.ASCII.GetBytes(s)

        stpw.Restart()
        For x As Integer = 1 To tries
            For k As Integer = 0 To s.Length - 1
                If (s(k) = "2"c AndAlso s(k) = "0"c) OrElse (s(k) = "0"c AndAlso s(k) = "2"c) Then
                    ' Increment count
                End If
            Next
        Next
        stpw.Stop()
        Debug.WriteLine(stpw.Elapsed.TotalMilliseconds)

        Const zero As Byte = 48 'ascii 0
        Const one As Byte = 49 'ascii 1
        Const two As Byte = 50 'ascii 2

        stpw.Restart()
        For x As Integer = 1 To tries
            For k As Integer = 0 To strCurrent.Length - 1
                'the comparison
                If (strCurrent(k) = two AndAlso strContrast(k) = zero) OrElse (strCurrent(k) = zero AndAlso strContrast(k) = two) Then
                    ' Increment count
                End If
            Next
        Next
        stpw.Stop()
        Debug.WriteLine(stpw.Elapsed.TotalMilliseconds)

    End Sub


This post has been edited by dbasnett: 25 June 2013 - 07:13 AM

Was This Post Helpful? 0
  • +
  • -

#15 C.Andrews  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 14
  • View blog
  • Posts: 169
  • Joined: 18-October 12

Re: Faster way to execute my loop

Posted 25 June 2013 - 07:31 AM

View Postdbasnett, on 25 June 2013 - 02:12 PM, said:

I kludged together this test and there is a savings, but maybe not in your case.


    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Const tries As Integer = 100
        Dim stpw As New Stopwatch
        Dim s As String = "*SNIP*"

        s &= s

        Dim strCurrent() As Byte
        Dim strContrast() As Byte

        strCurrent = System.Text.Encoding.ASCII.GetBytes(s)
        strContrast = System.Text.Encoding.ASCII.GetBytes(s)

        stpw.Restart()
        For x As Integer = 1 To tries
            For k As Integer = 0 To s.Length - 1
                If (s(k) = "2"c AndAlso s(k) = "0"c) OrElse (s(k) = "0"c AndAlso s(k) = "2"c) Then
                    ' Increment count
                End If
            Next
        Next
        stpw.Stop()
        Debug.WriteLine(stpw.Elapsed.TotalMilliseconds)

        Const zero As Byte = 48 'ascii 0
        Const one As Byte = 49 'ascii 1
        Const two As Byte = 50 'ascii 2

        stpw.Restart()
        For x As Integer = 1 To tries
            For k As Integer = 0 To strCurrent.Length - 1
                'the comparison
                If (strCurrent(k) = two AndAlso strContrast(k) = zero) OrElse (strCurrent(k) = zero AndAlso strContrast(k) = two) Then
                    ' Increment count
                End If
            Next
        Next
        stpw.Stop()
        Debug.WriteLine(stpw.Elapsed.TotalMilliseconds)

    End Sub



Interesting, this isn't quite how I went about it, so maybe the problem was me. I'll see if I can adapt this method to what I'm doing and see if I can get your speed increase. Every µs counts!

Edit:
Ok, I tried this byte comparison method and I actually lost quite a bit of time, but I think the problem is the overhead in converting my input string to a byte array; to make this work I think I'd have to read the datafile in as bytes and skip the conversion altogether. I'm not really clear on how I could do that, maybe with some kind of 2d array? Here's the loop that I used, which loses enormous amounts of time over previous iterations:

For i As Integer = 0 To AnimalCount - 2
            strCurrent = System.Text.Encoding.ASCII.GetBytes(aryGenome(i))
            For j = (i + 1) To AnimalCount - 1
                strContrast = System.Text.Encoding.ASCII.GetBytes(aryGenome(j))
                For k As Integer = 0 To GeneCount - 1
                    If (strCurrent(k) = two AndAlso strContrast(k) = zero) OrElse (strCurrent(k) = zero AndAlso strContrast(k) = two) Then intDiffCount += 1
                    'If (strCurrent(k) = "2"c And strContrast(k) = "0"c) Or (strCurrent(k) = "0"c And strContrast(k) = "2"c) Then intDiffCount += 1
                Next
                aryResults.Add(intDiffCount.ToString())
                intDiffCount = 0
            Next
        Next


This post has been edited by C.Andrews: 25 June 2013 - 07:48 AM

Was This Post Helpful? 0
  • +
  • -

  • (4 Pages)
  • +
  • 1
  • 2
  • 3
  • Last »