[hurremSultan]

Hi,

I'm just strating to work with files and databases..
In my project I'm getting a word and then search for it in my text file. The text file has just words splitted by ";" and the words are not sorted.. I need an advice, how should I use this text file?

Should I convert it to CSV file, or insert it to the database.. myself, I actually don't want to use database, I think working with files is faster and the program will be lighter.. am I right?
What kind of classes should I use to search inside the file, to make the search faster, should I use regular StreamReader or there is something else with better solution?

I will be glad for any advice.. thank you

[tlhIn`toq]

hurremSultan, on 15 January 2013 - 05:55 PM, said:

I actually don't want to use database, I think working with files is faster and the program will be lighter.. am I right?

No.

Regardless of hte source, database or file the fastest thing is memory over harddrive. So if you have a small number of words (less than a quarter million) then just load them all into memory and deal with it from there.

But the size of the word list is going to be limited to the PC.

If you are dealing with text files regardless of format, reading the file over and over will take a long time.

A true database will be the most versatile since it won't limit the number of words/object to you deal with to just what can be stored in memory.

To summarize: The direction you should take is dependent on the purpose of the application, the required speed of response and the size of the collection you are dealing with - NOT the formate of the text file.

[b0zhidar]

Hmm, I am not sure if I understood your question, but if it's what I thought, it'd be something like this:

    string file = "D:\\words.txt"

    string text = System.IO.File.ReadAllText(file);
    string[] splitted = text.Split(';');
    foreach (var line in splitted)
 { 
 if(line=="yourSearchWord")
{
MessageBox.Show("Word found: " + line);
}

 }

Sorry for any mistakes, coded it without IDE, here and free hand

[hurremSultan]

Thanks very much guys. As I understand, if my word list is small enough to fit into memory then I can use the text file as data source, using System.IO.File.ReadAllText(file).. my word list is small, it contains abount 2000 words...

Should I sort the words? or it doesn't make sense?

[b0zhidar]

I don't see how sorting would do anything. It would loop through every word (splitted by and check if it's the word you have searched.

[Skydiver]

Actually it depends on your usage. If you are going to search for only a single word within the text file, then sorting is not going to buy you anything. As mentioned before at worse you'll have to go through the entire file. If you are going to search for multiple words, then you can be more efficient with your searches because your searches can be quicker if you apply a binary search, or some kind of indexed search (eg B+ trees, tries, etc.) on your sorted key.

[eclipsed4utoo]

If you are dead set on using a file instead of a database, I would have the words sorted. That way, you can use a binary search to quickly find the word instead of looping through all of the words. A binary search will greatly speed up the search, but is useless if it's not sorted.

[hurremSultan]

I tried the code above, and it took me very long to search, as I searching for every form of word!!!!

For example if I get the word "computers", first I search for computers and compare it with all my 2000 words, and as there is no word computeres, I'll get and search for "computer"...

I thnk I need to sort my file and try tries, I hope there is a ready class to use tries.

[hurremSultan]

I was reading about different collections of C#, and stoped my mind on using HashSet..

Please, could anybody show me, how to put the words from txt file into hash... please

[Skydiver]

Ummm... I found the sample code in the MSDN documentation for the HashSet to be really instructive. Did you have problems extending that ideas there to strings instead of integers?
http://msdn.microsof...y/bb359438.aspx

[b0zhidar]

As far as I am aware, for() loop is 5 times faster than foreach(). Use that instead.

for(int i = 0; i<=array.Length;i++)
{
if(array[i]=="yourSearchedWord")
{
//
}
}

Anyway this is not the most suitable solution, but it will do the job.

[eclipsed4utoo]

A for loop is not 5 times faster. That would mean that if a foreach loop took 10 seconds to process a loop, then a for loop would take 2 seconds. For the most part, the performance difference is negligible. The only time I choose for over foreach is when I need the index.

[Curtis Rutland]

The relative speed difference between for and foreach would depend on the implementation of the IEnumerable's IEnumerator, since that handles the iteration. It's theoretically possible to write an extremely poorly implemented enumerator that would make the for loop much faster, but I wouldn't bet that the framework's enumerators are that inefficient.

[Curtis Rutland]

In fact, I decided to test this theory. Seems to not be true at all. Code for the test is attached. Gist is, I used two loops to iterate over a collection of a million ints, and copy each into a new list (just a trivial task to do something in the loop). I averaged the results for 1000 test runs.

For Loop Average Time: 		00:00:00.0154793
Foreach Loop Average Time: 	00:00:00.0143463

Spoiler

public static void Main()
{
    var list1 = Enumerable.Repeat(1, 1000000).ToList();
    var list2 = new List<int>();
    var forResults = new List<TimeSpan>();
    var foreachResults = new List<TimeSpan>();
    var stopwatch = new Stopwatch();
    for (var z = 0; z < 1000; z++)
    {
        Console.WriteLine("Test {0} in progress", z);
        //for loop
        stopwatch.Start();
        DoForLoop(list1, list2);
        stopwatch.Stop();
        forResults.Add(stopwatch.Elapsed);
        list2.Clear();
        stopwatch.Reset();
        //foreach loop
        stopwatch.Start();
        DoForeachLoop(list1, list2);
        stopwatch.Stop();
        foreachResults.Add(stopwatch.Elapsed);
        list2.Clear();
        stopwatch.Reset();
    }
    Console.WriteLine("Averages");
    var forAverage = AverageTimeSpan(forResults);
    Console.WriteLine("For Loop Average Time: {0}", forAverage);
    var foreachAverage = AverageTimeSpan(foreachResults);
    Console.WriteLine("Foreach Loop Average Time: {0}", foreachAverage);
    Console.ReadKey();
}

public static void DoForLoop(List<int> list1, List<int> list2)
{
    for (var i = 0; i < list1.Count; i++)
    {
        //some trivial action
        list2.Add(list1[i]);
    }
}

public static void DoForeachLoop(List<int> list1, List<int> list2)
{
    foreach (var item in list1)
    {
        //some trivial action
        list2.Add(item);
    }
}

public static TimeSpan AverageTimeSpan(IEnumerable<TimeSpan> collection)
{
    var averageTicks = Convert.ToInt64(collection.Average(x => x.Ticks));
    return new TimeSpan(averageTicks);
}

[h4nnib4l]

Would that be because it's faster to access the foreach loop's local variable than it is to go to the list index in the for loop on each iteration? I'm not claiming to know, I'm genuinely asking/speculating here.