how to use a text file as a database

  • (2 Pages)
  • +
  • 1
  • 2

18 Replies - 1645 Views - Last Post: 17 January 2013 - 09:57 AM Rate Topic: -----

#1 hurremSultan  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 10
  • Joined: 15-January 13

how to use a text file as a database

Posted 15 January 2013 - 04:55 PM

Hi,

I'm just strating to work with files and databases..
In my project I'm getting a word and then search for it in my text file. The text file has just words splitted by ";" and the words are not sorted.. I need an advice, how should I use this text file?

Should I convert it to CSV file, or insert it to the database.. myself, I actually don't want to use database, I think working with files is faster and the program will be lighter.. am I right?
What kind of classes should I use to search inside the file, to make the search faster, should I use regular StreamReader or there is something else with better solution?

I will be glad for any advice.. thank you

Is This A Good Question/Topic? 0
  • +

Replies To: how to use a text file as a database

#2 tlhIn`toq  Icon User is offline

  • Please show what you have already tried when asking a question.
  • member icon

Reputation: 5571
  • View blog
  • Posts: 11,910
  • Joined: 02-June 10

Re: how to use a text file as a database

Posted 15 January 2013 - 05:29 PM

View PosthurremSultan, on 15 January 2013 - 05:55 PM, said:

I actually don't want to use database, I think working with files is faster and the program will be lighter.. am I right?


No.

Regardless of hte source, database or file the fastest thing is memory over harddrive. So if you have a small number of words (less than a quarter million) then just load them all into memory and deal with it from there.

But the size of the word list is going to be limited to the PC.

If you are dealing with text files regardless of format, reading the file over and over will take a long time.

A true database will be the most versatile since it won't limit the number of words/object to you deal with to just what can be stored in memory.

To summarize: The direction you should take is dependent on the purpose of the application, the required speed of response and the size of the collection you are dealing with - NOT the formate of the text file.
Was This Post Helpful? 2
  • +
  • -

#3 b0zhidar  Icon User is offline

  • New D.I.C Head

Reputation: 4
  • View blog
  • Posts: 24
  • Joined: 21-December 12

Re: how to use a text file as a database

Posted 15 January 2013 - 07:27 PM

Hmm, I am not sure if I understood your question, but if it's what I thought, it'd be something like this:

    string file = "D:\\words.txt"

    string text = System.IO.File.ReadAllText(file);
    string[] splitted = text.Split(';');
    foreach (var line in splitted)
 { 
 if(line=="yourSearchWord")
{
MessageBox.Show("Word found: " + line);
}

 }


Sorry for any mistakes, coded it without IDE, here and free hand :D
Was This Post Helpful? 1
  • +
  • -

#4 hurremSultan  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 10
  • Joined: 15-January 13

Re: how to use a text file as a database

Posted 16 January 2013 - 01:01 AM

Thanks very much guys. As I understand, if my word list is small enough to fit into memory then I can use the text file as data source, using System.IO.File.ReadAllText(file).. my word list is small, it contains abount 2000 words...

Should I sort the words? or it doesn't make sense?
Was This Post Helpful? 0
  • +
  • -

#5 b0zhidar  Icon User is offline

  • New D.I.C Head

Reputation: 4
  • View blog
  • Posts: 24
  • Joined: 21-December 12

Re: how to use a text file as a database

Posted 16 January 2013 - 01:07 AM

I don't see how sorting would do anything. It would loop through every word (splitted by ;) and check if it's the word you have searched.
Was This Post Helpful? 1
  • +
  • -

#6 Skydiver  Icon User is online

  • Code herder
  • member icon

Reputation: 3616
  • View blog
  • Posts: 11,267
  • Joined: 05-May 12

Re: how to use a text file as a database

Posted 16 January 2013 - 05:43 AM

Actually it depends on your usage. If you are going to search for only a single word within the text file, then sorting is not going to buy you anything. As mentioned before at worse you'll have to go through the entire file. If you are going to search for multiple words, then you can be more efficient with your searches because your searches can be quicker if you apply a binary search, or some kind of indexed search (eg B+ trees, tries, etc.) on your sorted key.
Was This Post Helpful? 0
  • +
  • -

#7 eclipsed4utoo  Icon User is offline

  • Not Your Ordinary Programmer
  • member icon

Reputation: 1524
  • View blog
  • Posts: 5,960
  • Joined: 21-March 08

Re: how to use a text file as a database

Posted 16 January 2013 - 06:22 AM

If you are dead set on using a file instead of a database, I would have the words sorted. That way, you can use a binary search to quickly find the word instead of looping through all of the words. A binary search will greatly speed up the search, but is useless if it's not sorted.
Was This Post Helpful? 0
  • +
  • -

#8 hurremSultan  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 10
  • Joined: 15-January 13

Re: how to use a text file as a database

Posted 16 January 2013 - 07:11 AM

I tried the code above, and it took me very long to search, as I searching for every form of word!!!!

For example if I get the word "computers", first I search for computers and compare it with all my 2000 words, and as there is no word computeres, I'll get and search for "computer"...

I thnk I need to sort my file and try tries, I hope there is a ready class to use tries.
Was This Post Helpful? 0
  • +
  • -

#9 hurremSultan  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 10
  • Joined: 15-January 13

Re: how to use a text file as a database

Posted 16 January 2013 - 02:12 PM

I was reading about different collections of C#, and stoped my mind on using HashSet..

Please, could anybody show me, how to put the words from txt file into hash... please
Was This Post Helpful? 0
  • +
  • -

#10 Skydiver  Icon User is online

  • Code herder
  • member icon

Reputation: 3616
  • View blog
  • Posts: 11,267
  • Joined: 05-May 12

Re: how to use a text file as a database

Posted 16 January 2013 - 03:13 PM

Ummm... I found the sample code in the MSDN documentation for the HashSet to be really instructive. Did you have problems extending that ideas there to strings instead of integers?
http://msdn.microsof...y/bb359438.aspx
Was This Post Helpful? 0
  • +
  • -

#11 b0zhidar  Icon User is offline

  • New D.I.C Head

Reputation: 4
  • View blog
  • Posts: 24
  • Joined: 21-December 12

Re: how to use a text file as a database

Posted 16 January 2013 - 11:21 PM

As far as I am aware, for() loop is 5 times faster than foreach(). Use that instead.
for(int i = 0; i<=array.Length;i++)
{
if(array[i]=="yourSearchedWord")
{
//
}
}


Anyway this is not the most suitable solution, but it will do the job.
Was This Post Helpful? 1
  • +
  • -

#12 eclipsed4utoo  Icon User is offline

  • Not Your Ordinary Programmer
  • member icon

Reputation: 1524
  • View blog
  • Posts: 5,960
  • Joined: 21-March 08

Re: how to use a text file as a database

Posted 17 January 2013 - 06:12 AM

A for loop is not 5 times faster. That would mean that if a foreach loop took 10 seconds to process a loop, then a for loop would take 2 seconds. For the most part, the performance difference is negligible. The only time I choose for over foreach is when I need the index.
Was This Post Helpful? 3
  • +
  • -

#13 Curtis Rutland  Icon User is online

  • (╯□)╯︵ (~ .o.)~
  • member icon


Reputation: 4526
  • View blog
  • Posts: 7,893
  • Joined: 08-June 10

Re: how to use a text file as a database

Posted 17 January 2013 - 08:18 AM

The relative speed difference between for and foreach would depend on the implementation of the IEnumerable's IEnumerator, since that handles the iteration. It's theoretically possible to write an extremely poorly implemented enumerator that would make the for loop much faster, but I wouldn't bet that the framework's enumerators are that inefficient.
Was This Post Helpful? 1
  • +
  • -

#14 Curtis Rutland  Icon User is online

  • (╯□)╯︵ (~ .o.)~
  • member icon


Reputation: 4526
  • View blog
  • Posts: 7,893
  • Joined: 08-June 10

Re: how to use a text file as a database

Posted 17 January 2013 - 08:58 AM

In fact, I decided to test this theory. Seems to not be true at all. Code for the test is attached. Gist is, I used two loops to iterate over a collection of a million ints, and copy each into a new list (just a trivial task to do something in the loop). I averaged the results for 1000 test runs.

For Loop Average Time: 		00:00:00.0154793
Foreach Loop Average Time: 	00:00:00.0143463


Spoiler

Was This Post Helpful? 1
  • +
  • -

#15 h4nnib4l  Icon User is offline

  • The Noid
  • member icon

Reputation: 1182
  • View blog
  • Posts: 1,677
  • Joined: 24-August 11

Re: how to use a text file as a database

Posted 17 January 2013 - 09:10 AM

Would that be because it's faster to access the foreach loop's local variable than it is to go to the list index in the for loop on each iteration? I'm not claiming to know, I'm genuinely asking/speculating here.
Was This Post Helpful? 0
  • +
  • -

  • (2 Pages)
  • +
  • 1
  • 2