The project basically has 3 console apps and 1 windows application
The first app collects the information i need and stores it into a xml database
The second app takes the longest to run because it downloads all the pages based on the info in the xml file while also editing the pages to remove things i don't want and making links local. this one is for making the pages and database from scratch and shouldn't need to run it again unless something happens to the whole. (currently downloading 32,147 pages slowly over the next 8 hours so i wont overload there server and so i can sleep and let it do all the work.)
The third app will be for updating the database and pages
and the last windows app will be for searching the database and opening the page's.
i must say for something like this i really like the consoles and there so easy to set to run automaticly.
So again i say thank you. I was making this for myself and for fun and with this site i didn't get stuck for long and never got frustrated. I also learned a lot and can't wait to make my next scrapper a better one.
This post has been edited by sired22: 11 August 2008 - 05:55 PM