13 Replies - 1428 Views - Last Post: 20 April 2009 - 02:38 PM Rate Topic: -----

#1 Glaive  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 9
  • Joined: 17-April 09

C# Command-Line Instructions Help

Post icon  Posted 17 April 2009 - 06:16 AM

Hi all,

I've been trying to get some ideas about a project I am doing for my course, and have decided to create a web crawler. Whilst searching for examples I came across this sample program and it is very similar to what I want to do, so I thought I'd try it out to see what it does and what it does not do, so that I get an idea of what features to have in my own program. The problem is, although it comes with brief instructions at the top of the program, I have no idea how to use this program. I am using Visual Studio 2008 and when I start debugging it, it seems to debug fine with no errors and a console screen flashes quickly across the screen and then I am back looking at the code window.

Thanks in advance.

/* MiniCrawler: A skeletal Web crawler.
Usage:
To start crawling, specify a starting
URI on the command line. For example,
to start at McGraw-Hill.com, use this
command line:
MiniCrawler http://McGraw-Hill.com
*/

using System.Collections.Generic;
using System.Linq;
using System.Text;
using System;
using System.Net;
using System.IO;
class MiniCrawler
{
	// Find a link in a content string.
	static string FindLink(string htmlstr,
	ref int startloc)
	{
		int i;
		int start, end;
		string uri = null;
		string lowcasestr = htmlstr.ToLower();
		i = lowcasestr.IndexOf("href=\"http", startloc);
		if (i != -1)
		{
			start = htmlstr.IndexOf('"', i) + 1;
			end = htmlstr.IndexOf('"', start);
			uri = htmlstr.Substring(start, end - start);
			startloc = end;
		}
		return uri;
	}
	static void Main(string[] args)
	{
		string link = null;
		string str;
		string answer;
		int curloc; // holds current location in response
		if (args.Length != 1)
		{
			Console.WriteLine("Usage: MiniCrawler <uri>");
			return;
		}
		string uristr = args[0]; // holds current URI
		try
		{
			do
			{
				Console.WriteLine("Linking to " + uristr);
				// Create a WebRequest to the specified URI.
				HttpWebRequest req = (HttpWebRequest)
				WebRequest.Create(uristr);
				uristr = null; // disallow further use of this URI
				// Send that request and return the response.
				HttpWebResponse resp = (HttpWebResponse)
				req.GetResponse();
				// From the response, obtain an input stream.
				Stream istrm = resp.GetResponseStream();
				// Wrap the input stream in a StreamReader.
				StreamReader rdr = new StreamReader(istrm);
				// Read in the entire page.
				str = rdr.ReadToEnd();
				curloc = 0;
				do
				{
					// Find the next URI to link to.
					link = FindLink(str, ref curloc);
					if (link != null)
					{
						Console.WriteLine("Link found: " + link);
						Console.Write("Link, More, Quit?");
						answer = Console.ReadLine();
						if (string.Compare(answer, "L", true) == 0)
						{
							uristr = string.Copy(link);
							break;
						}
						else if (string.Compare(answer, "Q", true) == 0)
						{
							break;
						}
						else if (string.Compare(answer, "M", true) == 0)
						{
							Console.WriteLine("Searching for another link.");
						}
					}
					else
					{
						Console.WriteLine("No link found.");
						break;
					}
				} while (link.Length > 0);
				// Close the response.
				resp.Close();
			} while (uristr != null);
		}
		catch (WebException exc)
		{
			Console.WriteLine("Network Error: " + exc.Message +
			"\nStatus code: " + exc.Status);
		}
		catch (ProtocolViolationException exc)
		{
			Console.WriteLine("Protocol Error: " + exc.Message);
		}
		catch (UriFormatException exc)
		{
			Console.WriteLine("URI Format Error: " + exc.Message);
		}
		catch (NotSupportedException exc)
		{
			Console.WriteLine("Unknown Protocol: " + exc.Message);
		}
		catch (IOException exc)
		{
			Console.WriteLine("I/O Error: " + exc.Message);
		}
		Console.WriteLine("Terminating MiniCrawler.");
	}
}



Is This A Good Question/Topic? 0
  • +

Replies To: C# Command-Line Instructions Help

#2 SixOfEleven  Icon User is offline

  • using Caffeine;
  • member icon

Reputation: 945
  • View blog
  • Posts: 6,342
  • Joined: 18-October 08

Re: C# Command-Line Instructions Help

Posted 17 April 2009 - 06:29 AM

View PostGlaive, on 17 Apr, 2009 - 05:16 AM, said:

Hi all,

I've been trying to get some ideas about a project I am doing for my course, and have decided to create a web crawler. Whilst searching for examples I came across this sample program and it is very similar to what I want to do, so I thought I'd try it out to see what it does and what it does not do, so that I get an idea of what features to have in my own program. The problem is, although it comes with brief instructions at the top of the program, I have no idea how to use this program. I am using Visual Studio 2008 and when I start debugging it, it seems to debug fine with no errors and a console screen flashes quickly across the screen and then I am back looking at the code window.

Thanks in advance.


The program is designed to run from a command prompt. What you need to do is (in XP, I don't have Vista) click start, run and type cmd.

You will have to use the cd command to navigate to where the exe of the program is and type the name of the program.
Was This Post Helpful? 0
  • +
  • -

#3 danzar  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 8
  • View blog
  • Posts: 108
  • Joined: 10-December 08

Re: C# Command-Line Instructions Help

Posted 17 April 2009 - 09:37 AM

The program is designed to run from a command prompt. What you need to do is (in XP, I don't have Vista) click start, run and type cmd.

You will have to use the cd command to navigate to where the exe of the program is and type the name of the program.
[/quote]


Make sure when you put it in the CMD prompt to put in the url want to crawl also.. example =C:/> Webcrawler http://www.yahoo.com
Was This Post Helpful? 0
  • +
  • -

#4 Glaive  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 9
  • Joined: 17-April 09

Re: C# Command-Line Instructions Help

Posted 18 April 2009 - 10:47 AM

Thank you for the replies, I've tried to type the command line into the command prompt, however it says it is "not recognized as an internal or external command, operable program or batch file". Am I supposed to compile the file somehow? I've just pasted into Visual Studio 2008 and clicked save, is there a way for me to build it into a .exe file so that I can run that via the CMD prompt? From the menu toolbars perhaps (I've searched but unfortunately haven't been able to find it)

Once again, thank you very much, I really appreciate the help, and I apologies for these novice questions.
Was This Post Helpful? 0
  • +
  • -

#5 SixOfEleven  Icon User is offline

  • using Caffeine;
  • member icon

Reputation: 945
  • View blog
  • Posts: 6,342
  • Joined: 18-October 08

Re: C# Command-Line Instructions Help

Posted 18 April 2009 - 11:06 AM

In the Build menu you click Build project to compile it to an .exe. Once you have compiled the project open Windows Explorer. Go to the location of the project folder. Navigate to the debug folder. Go to the bin folder in debug. Copy the files into a directory such as C:\MyProgram. Goto the CMD prompt. At the propmpt type:

cd C:\MyProgram
Webcrawler [url=http://www.yahoo.com]http://www.yahoo.com[/url]


Was This Post Helpful? 0
  • +
  • -

#6 Glaive  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 9
  • Joined: 17-April 09

Re: C# Command-Line Instructions Help

Posted 19 April 2009 - 06:27 AM

View PostSixOfEleven, on 18 Apr, 2009 - 10:06 AM, said:

In the Build menu you click Build project to compile it to an .exe. Once you have compiled the project open Windows Explorer. Go to the location of the project folder. Navigate to the debug folder. Go to the bin folder in debug. Copy the files into a directory such as C:\MyProgram. Goto the CMD prompt. At the propmpt type:

cd C:\MyProgram
Webcrawler [url=http://www.yahoo.com]http://www.yahoo.com[/url]



Cheers for the help mate, I'll try that out and I'll let you know the result either tonight or tomorrow morning, thank you very much.
Was This Post Helpful? 0
  • +
  • -

#7 SixOfEleven  Icon User is offline

  • using Caffeine;
  • member icon

Reputation: 945
  • View blog
  • Posts: 6,342
  • Joined: 18-October 08

Re: C# Command-Line Instructions Help

Posted 19 April 2009 - 06:37 AM

Just realized something. The order of the folders is bin then debug or if you are compiling the release version you will want to go to bin then release.
Was This Post Helpful? 0
  • +
  • -

#8 Glaive  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 9
  • Joined: 17-April 09

Re: C# Command-Line Instructions Help

Posted 20 April 2009 - 06:30 AM

Thanks for the help SixOfEleven, however unfortunately when I do what you told me however I get the CMD Message:

'Webcrawler' is not recognized as an internal or external command, operable program or batch file.

Do you know why this is, is it because I need a system32 file or something? If you tried compiling and running this program yourself, did it work for you? Thanks

This post has been edited by Glaive: 20 April 2009 - 06:31 AM

Was This Post Helpful? 0
  • +
  • -

#9 SixOfEleven  Icon User is offline

  • using Caffeine;
  • member icon

Reputation: 945
  • View blog
  • Posts: 6,342
  • Joined: 18-October 08

Re: C# Command-Line Instructions Help

Posted 20 April 2009 - 07:00 AM

View PostGlaive, on 20 Apr, 2009 - 05:30 AM, said:

Thanks for the help SixOfEleven, however unfortunately when I do what you told me however I get the CMD Message:

'Webcrawler' is not recognized as an internal or external command, operable program or batch file.

Do you know why this is, is it because I need a system32 file or something? If you tried compiling and running this program yourself, did it work for you? Thanks


I will download it and see if I can help you out.
Was This Post Helpful? 0
  • +
  • -

#10 magius96  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 71
  • View blog
  • Posts: 777
  • Joined: 15-April 09

Re: C# Command-Line Instructions Help

Posted 20 April 2009 - 07:00 AM

When you setup your project, what name did you give the project? What files are in the <project>/bin/debug folder? There should be an executable file in there, that executable file is your app. The instructions that come with the script you are trying to use assumes that your app will be called Webcrawler.

So look and see what the exe name is in your /bin/debug directory, then type that in place of Webcrawler. For instance if it's called "test.exe" then you'll type "text <web address>".
Was This Post Helpful? 0
  • +
  • -

#11 SixOfEleven  Icon User is offline

  • using Caffeine;
  • member icon

Reputation: 945
  • View blog
  • Posts: 6,342
  • Joined: 18-October 08

Re: C# Command-Line Instructions Help

Posted 20 April 2009 - 07:12 AM

I've done something better for you than trying to get it to run from the command line. I've updated the program so it will run directly from Visual Studio. All you have to do is enter the URL when you are prompted for it. When the next part comes up you enter M, L or Q.

I hope this work for you. I got it to find yahoo by entering: http://www.yahoo.com

/* MiniCrawler: A skeletal Web crawler.
Usage:
To start crawling, specify a starting
URI on the command line. For example,
to start at McGraw-Hill.com, use this
command line:
MiniCrawler [url="http://McGraw-Hill.com"]http://McGraw-Hill.com[/url]
*/

using System.Collections.Generic;
using System.Linq;
using System.Text;
using System;
using System.Net;
using System.IO;
class MiniCrawler
{
    // Find a link in a content string.
    static string FindLink(string htmlstr,
    ref int startloc)
    {
        int i;
        int start, end;
        string uri = null;
        string lowcasestr = htmlstr.ToLower();
        i = lowcasestr.IndexOf("href=\"http", startloc);
        if (i != -1)
        {
            start = htmlstr.IndexOf('"', i) + 1;
            end = htmlstr.IndexOf('"', start);
            uri = htmlstr.Substring(start, end - start);
            startloc = end;
        }
        return uri;
    }
    static void Main(string[] args)
    {
        string link = null;
        string str;
        string answer;
        int curloc; // holds current location in response

        string uristr; // holds current URI
        Console.WriteLine("Please enter a URI:");

        uristr = Console.ReadLine();

        try
        {
            do
            {
                Console.WriteLine("Linking to " + uristr);
                // Create a WebRequest to the specified URI.
                HttpWebRequest req = (HttpWebRequest)
                WebRequest.Create(uristr);
                uristr = null; // disallow further use of this URI
                // Send that request and return the response.
                HttpWebResponse resp = (HttpWebResponse)
                req.GetResponse();
                // From the response, obtain an input stream.
                Stream istrm = resp.GetResponseStream();
                // Wrap the input stream in a StreamReader.
                StreamReader rdr = new StreamReader(istrm);
                // Read in the entire page.
                str = rdr.ReadToEnd();
                curloc = 0;
                do
                {
                    // Find the next URI to link to.
                    link = FindLink(str, ref curloc);
                    if (link != null)
                    {
                        Console.WriteLine("Link found: " + link);
                        Console.Write("Link, More, Quit?");
                        answer = Console.ReadLine();
                        if (string.Compare(answer, "L", true) == 0)
                        {
                            uristr = string.Copy(link);
                            break;
                        }
                        else if (string.Compare(answer, "Q", true) == 0)
                        {
                            break;
                        }
                        else if (string.Compare(answer, "M", true) == 0)
                        {
                            Console.WriteLine("Searching for another link.");
                        }
                    }
                    else
                    {
                        Console.WriteLine("No link found.");
                        break;
                    }
                } while (link.Length > 0);
                // Close the response.
                resp.Close();
            } while (uristr != null);
        }
        catch (WebException exc)
        {
            Console.WriteLine("Network Error: " + exc.Message +
            "\nStatus code: " + exc.Status);
        }
        catch (ProtocolViolationException exc)
        {
            Console.WriteLine("Protocol Error: " + exc.Message);
        }
        catch (UriFormatException exc)
        {
            Console.WriteLine("URI Format Error: " + exc.Message);
        }
        catch (NotSupportedException exc)
        {
            Console.WriteLine("Unknown Protocol: " + exc.Message);
        }
        catch (IOException exc)
        {
            Console.WriteLine("I/O Error: " + exc.Message);
        }
        Console.WriteLine("Terminating MiniCrawler.");
        Console.ReadLine();
    }
}


Was This Post Helpful? 1
  • +
  • -

#12 Glaive  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 9
  • Joined: 17-April 09

Re: C# Command-Line Instructions Help

Posted 20 April 2009 - 07:43 AM

View Postmagius96, on 20 Apr, 2009 - 06:00 AM, said:

When you setup your project, what name did you give the project? What files are in the <project>/bin/debug folder? There should be an executable file in there, that executable file is your app. The instructions that come with the script you are trying to use assumes that your app will be called Webcrawler.

So look and see what the exe name is in your /bin/debug directory, then type that in place of Webcrawler. For instance if it's called "test.exe" then you'll type "text <web address>".


Thank you for the input, I really appreciate it, however SixOfEleven's latter post has been the most helpful and I've decided to use that.


View PostSixOfEleven, on 20 Apr, 2009 - 06:12 AM, said:

I've done something better for you than trying to get it to run from the command line. I've updated the program so it will run directly from Visual Studio. All you have to do is enter the URL when you are prompted for it. When the next part comes up you enter M, L or Q.

I hope this work for you. I got it to find yahoo by entering: http://www.yahoo.com


SixOfEleven I could kiss you! =) Thank you ever so much for all your help throughout, now I can see what this program does and it works perfectly thanks to your tweaking. I'll attempt to make my own version of it with some new features, and use this as a guide. Once again thank you to all who helped, I am very grateful.

Is there a way to thank you for this? I'll press the "Found post helpful button" if there is another way let me know too =)


Regards,

Annii
Was This Post Helpful? 0
  • +
  • -

#13 SixOfEleven  Icon User is offline

  • using Caffeine;
  • member icon

Reputation: 945
  • View blog
  • Posts: 6,342
  • Joined: 18-October 08

Re: C# Command-Line Instructions Help

Posted 20 April 2009 - 09:02 AM

View PostGlaive, on 20 Apr, 2009 - 06:43 AM, said:

SixOfEleven I could kiss you! =) Thank you ever so much for all your help throughout, now I can see what this program does and it works perfectly thanks to your tweaking. I'll attempt to make my own version of it with some new features, and use this as a guide. Once again thank you to all who helped, I am very grateful.

Is there a way to thank you for this? I'll press the "Found post helpful button" if there is another way let me know too =)


Regards,

Annii


You're welcome and thanks for the Found post helpful. That is all I can really ask for. :)

Jamie

*edit*
You may want to edit your original post and put <Solved> in the topic. :)

This post has been edited by SixOfEleven: 20 April 2009 - 09:04 AM

Was This Post Helpful? 0
  • +
  • -

#14 Glaive  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 9
  • Joined: 17-April 09

Re: C# Command-Line Instructions Help

Posted 20 April 2009 - 02:38 PM

For some reason it won't let me edit my original post so I can put <Solved> I will probably have to let a moderator/admin edit it, if possible.

And you are also welcome for the thanks :P
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1