[Ferencn]

Sounds like you're trying to leech a bunch of files from a website.

Please show us some code and we may be able to help you fix it. We will not write the application for you.

[JackOfAllTrades]

WTH does Bing have to do with anything??? That's weird.

[Sergio Tapia]

You can download a websites HTML source code using the WebClient.Downloadstring() method.

[Sergio Tapia]

Yes, it is! Why don't you join the site, you ask questions showing ideas and not simple GIEF ME CODEZ PLX like some people.

What you can do is look into HTMLAgilityPack. It's a library (.dll) that you can use to filter whatever you need out of source code.

So you could fetch all of the IMG tags from the source and then use some sort of REGEX to fit things according to what month the user selected.

Just keep in mind that the second the webmaster changes his date scheme your application is DEAD.

[JackOfAllTrades]

Where directory browsing isn't allowed, no API is going to help. You will need to download the HTML source and scrape the data out (hence the term web scraping) for subsequent retrieval.

[Ferencn]

r3bb, on 09 February 2010 - 08:14 AM, said:

So now my question has become: is it possible to filter those filenames out of the huge file that contains the whole source code (which I could easily create), taking them from either one of these two tags?

Yes that is possible. Note that you could find some rules that allow you to compose the correct path to an image from a combination of the two paths in the thumbnail.
Stapia gutierrez gives some pointers and correctly observes that as soon as the naming convention, or the way the thumbnailpage is built changes, your application wil stop to function.

[Sergio Tapia]

I'll give you some help with the HTMLAgilityPack aspect. Using this you can do find all the img tags in an HTML source code file. Hope this helps.

Our target:

<img src="blabalbalbal.jpeg" />

How to fetch the actual URL's of the images:

var document = new HtmlWeb().Load(url);
var urls = document.DocumentNode.Descendants("img")
                                .Select(e => e.GetAttributeValue("src", null))
                                .Where(s => !String.IsNullOrEmpty(s));

[r3bb]

Quote

Stapia gutierrez gives some pointers and correctly observes that as soon as the naming convention, or the way the thumbnailpage is built changes, your application wil stop to function.

Yes, I already thought of that. I guess that when that'll happen I'll change my program

Thanks for the help, when I get home I'll start experimenting with that. And then probably I'll annoy you again

By the way, as you can see I joined

[Sergio Tapia]

Great, don't hesitate to ask for help.

[r3bb]

So, the WebClient.Downloadstring() method you mentioned before would give me a simple wall of text, while the var document = new HtmlWeb().Load(url); thing produces something in which I can in some way browse, something like Javascript's DOM, am I right?

[Sergio Tapia]

The DownloadString() method downloads exactly what you see when you press Ctrl+U in Firefox. The source html markup.

[r3bb]

After thinking about it, I've come to the conclusion that I'll use DownloadString() and a regex to extract what I want.
I already have thought about the regex

string exp = @"(?<=http://images.comicbookresources.com/solicits/marvelcomics/" + date + @"/sm/)\w+(?<!tpb|hc|hcvar)\.jpg";

This one does just what I need, I've already tested it (yes, it heavily relies on the way the website is structured, I know).
I've also written a method that puts the source into a string

private string getPageSource(string url)
		{
			try
			{
				WebClient client = new WebClient();
				string source = client.DownloadString(url);
				webClient.Dispose();
			}
			catch (Exception e)
			{
				MessageBox.Show("Problemi nella connessione HTTP.");
			}
			return source;
		}

but I've got a couple of doubts.
The first one is whether or not it is a good idea to have such a huge string in memory, or maybe it would be better to use a text file to store it (the source is more or less 1400-1500 lines...).
The second one is how should I actually apply the regex to such string/file.
While using this very useful tool this morning, putting the regex (of course with an actual number instead of my date parameter, such as 201001) and putting the whole source in the Source texbox, I was given all the results I needed. In what form will the results be? My ultimate goal as I said in the first post is to create a text file that has every image name I filtered as a line. Will the result be a string alreay formatted this way? Will it be an array with each result as an element? Have I misunderstood the page I linked before and will it just be a string with the first match (because, as I read, regular expression return the first match)?