10 Replies - 1155 Views - Last Post: 25 July 2010 - 02:20 PM Rate Topic: -----

#1 creativecoding  Icon User is offline

  • Hash != Encryption
  • member icon


Reputation: 926
  • View blog
  • Posts: 3,205
  • Joined: 19-January 10

Pictures downloader.

Posted 25 July 2010 - 09:46 AM

This downloader is not your average one. Its for downloading textures. You know, for 3d models. But the website I have choosen for the user to browse opens the textures in a new window. To prevent that, I added this code:

private void textureWeb_Navigating(object sender, WebBrowserNavigatingEventArgs e)
        {
            newURL = e.Url.LocalPath.ToString();
        }

        private void textureWeb_NewWindow(object sender, CancelEventArgs e)
        {
            e.Cancel = true;
            textureWeb.Navigate(newURL.ToString());
        }


but for some reason, the newURL variable is set to an ad. As in, when I click on an image that would normally open IE, would now open JUST the add bar from google. Why is this?



btw, the website I am using is textureking.

Is This A Good Question/Topic? 0
  • +

Replies To: Pictures downloader.

#2 Core  Icon User is offline

  • using System.Linq;
  • member icon

Reputation: 774
  • View blog
  • Posts: 5,097
  • Joined: 08-December 08

Re: Pictures downloader.

Posted 25 July 2010 - 10:43 AM

From what I can see, TextureKing is not opening the image in a new window but in the same one. Are you using this?

We would need the other piece of code that retrieves the initial URL to see what's wrong here.
Was This Post Helpful? 0
  • +
  • -

#3 creativecoding  Icon User is offline

  • Hash != Encryption
  • member icon


Reputation: 926
  • View blog
  • Posts: 3,205
  • Joined: 19-January 10

Re: Pictures downloader.

Posted 25 July 2010 - 10:58 AM

View PostCore, on 25 July 2010 - 09:43 AM, said:

From what I can see, TextureKing is not opening the image in a new window but in the same one. Are you using this?

We would need the other piece of code that retrieves the initial URL to see what's wrong here.



that's chrome/firefox for you. The web browser is basically dumbed down Internet Explorer. Try it in IE.
Was This Post Helpful? 0
  • +
  • -

#4 Core  Icon User is offline

  • using System.Linq;
  • member icon

Reputation: 774
  • View blog
  • Posts: 5,097
  • Joined: 08-December 08

Re: Pictures downloader.

Posted 25 July 2010 - 11:02 AM

It is still targeting the correct URL though. How are you getting the URL?
Was This Post Helpful? 0
  • +
  • -

#5 creativecoding  Icon User is offline

  • Hash != Encryption
  • member icon


Reputation: 926
  • View blog
  • Posts: 3,205
  • Joined: 19-January 10

Re: Pictures downloader.

Posted 25 July 2010 - 11:09 AM

View PostCore, on 25 July 2010 - 10:02 AM, said:

It is still targeting the correct URL though. How are you getting the URL?



Yes that is the weird part. It works in any browser, except for mine...

Basically, that is how I am getting the URL of the window that is trying to open (see op).


First:
Posted Image



then when I click on one of the images:
Posted Image
Was This Post Helpful? 0
  • +
  • -

#6 Core  Icon User is offline

  • using System.Linq;
  • member icon

Reputation: 774
  • View blog
  • Posts: 5,097
  • Joined: 08-December 08

Re: Pictures downloader.

Posted 25 July 2010 - 11:14 AM

My first advice would be this - change the event handler from Navigating to DocumentCompleted. That way you will get the URL of the page that is loaded, not loading.
Was This Post Helpful? 0
  • +
  • -

#7 creativecoding  Icon User is offline

  • Hash != Encryption
  • member icon


Reputation: 926
  • View blog
  • Posts: 3,205
  • Joined: 19-January 10

Re: Pictures downloader.

Posted 25 July 2010 - 11:17 AM

View PostCore, on 25 July 2010 - 10:14 AM, said:

My first advice would be this - change the event handler from Navigating to DocumentCompleted. That way you will get the URL of the page that is loaded, not loading.



still does not work...
Was This Post Helpful? 0
  • +
  • -

#8 Core  Icon User is offline

  • using System.Linq;
  • member icon

Reputation: 774
  • View blog
  • Posts: 5,097
  • Joined: 08-December 08

Re: Pictures downloader.

Posted 25 July 2010 - 11:18 AM

Now the question is - how are you navigating to the target URL? Need to see the code for this.
Was This Post Helpful? 0
  • +
  • -

#9 creativecoding  Icon User is offline

  • Hash != Encryption
  • member icon


Reputation: 926
  • View blog
  • Posts: 3,205
  • Joined: 19-January 10

Re: Pictures downloader.

Posted 25 July 2010 - 11:28 AM

weird... My last post didn't go through...

private void download_Tick(object sender, EventArgs e)
        {


            try
            {
                strURL = textureWeb.Url.ToString();

            string strFilePath = "C:\\users\\" + Environment.UserName + "\\Documents\\Textures\\";
            download.Interval = Settings1.Default.speed;
            if (textureWeb.Url.ToString().ToLower().EndsWith("jpg"))
            {
                WebRequest myWebRequest = WebRequest.Create(strURL);

                WebResponse myWebResponse = myWebRequest.GetResponse();

                Stream ReceiveStream = myWebResponse.GetResponseStream();

                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");

                StreamReader readStream = new StreamReader(ReceiveStream, encode);

                string strResponse = readStream.ReadToEnd();

                StreamWriter oSw = new StreamWriter(strFilePath);

                oSw.WriteLine(strResponse);

                oSw.Close();

                readStream.Close();

                myWebResponse.Close();

                textureWeb.GoBack();

            }
            if (textureWeb.Url.ToString().ToLower().EndsWith("png"))
            {
                WebRequest myWebRequest = WebRequest.Create(strURL);

                WebResponse myWebResponse = myWebRequest.GetResponse();

                Stream ReceiveStream = myWebResponse.GetResponseStream();

                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");

                StreamReader readStream = new StreamReader(ReceiveStream, encode);

                string strResponse = readStream.ReadToEnd();

                StreamWriter oSw = new StreamWriter(strFilePath);

                oSw.WriteLine(strResponse);

                oSw.Close();

                readStream.Close();

                myWebResponse.Close();

                textureWeb.GoBack();
            }
            if (textureWeb.Url.ToString().ToLower().EndsWith("gif"))
            {
                WebRequest myWebRequest = WebRequest.Create(strURL);

                WebResponse myWebResponse = myWebRequest.GetResponse();

                Stream ReceiveStream = myWebResponse.GetResponseStream();

                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");

                StreamReader readStream = new StreamReader(ReceiveStream, encode);

                string strResponse = readStream.ReadToEnd();

                StreamWriter oSw = new StreamWriter(strFilePath);

                oSw.WriteLine(strResponse);

                oSw.Close();

                readStream.Close();

                myWebResponse.Close();

                textureWeb.GoBack();
            }
            if (textureWeb.Url.ToString().ToLower().EndsWith("bmp"))
            {
                WebRequest myWebRequest = WebRequest.Create(strURL);

                WebResponse myWebResponse = myWebRequest.GetResponse();

                Stream ReceiveStream = myWebResponse.GetResponseStream();

                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");

                StreamReader readStream = new StreamReader(ReceiveStream, encode);

                string strResponse = readStream.ReadToEnd();

                StreamWriter oSw = new StreamWriter(strFilePath);

                oSw.WriteLine(strResponse);

                oSw.Close();

                readStream.Close();

                myWebResponse.Close();

                textureWeb.GoBack();
            }
            if (textureWeb.Url.ToString().ToLower().EndsWith("tif"))
            {
                WebRequest myWebRequest = WebRequest.Create(strURL);

                WebResponse myWebResponse = myWebRequest.GetResponse();

                Stream ReceiveStream = myWebResponse.GetResponseStream();

                Encoding encode = System.Text.Encoding.GetEncoding("utf-8");

                StreamReader readStream = new StreamReader(ReceiveStream, encode);

                string strResponse = readStream.ReadToEnd();

                StreamWriter oSw = new StreamWriter(strFilePath);

                oSw.WriteLine(strResponse);

                oSw.Close();

                readStream.Close();

                myWebResponse.Close();

                textureWeb.GoBack();
            }

            }
            catch (Exception ex)
            {
                //retard script
            }
        }

        private void textureWeb_Navigating(object sender, WebBrowserNavigatingEventArgs e)
        {
            newURL = e.Url.ToString();
        }

        private void textureWeb_NewWindow(object sender, CancelEventArgs e)
        {
            e.Cancel = true;
            textureWeb.Navigate(newURL.ToString());
        }




This is all I have for that browser. The home URL is textureking so that it loads on startup.



ha ha. Now that I think of it, that code could have been WAY simpler.

This post has been edited by creativecoding: 25 July 2010 - 11:29 AM

Was This Post Helpful? 0
  • +
  • -

#10 PsychoCoder  Icon User is offline

  • Google.Sucks.Init(true);
  • member icon

Reputation: 1641
  • View blog
  • Posts: 19,853
  • Joined: 26-July 07

Re: Pictures downloader.

Posted 25 July 2010 - 12:09 PM

My first question, why all the if statements, it appears you're doing the same thing inside each if.
Was This Post Helpful? 0
  • +
  • -

#11 Core  Icon User is offline

  • using System.Linq;
  • member icon

Reputation: 774
  • View blog
  • Posts: 5,097
  • Joined: 08-December 08

Re: Pictures downloader.

Posted 25 July 2010 - 02:20 PM

Well you are definitely taking the long-road approach and you are producing a lot of spaghetti code. Taking the principles you mentioned, here is what I came up with - a sample console application that downloads all images from a specific web page, as long as those are inside a link (therefore, a link points to it).

using System;
using System.Collections.Generic;
using System.Net;
using System.Threading;
using System.IO;
using System.Text.RegularExpressions;
using System.Drawing;

namespace ConsoleApplication
{
    class Program
    {
        static int totalFiles = 0;
        static int currentFiles = 0;

        static void Main(string[] args)
        {
            GetImages("http://www.textureking.com/index.php/category/all-textures");
        }

        static void GetImages(string url)
        {
            string responseString;
            
            HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
            using (HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse())
            {
                using (StreamReader reader = new StreamReader(response.GetResponseStream()))
                {
                    responseString = reader.ReadToEnd();
                }
            }

            List<string> imageset = new List<string>();
            Regex regex = new Regex(@"f=""[^""]*jpg|bmp|tif|gif|png",RegexOptions.IgnoreCase);
            
            foreach (Match m in regex.Matches(responseString))
            {
                if (!imageset.Contains(m.Value))
                    imageset.Add(m.Value);
            }

            for (int i = 0; i < imageset.Count; i++)
                imageset[i] = imageset[i].Remove(0, 3);

            totalFiles = imageset.Count;
            currentFiles = totalFiles;

            Console.WriteLine(totalFiles.ToString() + " images will be downloaded.");

            foreach (string f in imageset)
            {
                ThreadPool.QueueUserWorkItem(new WaitCallback(DownloadImage), f);
            }

            Console.Read();
        }

        static void DownloadImage(object path)
        {
            currentFiles--;
            Console.WriteLine("Downloading " + Path.GetFileName(path.ToString()) + "... (" + (totalFiles - currentFiles).ToString() + "/" + totalFiles + ")");
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(path.ToString());
            using (HttpWebResponse _response = (HttpWebResponse)request.GetResponse())
            {
                Image image = Image.FromStream(_response.GetResponseStream());
                image.Save(@"D:\Temporary\" + Path.GetFileName(path.ToString()));
            }
            Console.WriteLine(Path.GetFileName(path.ToString()) + " downloaded.");
        }
    }

}


As you can see here, I am using regular expressions to actually find the image links and then I am putting them inside a generic collection (List). I am also checking against duplicates, since if you looked at the page source, there are actually two links to the same image at TextureKing, therefore you don't want the same file re-written.

The application is multi-threaded - each file is downloaded in its own thread. This saves you some time - instead of waiting for one file to download before starting another, you are actually downloading several files at once.

You'll have to figure out the URLs for the request on your own already.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1