4 Replies - 1442 Views - Last Post: 19 December 2011 - 03:49 AM Rate Topic: -----

#1 Lord Ra  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 14
  • Joined: 24-December 09

Web Image Crawler: Shopbot

Posted 18 December 2011 - 01:42 PM

Hi folks,

I am designing a web image crawler that focuses on using an initial image to find where the object in the image can be purchased.

Basically the aim is to upload an image, use an image processing software package and then find a similar image online including details for purchase i.e. what the object is, where can it be bought, how much is costing etc.

I was thinking that my crawling would be focused in type and target marketing websites like ebay.com, amazon.com, comparethemarket.com etc. but I am not sure if this is reliable method and if I may run into legal problems.

Another possibility is to try and use search engines (Google, Yahoo, Bing, MSN etc.): the image search tool to find and compare their indexed images with my uploaded image. However I am not sure if it's possible to browse their entire image index and if it's allowed.

I am requesting any ideas/ suggestions on how I might proceed or any concepts I should look up.

Thanks in advance for your time,

Regards,

RA

Is This A Good Question/Topic? 0
  • +

Replies To: Web Image Crawler: Shopbot

#2 tlhIn`toq  Icon User is offline

  • Closing in on 5,000
  • member icon

Reputation: 4928
  • View blog
  • Posts: 10,465
  • Joined: 02-June 10

Re: Web Image Crawler: Shopbot

Posted 18 December 2011 - 02:46 PM

I don't want you to write my code, just give me ideas on how to solve my problem.
Was This Post Helpful? 0
  • +
  • -

#3 Lord Ra  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 14
  • Joined: 24-December 09

Re: Web Image Crawler: Shopbot

Posted 18 December 2011 - 06:19 PM

I am sorry mate, I didn't ask for any code. I am just bouncing ideas; if you got any ideas how I might proceed it'd be very helpful.

Further background info
For the moment I am able to crawl to a depth 3 and extract all images SRC, starting from a given a URL. However I am out of ideas on how to proceed because I heard from a mate that it was not reliable crawling from marketing websites because eventually my IP address would be blocked and my application would not work.

Also I am wondering if there's a search word that would return all images indexed by the search engines.

Kind Regards,

RA
Was This Post Helpful? 0
  • +
  • -

#4 tlhIn`toq  Icon User is offline

  • Closing in on 5,000
  • member icon

Reputation: 4928
  • View blog
  • Posts: 10,465
  • Joined: 02-June 10

Re: Web Image Crawler: Shopbot

Posted 18 December 2011 - 07:03 PM

Might just be easier to download Web Devil and use that.
Was This Post Helpful? 1
  • +
  • -

#5 RexGrammer  Icon User is offline

  • Coding Dynamo
  • member icon

Reputation: 178
  • View blog
  • Posts: 750
  • Joined: 27-October 11

Re: Web Image Crawler: Shopbot

Posted 19 December 2011 - 03:49 AM

I think that you should first look up to how to make a ordinary web crawler, then focus on image specific.

Refer to these links for more info:

Web Crawler (DotNetSpider Article)
C# A Simple Web Crawler (ExampleCode Article)
Building a web crawler (developerfusion Tutorial)
Was This Post Helpful? 1
  • +
  • -

Page 1 of 1