7 Replies - 639 Views - Last Post: 28 October 2009 - 08:41 AM Rate Topic: -----

#1 Dannyboy997  Icon User is offline

  • D.I.C Regular
  • member icon

Reputation: 64
  • View blog
  • Posts: 258
  • Joined: 17-April 09

Implementing Bots?

Posted 27 October 2009 - 01:57 PM

hi, i am currently working on a forum software and i was wondering how would i get the bots to track my forum like the other softwares do they have a couple hundred registered as bots!!! i'm not sure if you can do this your self or is it all automated. Like do you just have to wait until your forum gets indexed or something!!

thanks!

and also if this is the wrong place to post just move it if you can i'm not sure where to post it!!!!

Is This A Good Question/Topic? 0
  • +

Replies To: Implementing Bots?

#2 no2pencil  Icon User is online

  • Admiral Fancy Pants
  • member icon

Reputation: 5411
  • View blog
  • Posts: 27,420
  • Joined: 10-May 07

Re: Implementing Bots?

Posted 27 October 2009 - 07:47 PM

View PostDannyboy997, on 27 Oct, 2009 - 02:57 PM, said:

how would i get the bots to track my forum like the other softwares

What other softwares?

Quote

i'm not sure if you can do this your self or is it all automated.

What are you trying to track?
Was This Post Helpful? 0
  • +
  • -

#3 Master Jake  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 16
  • View blog
  • Posts: 106
  • Joined: 27-February 09

Re: Implementing Bots?

Posted 27 October 2009 - 07:53 PM

I think he means like other forum software and the such. You know how you can look down at the "Who's Online" list sometimes, and on most forum softwares it can tell you when a bot is browsing your forum, indexing data.

Mine says "Google [Bot]" and sometimes "Google Adsense [Bot]" on my PHPBB forum, which is odd because I don't even have adsense.

But anyway, OP, Google will find your site and index data on it automatically every so often. To help it find you easier, use HTML Meta tags.

Good keywords, description, etc. will help search engines find you.

Read more about Meta Here.
Was This Post Helpful? 0
  • +
  • -

#4 no2pencil  Icon User is online

  • Admiral Fancy Pants
  • member icon

Reputation: 5411
  • View blog
  • Posts: 27,420
  • Joined: 10-May 07

Re: Implementing Bots?

Posted 27 October 2009 - 07:55 PM

View PostMaster Jake, on 27 Oct, 2009 - 08:53 PM, said:

"Who's Online" list

I used this tutorial on Yard Sales Galore. The admin panel shows the exact users online.
Was This Post Helpful? 0
  • +
  • -

#5 Dannyboy997  Icon User is offline

  • D.I.C Regular
  • member icon

Reputation: 64
  • View blog
  • Posts: 258
  • Joined: 17-April 09

Re: Implementing Bots?

Posted 28 October 2009 - 07:39 AM

ok. thanks thats what i wanted to know about the google[bots] on the phpbb forums or such..

thanks
Was This Post Helpful? 0
  • +
  • -

#6 Master Jake  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 16
  • View blog
  • Posts: 106
  • Joined: 27-February 09

Re: Implementing Bots?

Posted 28 October 2009 - 08:33 AM

Thought so :)

By the way, I think every search engine has a unique "user agent" code that it sends when it visits a page. I believe forum softwares such as PHPBB take advantage of these codes by reading them and matching them with the respective search engine.

For example, Google's user agent is Googlebot/2.1 (http://www.googlebot.com/bot.html)

I believe that is how PHPBB displays WHICH bot is visiting and when.
Was This Post Helpful? 1
  • +
  • -

#7 Dannyboy997  Icon User is offline

  • D.I.C Regular
  • member icon

Reputation: 64
  • View blog
  • Posts: 258
  • Joined: 17-April 09

Re: Implementing Bots?

Posted 28 October 2009 - 08:36 AM

ok that makes total sense thanks thats even better!
Was This Post Helpful? 0
  • +
  • -

#8 Master Jake  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 16
  • View blog
  • Posts: 106
  • Joined: 27-February 09

Re: Implementing Bots?

Posted 28 October 2009 - 08:41 AM

No problem. Glad to have helped.

Oh yeah! As a bonus, I found this short list of common user agents on a help article about scraping links with CURL and DOM.

Search Engine User Agents

* Google Googlebot/2.1 ( http://www.googlebot.com/bot.html)
* Google Image Googlebot-Image/1.0 ( http://www.googlebot.com/bot.html)
* MSN Live msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)
* Yahoo Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.co.../ysearch/slurp)
* ask

Browser User Agents

* Firefox (WindowsXP) Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6
* IE 7 Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)
* IE 6 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)
* Safari Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/522.11 (KHTML, like Gecko) Safari/3.0.2
* Opera Opera/9.00 (Windows NT 5.1; U; en)
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1