3 Replies - 223 Views - Last Post: 11 March 2014 - 09:42 AM Rate Topic: -----

#1 jfalco  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 2
  • Joined: 11-March 14

Best way to update References Page Automatically

Posted 11 March 2014 - 09:26 AM

I am working on a project where my client wants a references page full of peer reviewed sources from this other website which has the entire collection available. His site is a joomla site, which lists only specific resources (the newest) from the other source website.

Right now the content editors are just adding these books and articles by hand. Other projects have gotten in the way, and now they are a year behind on the references, and a bunch of new ones have been added to the other site already. I was hired to research how to automate the process if possible.

This is what I would like to make possible:

  • Updates the page as soon as a new reference from the other site is added so we don't have to worry about adding new references by hand.
  • Adds all the references that we already missed automatically. The latest entry is from Jun 2013 and the last we have is from April. There are quite a few sources that we missed, and the list on the other website is pretty disorganized.


I think PHP will be the best language in order to do this by, but I am also open to suggestions with XML and RSS. I'm also not sure we can do this without some type of server...Since we don't have one.

Is This A Good Question/Topic? 0
  • +

Replies To: Best way to update References Page Automatically

#2 modi123_1  Icon User is online

  • Suitor #2
  • member icon



Reputation: 9580
  • View blog
  • Posts: 36,300
  • Joined: 12-June 08

Re: Best way to update References Page Automatically

Posted 11 March 2014 - 09:36 AM

So.. First question's first.. does this "other website which has the entire collection available" allow bots, automation, and/or scraping of their data in their terms of use?
Was This Post Helpful? 0
  • +
  • -

#3 jfalco  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 2
  • Joined: 11-March 14

Re: Best way to update References Page Automatically

Posted 11 March 2014 - 09:39 AM

View Postmodi123_1, on 11 March 2014 - 09:36 AM, said:

So.. First question's first.. does this "other website which has the entire collection available" allow bots, automation, and/or scraping of their data in their terms of use?


Well they do have a feature where people can create their own RSS feeds of the items they have available. I'm not sure if this is considered automation, but if they allow that, I don't see it being a problem if we ask permission.
Was This Post Helpful? 0
  • +
  • -

#4 modi123_1  Icon User is online

  • Suitor #2
  • member icon



Reputation: 9580
  • View blog
  • Posts: 36,300
  • Joined: 12-June 08

Re: Best way to update References Page Automatically

Posted 11 March 2014 - 09:42 AM

I would suggest looking at this site's TOS/Terms of Use.. and also securing that permission first. That may - significantly - impact how you are able to proceed.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1