1 Replies - 207 Views - Last Post: 24 February 2014 - 08:28 AM Rate Topic: -----

#1 ChrisDLFC  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 22
  • Joined: 13-December 11

Getting data from a webpage problem.

Posted 24 February 2014 - 04:23 AM

I'm extremely new to Python but need to write scripts to get data from a webpage as part of a larger project , I have installed the python packages 'requests' and 'BeautifulSoup'. To gather the data I found a piece of tutorial code online and tried to re-use it for myself but the output isn't as expiated, this is the code:

#!/bin/sh

import requests
import BeautifulSoup

session = requests.session()

req = session.get('http://www.bbc.co.uk/sport/0/football/')

doc = BeautifulSoup.BeautifulSoup(req.content)

print doc.findAll('<a href="/sport/football/teams/', { "class" : "gp-share" })



This returns

[ ]



Basically, I want to find all the occurrences of '<a href="/sport/football/teams/' in the html document 'http://www.bbc.co.uk/sport/0/football/' and write it to an external file(although I haven't got that far yet so for now I'm just printing out the data in the terminal).

Thanks for any help in advance.

Is This A Good Question/Topic? 0
  • +

Replies To: Getting data from a webpage problem.

#2 modi123_1  Icon User is offline

  • Suitor #2
  • member icon



Reputation: 9187
  • View blog
  • Posts: 34,489
  • Joined: 12-June 08

Re: Getting data from a webpage problem.

Posted 24 February 2014 - 08:28 AM

Scraping content is against their terms of use, and we will not help you break it. Perhaps looking into their API.
http://www.bbc.co.uk...ology/apis.html
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1