6 Replies - 809 Views - Last Post: 21 April 2013 - 05:11 AM Rate Topic: -----

#1 Orochimaru  Icon User is offline

  • New D.I.C Head

Reputation: 3
  • View blog
  • Posts: 47
  • Joined: 26-December 12

How can I number increment my variable names within a loop?

Posted 20 April 2013 - 06:22 AM

Hi,

I want my variables (pageX, htmlX, soupX) to number increment when it is run by a loop. But I can't figure out the proper syntax or the correct way to go about this. Help please. :helpsmilie:/>/>

mech = Browser()
url = "http://www.palewire.com/scrape/albums/2007.html"
page1 = mech.open(url)
html1 = page1.read()
soup1 = BeautifulSoup(html1)
#extract(soup1, 2007)

i = 1

while True:
    page2 = mech.follow_link(text_regex="Next")
    html2 = page2.read()
    soup2 = BeautifulSoup(html2)
    extract(soup2, 2006)

    i =+ 1

#----------------------------------------------------------
# So that it looks something like this:

i = 1

while True:
    page"${i}" = mech.follow_link(text_regex="Next")
    html"${i}" = page"${i}".read()
    soup"${i}" = BeautifulSoup(html"${i}")
    extract(soup"${i}", 2006)

    i =+ 1




Is This A Good Question/Topic? 0
  • +

Replies To: How can I number increment my variable names within a loop?

#2 sepp2k  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2118
  • View blog
  • Posts: 3,244
  • Joined: 21-June 11

Re: How can I number increment my variable names within a loop?

Posted 20 April 2013 - 06:25 AM

i =+ 1 is i = + 1 is i = +1. In other words i =+ 1 sets i to positive 1. You want i += 1.
Was This Post Helpful? 1
  • +
  • -

#3 Orochimaru  Icon User is offline

  • New D.I.C Head

Reputation: 3
  • View blog
  • Posts: 47
  • Joined: 26-December 12

Re: How can I number increment my variable names within a loop?

Posted 20 April 2013 - 07:55 AM

Yes I will try to remember not to reverse += in the future. :smile2:/>/>

But my question still remains. How should I go about making my variables dynamically change names during a loop? I want these variables (page-${X}, html-${X}, soup-${X}) to be dynamic so that I don't have to manually type page1, page2, page3, ..., page99 code blocks to handle multiple paginations.

But I think my dynamic variable syntax is way off and I can't find any example codes for what I want to achieve. Help with pointing me in the right directions please. :helpsmilie:/>


19	# So that it looks something like this:
20	 
21	i = 1
22	 
23	while True:
24	    page-"${i}" = mech.follow_link(text_regex="Next")
25	    html-"${i}" = page-"${i}".read()
26	    soup-"${i}" = BeautifulSoup(html-"${i}")
27	    extract(soup-"${i}", 2006)
28	 
29	    i += 1

Was This Post Helpful? 0
  • +
  • -

#4 Orochimaru  Icon User is offline

  • New D.I.C Head

Reputation: 3
  • View blog
  • Posts: 47
  • Joined: 26-December 12

Re: How can I number increment my variable names within a loop?

Posted 20 April 2013 - 08:06 AM

The entire source code.

from mechanize import Browser
import mechanize
from bs4 import BeautifulSoup
import urllib, os

def extract(soup, year):
    table = soup.find("table", border=1)
    for row in table.findAll('tr')[1:]:
        col = row.findAll('td')
        rank = col[0].string
        artist = col[1].string
        album = col[2].string
        cover_link = col[3].img['src']
        record = (str(year), rank, artist, album, cover_link)
        print >> outfile, "|".join(record)
#        save_as = os.path.join("./images/", album + ".jpg")
#        urllib.urlretrieve("http://www.palewire.com" + cover_link, save_as)
#        print "Downloaded %s album cover" % album
        print "Row row, row your boat..."

outfile = open("albums.txt", "w")

mech = Browser()
url = "http://www.palewire.com/scrape/albums/2007.html"
page1 = mech.open(url)
html1 = page1.read()
soup1 = BeautifulSoup(html1)
#print soup.prettify()
#extract(soup1, 2007)

page2 = mech.follow_link(text_regex="Next")
html2 = page2.read()
soup2 = BeautifulSoup(html2)
extract(soup2, 2006)

try:
    page3 = mech.follow_link(text_regex="Next")
#except:
except mechanize.LinkNotFoundError:
    print "\nNo more pagination links found.\n"
else:
    print "\n\t --- Link found! --- \n"   

outfile.close()





Was This Post Helpful? 0
  • +
  • -

#5 Orochimaru  Icon User is offline

  • New D.I.C Head

Reputation: 3
  • View blog
  • Posts: 47
  • Joined: 26-December 12

Re: How can I number increment my variable names within a loop?

Posted 20 April 2013 - 08:17 AM

Or maybe I don't have to numerate my variables after all since every run in the loop saves everything in a textfile. I will experiment with this further tomorrow, eyes are too tired now. :chris:/>
Was This Post Helpful? 0
  • +
  • -

#6 sepp2k  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 2118
  • View blog
  • Posts: 3,244
  • Joined: 21-June 11

Re: How can I number increment my variable names within a loop?

Posted 20 April 2013 - 08:44 AM

Don't use numbered variables, use lists. Accessing variables dynamically is a pain and a maintainability nightmare. With lists you can just write pages[i] and that's it.
Was This Post Helpful? 1
  • +
  • -

#7 Orochimaru  Icon User is offline

  • New D.I.C Head

Reputation: 3
  • View blog
  • Posts: 47
  • Joined: 26-December 12

Re: How can I number increment my variable names within a loop?

Posted 21 April 2013 - 05:11 AM

[SOLVED]

from mechanize import Browser
import mechanize
from bs4 import BeautifulSoup
import urllib, os

def extract(soup, year):
    table = soup.find("table", border=1)
    for row in table.findAll('tr')[1:]:
        col = row.findAll('td')
        rank = col[0].string
        artist = col[1].string
        album = col[2].string
        cover_link = col[3].img['src']
        record = (str(year), rank, artist, album, cover_link)
        print >> outfile, "|".join(record)
#        save_as = os.path.join("./images/", album + ".jpg")
#        urllib.urlretrieve("http://www.palewire.com" + cover_link, save_as)
#        print "Downloaded %s album cover" % album
        print "Row row, row your boat..."

outfile = open("albums.txt", "w")

mech = Browser()
url = "http://www.palewire.com/scrape/albums/2007.html"
page1 = mech.open(url)
html1 = page1.read()
soup1 = BeautifulSoup(html1)
#print soup.prettify()
extract(soup1, 2007)

#page2 = mech.follow_link(text_regex="Next")
#html2 = page2.read()
#soup2 = BeautifulSoup(html2)
#extract(soup2, 2006)

year = 2007

while True:
    try:
        page = mech.follow_link(text_regex="Next")
        html = page.read()
        soup = BeautifulSoup(html)
        year -= 1
        extract(soup, year)
    except mechanize.LinkNotFoundError:
        print """
________________________________________________________________________________

Exception: mechanize.LinkNotFoundError

    * No more pagination links found, ending program.
"""
        break
    else:
        print "\nWritten content in the file successfully."   

outfile.close()



Was This Post Helpful? 0
  • +
  • -

Page 1 of 1