2 Replies - 399 Views - Last Post: 07 July 2015 - 06:42 AM Rate Topic: -----

#1 firoj26   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 1
  • Joined: 06-July 15

python script hang over the screen method in cloud

Posted 06 July 2015 - 09:01 PM

I have an issue with running python scripts on linode cloud server. I create a scrapping script to scrape data by check in and check out so that i run the script using screen method. It run over one day very smoothly but today it hang. No response. When i call getRequest() method then i print the proxy number and send request with proxy and timeout=10.

def getRequest(self,url):
		self.proxieName = random.choice(self.proxyDict.keys())
		print getYellow("Proxy: ")+getCyan(self.proxieName)
		self.proxie = self.proxyDict[self.proxieName]
		try:
			self.getRqst=requests.get(url,headers=self.nrmlHeaders,proxies=self.proxie, timeout=10)
		except Exception, e:
			print e
			time.sleep(10)
			return self.getRequest(url)
		else:
			if self.getRqst.status_code==200:
				self.rqstErr = 0
				return self.getRqst
			else:
				self.rqstErr+=1
				print getYellow("Hi Dear, Don't worry. May be Server down. Exception in Url: ")+url
				print getRed("Error!!! ")+getBlue("I get Status Code: ")+str(self.getRqst.status_code)
				print getGreen("I am sleeping now for 10 second")
				time.sleep(10)
				return self.getRequest(url)



Here is the snapshot of putty terminal:

Attached Image

after getting the content i parse them and save information into a CSV file and send another request.

And i see also that after 01:40 in 6th July the csv file alos not updated. So it's hanging.

Is This A Good Question/Topic? 0
  • +

Replies To: python script hang over the screen method in cloud

#2 andrewsw   User is offline

  • RequestedRangeNotSatisfiable
  • member icon

Reputation: 6560
  • View blog
  • Posts: 26,597
  • Joined: 12-December 12

Re: python script hang over the screen method in cloud

Posted 06 July 2015 - 11:01 PM

What is the purpose of this script?
Was This Post Helpful? 0
  • +
  • -

#3 modi123_1   User is online

  • Suitor #2
  • member icon



Reputation: 14146
  • View blog
  • Posts: 56,698
  • Joined: 12-June 08

Re: python script hang over the screen method in cloud

Posted 07 July 2015 - 06:42 AM

andrewsw is right to question this. Most, if not all, airlines are against scraping their data. As it is I will close the thread and ask you do not pursue it further here. If you have questions on the rules feel free to shoot me a PM.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1