3 Replies - 2432 Views - Last Post: 21 July 2015 - 04:45 PM Rate Topic: -----

#1 STOD   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 3
  • Joined: 19-July 15

help with urllib2/requests

Posted 19 July 2015 - 04:50 PM

hey guys, i just started learning python about a month ago and i am trying to code something for practice which is a script that checks websites for certain sub domains, so pretty much it asks user for website then runs through a wordlist(file) that has the sub domain names to try, then loop over every word and combine it with target, ask for the http response code and if its not 404(not found) to print the subdomain+target, status code.. but i run into a problem a error which i dont get whats going on, and in some sites it actually work but its not accurate.. i tried it with requests and urllib2 and i get i would guess pretty much the same error, if you guys can explain what im doing wrong i would appreciate it, anyways heres the codes for both urllib2 and requests :

import urllib2
import requests

print "\n USAGE [ TARGET>>> target.com ]\n"
target = str(raw_input("TARGET>>> "))
domainlist = open("domain.txt", "rU")   
for domain in domainlist:
    domain = domain.rstrip() + target
    url = urllib2.urlopen("http://%s" % domain)    
if url.getcode != 404:
	print "\n[%s]----->>>Status[%s]"%(domain,url.getcode())
else:
    print "\n[%s]----->>>Status[%s]"%(domain,url.getcode())
url.close()
domainlist.close()
      
'''
#doing it with REQUESTS!!!!!
print "\n USAGE [ TARGET>>> target.com ]\n"
target = str(raw_input("TARGET>>> "))
domainlist = open("domain.txt", "rU")   
for domain in domainlist:
    domain = domain.rstrip() + target
url= requests.get("http://%s" % domain)
if url.status_code != 404:
	print "\n[%s]----->>>Status[%s]"%(domain,url.status_code)
else:
    print "\n[%s]----->>>Status[%s]"%(domain,url.status_code)
url.close()
domainlist.close() 
'''  



heres an example of the domain.txt:

Quote

test.
test1.
test2.
test3.
www1.
www2.
www3.


now when i try and run it against "bing.com" it runs but it doesnt seem too accurate heres the response:

Quote

C:\Users\*****\Desktop>subdom.py

USAGE [ TARGET>>> target.com ]

TARGET>>> bing.com

[www3.bing.com]----->>>Status[200]


now when i try and run it on "google.com" or most of the websites i get this error
for urllib2

Quote

raise URLError(err)
urllib2.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
the line its on
File "C:\Users\******\Desktop\subdom.py", line 10, in <module>
url = urllib2.urlopen("%s" % domain)


for requests

Quote

raise Connectionerror(err, request=request)
requests.exceptions.Connectionerror: ('Connection aborted.', gaierror(11001, 'getaddrinfo failed'))
and same object..
File "C:\Users\dotszilla\Desktop\subdom.py", line 24, in <module>
url= requests.get("http://%s" % domain)


can someone please help me understand what iam doing wrong please, i know its something to do with that object im making.. i know its a noobie question but i just started learning and am trying to teach myself, thanks in advance guys

Is This A Good Question/Topic? 0
  • +

Replies To: help with urllib2/requests

#2 andrewsw   User is offline

  • no more Mr Potato Head
  • member icon

Reputation: 6957
  • View blog
  • Posts: 28,696
  • Joined: 12-December 12

Re: help with urllib2/requests

Posted 20 July 2015 - 02:04 AM

Quote

the line its on
File "C:\Users\******\Desktop\subdom.py", line 10, in <module>
url = urllib2.urlopen("%s" % domain)

That line doesn't occur in the code you've posted.

You have this:

Quote

url = urllib2.urlopen("http://%s" % domain)

Is this the line that you meant? Print out the values of domain.
Was This Post Helpful? 0
  • +
  • -

#3 STOD   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 3
  • Joined: 19-July 15

Re: help with urllib2/requests

Posted 21 July 2015 - 03:39 PM

View Postandrewsw, on 20 July 2015 - 02:04 AM, said:

Quote

the line its on
File "C:\Users\******\Desktop\subdom.py", line 10, in <module>
url = urllib2.urlopen("%s" % domain)

That line doesn't occur in the code you've posted.

You have this:

Quote

url = urllib2.urlopen("http://%s" % domain)

Is this the line that you meant? Print out the values of domain.


yeah thats it iam sorry, i changed up that line a bit so i didnt have to have the http:// in my domains.txt file.. and i copied the error to the previous one which didnt have the http:// .. so if you try it now you get
url = urllib2.urlopen("http://%s" % domain) in the error, sorry about that didnt notice it... but yeah anyways on another forum someone told me that the reason its happening is because you dont get a 404 error if there isnt a certain subdomain in the server... so i guess i gotta do some more research into that... but if anyone can help me achieve what i want so i can understand it, that would be great... thanks
Was This Post Helpful? 0
  • +
  • -

#4 STOD   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 3
  • Joined: 19-July 15

Re: help with urllib2/requests

Posted 21 July 2015 - 04:45 PM

PROBLEM SOLVED guys, someone helped me in another forum ill post the code here just so people know how you do it...
thank you anyways...
import urllib2

print "\n USE [ TARGET>>> target.com ]\n"
target = str(raw_input("TARGET>>> "))

with open("domain.txt", "rU") as domainlist:
	for domain in domainlist:
		domain = domain.rstrip()+"."+ target
		try:
			url = urllib2.urlopen("http://{}".format(domain))    
		except urllib2.URLError:
			pass
		else:
			print "\n[%s]----->>>Status[%s]"%(domain,url.getcode())
			url.close()


Was This Post Helpful? 0
  • +
  • -

Page 1 of 1