[crazy123]

It is possible with python, make a program to access a site (which provides data) and download the data?

[sepp2k]

Yes.

[crazy123]

sepp2k, on 07 May 2012 - 06:58 PM, said:

Yes.

How can I do this? With socket?

[sepp2k]

If all you want to do is download data form a given url, you can just use urllib.request.urlopen.

[crazy123]

sepp2k, on 07 May 2012 - 07:42 PM, said:

If all you want to do is download data form a given url, you can just use urllib.request.urlopen.

I need upload my file and download date of the site

[atraub]

I'm assuming uploading files is a normal function of the site?

[crazy123]

atraub, on 08 May 2012 - 12:52 PM, said:

I'm assuming uploading files is a normal function of the site?

Uploading is to search data in the site (I send a file .txt to the site) and download these data searched.

[crazy123]

I'd like to send data (urllib.urlencode??) for the field search of site. After obtaining the url of the search.
How can I do this?

[atraub]

I'm sorry, but I'm having difficulty understanding you. Could you try asking your question in a different way?

[crazy123]

atraub, on 08 May 2012 - 04:38 PM, said:

I'm sorry, but I'm having difficulty understanding you. Could you try asking your question in a different way?

OK

The site has a search field (like google). In my python script I like to send which must be searched and get the url of the site after searching.

[atraub]

so, you want to use a script to have the site run a search based on some input string.

Then, you want the URL of wherever that site sends you to?


If the site uses query strings, you might be able to get the URL of the search results without actually doing a search. For example, let's look at Google.

If I want to google the expression "gimme teh codez" I could just type in http://www.google.co...immeh+teh+codez if I wanted to search for "hello world" I could just go to http://www.google.co...h?q=hello+world Thus, for any string I want to search, I could replaces the spaces in the string with plus signs and append it to the back of http://www.google.com/search?q= It's possible you can utilize the same concept, but I'm really not the guy to ask.

P.S. this is called a query string.

[crazy123]

atraub, on 08 May 2012 - 06:56 PM, said:

so, you want to use a script to have the site run a search based on some input string.

Then, you want the URL of wherever that site sends you to?


If the site uses query strings, you might be able to get the URL of the search results without actually doing a search. For example, let's look at Google.

If I want to google the expression "gimme teh codez" I could just type in http://www.google.co...immeh+teh+codez if I wanted to search for "hello world" I could just go to http://www.google.co...h?q=hello+world Thus, for any string I want to search, I could replaces the spaces in the string with plus signs and append it to the back of http://www.google.com/search?q= It's possible you can utilize the same concept, but I'm really not the guy to ask.

P.S. this is called a query string.

Thanks. I can manipulate the url.
My problem now is that: I send my string to search the site and I can load the site. The site has a download button. How, in python, I click that button and I download files?

*I do not know the URL of the download

[crazy123]

On a site like this: http://www.geonames.org
I can send the information to search in site. How to download data from the search?

[atraub]

Research website scraping such as scrapy or beautiful soup.

[crazy123]

atraub, on 21 May 2012 - 06:06 PM, said:

Research website scraping such as scrapy or beautiful soup.

OK. Using scrapy and BeautifulSoup I can navigating in site. But how to download the contents of the site?