[Raminator]

Hey, i just started a new project that will get an inputed key word (like dog, or grey socks and google it, then it should get the 4 first results and save the site page, image, whatever in a default folder.
The big questions for me are:
1:How to make my program "acess" google.
2:How to make it save the results.
There is no code for now, i wasnt able to write anything down as i lack the knowledge to do the 2 main parts of the process.

This post has been edited by Raminator: 16 February 2013 - 12:59 PM

[tlhIn`toq]

"I don't know where to start" - This usually means you should go back to your instructor and admit you are this lost. Don't bluff your way through this course thinking that by chapter 10 it will all suddenly snap into place and become clear. It won't. Unlike history class where chapter 1 might be 17th century England and chapter 2 might be World War II, giving you a fresh start - Coding builds upon the lessons of the previous chapter. You have to use lesson 1 material to succeed in lesson 2. Chapter 10 builds upon and uses material from chapter 9. If you let your pride get in the way you will be too lost to recover and have wasted thousands of dollars in tuition.

So you're really asking ... "How do I architect and build my entire program?"

Would someone show me how to build my program? - Read this article

We have an area where you can post this as a job for hire. Would you like the thread moved there?

Quote

2:How to make it save the results.

[*]Q: ... save data, save properties, save environmental variables, serialize my data/class?
A:

• Separating data from GUI - PLUS - serializing the data to XML
  
• Working with environmental variables
  
• Writing a text file tutorial.
  
• Reading a text file tutorial.
  
• Easy Application Settings example

This post has been edited by tlhIn`toq: 16 February 2013 - 01:27 PM

[andrewsw]

You can either use the Google Search API (usage free up to 100 searches per day) or load the Google search page into a WebBrowser control and parse the HTML content of the page.

[torind_2000]

First, you need to learn how to do basic research.
This is the first thing I googled in regards to your topic.
Link

Also, as you've no doubt been told before, we aren't here to write your programs. We're only here to provide you guidance with problems in your code.
You must provide us YOUR code along with any changes you've made to get your code to function so that we can see what you have tried.
Only then can our experts help you.

Now, if you'd like to hire one of the experts here write your program for you, I'm sure one or more of them would be willing to discuss financial compensation for their coding services.

[Raminator]

tlhIn`toq, on 16 February 2013 - 01:15 PM, said:

"I don't know where to start" - This usually means you should go back to your instructor and admit you are this lost. Don't bluff your way through this course thinking that by chapter 10 it will all suddenly snap into place and become clear. It won't. Unlike history class where chapter 1 might be 17th century England and chapter 2 might be World War II, giving you a fresh start - Coding builds upon the lessons of the previous chapter. You have to use lesson 1 material to succeed in lesson 2. Chapter 10 builds upon and uses material from chapter 9. If you let your pride get in the way you will be too lost to recover and have wasted thousands of dollars in tuition.

So you're really asking ... "How do I architect and build my entire program?"

Would someone show me how to build my program? - Read this article

We have an area where you can post this as a job for hire. Would you like the thread moved there?

Quote

2:How to make it save the results.

[*]Q: ... save data, save properties, save environmental variables, serialize my data/class?
A:

• Separating data from GUI - PLUS - serializing the data to XML
  
• Working with environmental variables
  
• Writing a text file tutorial.
  
• Reading a text file tutorial.
  
• Easy Application Settings example

No, i do not what this as a job for hire, i really hope that this will help me improve my coding skills. I know how to architect my program, i just wanted some "tips" on how my questions. Thanks a lot for your answers on my second question, it was really helpfull, i can, at least, get started now />.

andrewsw, on 16 February 2013 - 01:16 PM, said:

You can either use the Google Search API (usage free up to 100 searches per day) or load the Google search page into a WebBrowser control and parse the HTML content of the page.

Ill get a try with the google API, thanks a lot.

torind_2000, on 16 February 2013 - 01:18 PM, said:

First, you need to learn how to do basic research.
This is the first thing I googled in regards to your topic.
Link

Also, as you've no doubt been told before, we aren't here to write your programs. We're only here to provide you guidance with problems in your code.
You must provide us YOUR code along with any changes you've made to get your code to function so that we can see what you have tried.
Only then can our experts help you.

Now, if you'd like to hire one of the experts here write your program for you, I'm sure one or more of them would be willing to discuss financial compensation for their coding services.

I really dont want anyone to make this for me, i had the project idea to help me improve my programming skiils, ill start coding then, using the API and the saving methods that tlhIn`toq showed.

This post has been edited by Raminator: 16 February 2013 - 06:06 PM

[Raminator]

So, after doing quite some research I was not able to set up my program to use googles API, I'm using this guide to help me, but i just can't set the web service, it always returns an error (404)

Quote

There was an error downloading 'http://api.google.com/GoogleSearch.wsdl/_vti_bin/ListData.svc/$metadata'.
The request failed with HTTP status 404: Not Found.
Metadata contains a reference that cannot be resolved: 'http://api.google.com/GoogleSearch.wsdl'.
There was no endpoint listening at http://api.google.co...ogleSearch.wsdl that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.
The remote server returned an error: (404) Not Found.
If the service is defined in the current solution, try building the solution and adding the service reference again.

what am i doing wrong? I have set up all the google API things, account, turning the custom search on and everything else.

[andrewsw]

That guide is for Visual Basic .NET 2003; i.e. 10 years old.

It is a good job that you only spent an hour on this . Although I'm not sure how you could fit "quite some research" in that same hour?!

[andrewsw]

If it were me, and I only wanted the first four search results, I would probably instantiate a WebBrowser control; the results' main details are in H3 tags, containing an A-link.

I would also try using the Navigating event to try and reference the fourth H3-heading. If and when I am able to do this I would cancel the browser-request as I have access to all the information I need.

Good luck.

[Raminator]

andrewsw, on 17 February 2013 - 01:48 AM, said:

That guide is for Visual Basic .NET 2003; i.e. 10 years old.

It is a good job that you only spent an hour on this />. Although I'm not sure how you could fit "quite some research" in that same hour?!

I made the GUI and wrote down my ideas and some logic steps for the program, then i started to do research on how to use the google API, wich i failed. Ill try your method

[Raminator]

So, i just managed to use the google API by using a dll file i found on the internet, this is how the code looks.

Imports Google.API.Search
Public Class Form1

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

    End Sub

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Dim search As New GwebSearchClient("")
        Dim results As IList(Of IWebResult) = search.Search(TextBox1.Text, 4, Nothing, Nothing, Nothing, Nothing, Nothing)
        ListBox1.Items.Clear()
        For Each result As IWebResult In results
            ListBox1.Items.Add(result.Title & result.Content & result.Url)
        Next
    End Sub

    Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click

    End Sub
End Class

So, i move to the second step, that is being harder then i thought. I found this guide for saving images from a url in vb.net that ill try to use and make some chnages and see what happens. I did some research but found no articles on how to actually save that web page on your pc.

[no2pencil]

Raminator, on 17 February 2013 - 10:23 AM, said:

I did some research but found no articles on how to actually save that web page on your pc.

Don't forget that you will only be save what the browser displays. Any server side logic (such as user authentication or database access) is not accessible on the client side.

Google searches + Web site storage, it sounds like you are building a web scrape program.

If you are looking for 'storing results', that is different than 'storing pages'.

[Raminator]

no2pencil, on 17 February 2013 - 09:37 AM, said:

Raminator, on 17 February 2013 - 10:23 AM, said:

I did some research but found no articles on how to actually save that web page on your pc.

Don't forget that you will only be save what the browser displays. Any server side logic (such as user authentication or database access) is not accessible on the client side.

Google searches + Web site storage, it sounds like you are building a web scrape program.

If you are looking for 'storing results', that is different than 'storing pages'.

I actually mean to store the page itself, for exemple if i searched for battlefield 3 aftermath one of the first 4 results is this page i would like to save this HTML page on the computer.

This post has been edited by Raminator: 17 February 2013 - 11:13 AM

[andrewsw]

Saving the page itself, rather than the search results' links, is a completely different proposition. You could use the WebRequest or HttpWebRequest class to retrieve and save the HTML content. This is useful if you are just interested in the text-content of the page.

If you want the page to be functional then you would need to also download and save all associated stylesheets, script files, and images; but there may be other files that you need to download to make the page fully functional, some of which may not be available for download.

Added: Nevertheless, I would still save the links first, cancelling the Google page when I had retrieved the links I need.

This post has been edited by andrewsw: 17 February 2013 - 11:56 AM

[Raminator]

I made a LOT of progress and i would like to share it with you guys, this is how the code looks now:

'Needs references for: Microsoft CDO For Windows 2000 Library, Microsoft ActiveX Data Objects 2.5 Library and a google search dll (not sure how to share it with you guys, i cant upload that type of file)
Imports Google.API.Search
Imports System.IO
Imports System.Net
Imports System.Drawing
Imports ADODB
Imports CDO
Public Class Form1
    Dim surl(5) As String
    Dim sname(5) As String
    Dim Counter As Integer = 0
    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        'This is just a feature i plan to add later, where the user can choose between web or image searcg
        ComboBox1.Items.Add("Google Images")
        ComboBox1.Items.Add("Google Web")
        ComboBox1.SelectedItem = "Google Web"
    End Sub

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Dim search As New GwebSearchClient("")
        Dim results As IList(Of IWebResult) = search.Search(TextBox1.Text, 4, Nothing, Nothing, Nothing, Nothing, Nothing)
        ListBox1.Items.Clear()
        For Each result As IWebResult In results
            ListBox1.Items.Add(result.Title & result.Content & result.Url)
            Counter += 1
            surl(Counter) = result.Url
            sname(Counter) = result.Title
        Next
    End Sub

    Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
        checkfolder()
        For c As Integer = 1 To 4
            SavePage(surl(c), String.Format("C:/GSearchPro/{0}.mht", sname(c)))
        Next
    End Sub
    Function checkfolder() As Integer
        If Not Directory.Exists("C:/GSearchPro") Then
            Directory.CreateDirectory("C:/GSearchPro")
        Else
        End If
    End Function
    Function GetImage(ByVal URL As String) As System.Drawing.Image
        Dim Request As System.Net.HttpWebRequest
        Dim Response As System.Net.HttpWebResponse
        Request = System.Net.WebRequest.Create(URL)
        Response = CType(Request.GetResponse, System.Net.WebResponse)
        If Request.HaveResponse Then
            If Response.StatusCode = Net.HttpStatusCode.OK Then
                GetImage = System.Drawing.Image.FromStream(Response.GetResponseStream)
            End If
        End If
        Try
        Catch e As System.Net.WebException
            MsgBox("A web exception has occured [" & URL & "]." & vbCrLf & " System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
            Exit Try
        Catch e As System.Net.ProtocolViolationException
            MsgBox("A protocol violation has occured [" & URL & "]." & vbCrLf & "  System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
            Exit Try
        Catch e As System.Net.Sockets.SocketException
            MsgBox("Socket error [" & URL & "]." & vbCrLf & "  System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
            Exit Try
        Catch e As System.IO.EndOfStreamException
            MsgBox("An IO stream exception has occured. System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
            Exit Try
        Finally
        End Try
    End Function
    Private Sub SavePage(ByVal Url As String, ByVal FilePath As String)
        Dim iMessage As CDO.Message = New CDO.Message
        iMessage.CreateMHTMLBody(Url, _
        CDO.CdoMHTMLFlags.cdoSuppressNone, "", "")
        Dim adodbstream As ADODB.Stream = New ADODB.Stream
        adodbstream.Type = ADODB.StreamTypeEnum.adTypeText
        adodbstream.Charset = "US-ASCII"
        adodbstream.Open()
        iMessage.DataSource.SaveToObject(adodbstream, "_Stream")
        adodbstream.SaveToFile(FilePath, _
                  ADODB.SaveOptionsEnum.adSaveCreateOverWrite)
    End Sub
End Class

So, any suggestions on my code? Anything i could make better? Im going to start to work on the google images/web selection now.

[andrewsw]

Eek, are you leaving your form, and control-names as Form1, etc?!

There is nothing in your Try so none of the Catches will catch anything, and all those Exit Try's are redundant. Your Finally is empty as well and could be deleted if you don't plan to use it.

If you are expecting from a variety of errors then I would add Catch e As Exception at the end to catch any unexpected system error.

I might push all those error messages through a helper function, passing the error message and (optional) URL. Just a matter of style I suppose, but it would make the code slightly easier to read.

Just some suggestions