Making a "googler" program.

  • (4 Pages)
  • +
  • « First
  • 2
  • 3
  • 4

53 Replies - 2086 Views - Last Post: 28 February 2013 - 10:11 AM Rate Topic: -----

#46 Raminator  Icon User is offline

  • D.I.C Regular

Reputation: 0
  • View blog
  • Posts: 283
  • Joined: 16-July 12

Re: Making a "googler" program.

Posted 24 February 2013 - 06:26 PM

View Postandrewsw, on 25 February 2013 - 12:44 AM, said:

Print out the values of URL and FilePath just before these lines so that you can check that they are correctly formed.

In the URL case, as i sad, it is an empty variable, thats becase the save loop is looping more then it should (i believe), i wasnt able to fix it at all.
The FilePath case the location seems to be fine, maybe its a bug caused by unsupported characters? How could i limit it to letters and numbers and the other supported characters (if thats the bug).
Was This Post Helpful? 0
  • +
  • -

#47 Raminator  Icon User is offline

  • D.I.C Regular

Reputation: 0
  • View blog
  • Posts: 283
  • Joined: 16-July 12

Re: Making a "googler" program.

Posted 25 February 2013 - 02:11 PM

So, andrewsw? Anyone?
Was This Post Helpful? 0
  • +
  • -

#48 andrewsw  Icon User is offline

  • Fire giant boob nipple gun!
  • member icon

Reputation: 2885
  • View blog
  • Posts: 9,576
  • Joined: 12-December 12

Re: Making a "googler" program.

Posted 25 February 2013 - 02:37 PM

Step through your code examining what happens to your variables and arrays at each point, or print out lots of these values. You can use Debug.Print() to send information to the Immediate window.

You need to trace:
What happens to your variables at each stage;
What do your arrays store (at each stage) and what are their sizes;
What are the values URL and FilePath (at each stage) and do they suddenly change at the point that your code fails.

A possibility is that the WebRequest is getting blocked, but I believe that it runs synchronously by default. That is, your code pauses until the request is complete, so I would concentrate on checking your variables and arrays firstly.

Invest the time to improve your debugging knowledge - you will learn a lot and gain a valuable skill!

Likely causes are that you are trying to read beyond the current length of an array; or trying to read an array elements' value that hasn't been set (defaulting, for a string array, to ""); or your array indexing doesn't progress from 0 to array.length-1.
Was This Post Helpful? 1
  • +
  • -

#49 andrewsw  Icon User is offline

  • Fire giant boob nipple gun!
  • member icon

Reputation: 2885
  • View blog
  • Posts: 9,576
  • Joined: 12-December 12

Re: Making a "googler" program.

Posted 25 February 2013 - 02:49 PM

BTW your function checkfile() doesn't do much; it creates, and returns, a value which is simply discarded.
Was This Post Helpful? 1
  • +
  • -

#50 Raminator  Icon User is offline

  • D.I.C Regular

Reputation: 0
  • View blog
  • Posts: 283
  • Joined: 16-July 12

Re: Making a "googler" program.

Posted 25 February 2013 - 05:47 PM

View Postandrewsw, on 25 February 2013 - 09:37 PM, said:

Step through your code examining what happens to your variables and arrays at each point, or print out lots of these values. You can use Debug.Print() to send information to the Immediate window.

You need to trace:
What happens to your variables at each stage;
What do your arrays store (at each stage) and what are their sizes;
What are the values URL and FilePath (at each stage) and do they suddenly change at the point that your code fails.

A possibility is that the WebRequest is getting blocked, but I believe that it runs synchronously by default. That is, your code pauses until the request is complete, so I would concentrate on checking your variables and arrays firstly.

Invest the time to improve your debugging knowledge - you will learn a lot and gain a valuable skill!

Likely causes are that you are trying to read beyond the current length of an array; or trying to read an array elements' value that hasn't been set (defaulting, for a string array, to ""); or your array indexing doesn't progress from 0 to array.length-1.

Ok, so, i fixed the save webpage button by simply addint this chunk of code(the if statement)
        If Servicebox.SelectedItem = "Google Web" Then
            For c As Integer = 0 To Numberbox.SelectedItem
                ' checkfile(sname(c))
                If (String.IsNullOrEmpty(surl(c))) Then
                    MessageBox.Show("Empty surl(c) String")
                    Exit For
                End If
                SavePage(surl(c), String.Format("C:/GSearchPro/{0}.mht", sname(c)))
            Next

Now I'll focus on fixing the damn save image bug, I'll post an update as soon as i have somthing.
Ps.Instead of using Debug.Print() I useMessageBox.Show, that is also fine, right?
Was This Post Helpful? 0
  • +
  • -

#51 Raminator  Icon User is offline

  • D.I.C Regular

Reputation: 0
  • View blog
  • Posts: 283
  • Joined: 16-July 12

Re: Making a "googler" program.

Posted 26 February 2013 - 10:13 AM

So, i fixed the Google Image bug by adding a line similar to the one that solved the Google Web bug. Ill post the whole code at the end.
So, as a starter programmer i know in a few months or years i'll look at this code and think "i was obviously half retarded". I wanted to know, is there any improvments in efficiency or security or anything you guys think i could do?
Thanks a lot for all the help in this project, speccialy to andrewsw.
Imports Google.API.Search
Imports System.IO
Imports System.Net
Imports System.Drawing
Imports ADODB
Imports CDO
Imports System.Numerics
Public Class GSearchPro
    Dim surl(64) As String
    Dim sname(64) As String
    Dim Counter As Integer = 0
    Dim count As Integer
    Private Sub GSearchPro_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        Servicebox.Items.Add("Google Web")
        Servicebox.Items.Add("Google Images")
        Servicebox.SelectedItem = "Google Web"
        For count = 1 To 64
            Numberbox.Items.Add(count)
        Next
        Numberbox.SelectedItem = 4
    End Sub
    Private Sub Searchbutton_Click(sender As Object, e As EventArgs) Handles Searchbutton.Click
        Servicebox.Enabled = False
        Numberbox.Enabled = False
        If Servicebox.SelectedItem = "Google Web" Then
            Dim search As New GwebSearchClient("")
            Dim results As IList(Of IWebResult) = search.Search(Searchtextbox.Text, Numberbox.SelectedItem, Nothing, Nothing, Nothing, Nothing, Nothing)
            Resultsbox.Items.Clear()
            Counter = 0
            For Each result As IWebResult In results
                Resultsbox.Items.Add(result.Title)
                surl(Counter) = result.Url
                sname(Counter) = result.Title
                Counter += 1
            Next
        Else
            Dim search As New GimageSearchClient("")
            Dim results As IList(Of IImageResult) = search.Search(Searchtextbox.Text, Numberbox.SelectedItem, Nothing, Nothing, Nothing, Nothing, Nothing)
            Resultsbox.Items.Clear()
            Counter = 0
            For Each result As IImageResult In results
                Resultsbox.Items.Add(result.Title)
                surl(Counter) = result.Url
                sname(Counter) = result.Title
                Counter += 1
                If Counter = Numberbox.SelectedItem Then Exit For
            Next
        End If
    End Sub
    Private Sub Savebutton_Click(sender As Object, e As EventArgs) Handles Savebutton.Click
        checkfolder()
        If Servicebox.SelectedItem = "Google Web" Then
            For c As Integer = 0 To Numberbox.SelectedItem
                If (String.IsNullOrEmpty(surl(c))) Then
                    ' MessageBox.Show("Empty surl(c) String")
                    Exit For
                End If
                SavePage(surl(c), String.Format("C:/GSearchPro/{0}.mht", sname(c)))
            Next
        Else
            Dim MyImage As System.Drawing.Image
            Dim FileName As String, URLpieces As String()
            Dim location As String = "C:/GSearchPro/"
            For c2 As Integer = 0 To Numberbox.SelectedItem
                URLpieces = Split(surl(c2), "/")
                FileName = location & URLpieces.GetValue(UBound(URLpieces))
                If (String.IsNullOrEmpty(surl(c2))) Then
                    ' MessageBox.Show("Empty surl(c2) String")
                    Exit For
                End If
                MyImage = GetImage(surl(c2))
                MyImage.Save(FileName)
                MyImage = Nothing
                If c2 = Numberbox.SelectedItem Then Exit For
            Next
        End If
        Servicebox.Enabled = True
        Numberbox.Enabled = True
    End Sub
    Private Sub checkfolder()
        If Not Directory.Exists("C:/GSearchPro") Then
            Directory.CreateDirectory("C:/GSearchPro")
        Else
        End If
    End Sub
    Function GetImage(ByVal URL As String) As System.Drawing.Image
        Dim Request As System.Net.HttpWebRequest
        Dim Response As System.Net.HttpWebResponse
        Try
            Request = System.Net.WebRequest.Create(URL)
            Response = CType(Request.GetResponse, System.Net.WebResponse)
            If Request.HaveResponse Then
                If Response.StatusCode = Net.HttpStatusCode.OK Then
                    GetImage = System.Drawing.Image.FromStream(Response.GetResponseStream)
                End If
            End If
        Catch e As System.Net.WebException
            MsgBox("A web exception has occured [" & URL & "]." & vbCrLf & " System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        Catch e As System.Net.ProtocolViolationException
            MsgBox("A protocol violation has occured [" & URL & "]." & vbCrLf & "  System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        Catch e As System.Net.Sockets.SocketException
            MsgBox("Socket error [" & URL & "]." & vbCrLf & "  System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        Catch e As System.IO.EndOfStreamException
            MsgBox("An IO stream exception has occured. System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        End Try
    End Function
    Private Sub SavePage(ByVal Url As String, ByVal FilePath As String)
        Dim iMessage As CDO.Message = New CDO.Message
        iMessage.CreateMHTMLBody(Url, CDO.CdoMHTMLFlags.cdoSuppressNone, "", "")
        Dim adodbstream As ADODB.Stream = New ADODB.Stream
        adodbstream.Type = ADODB.StreamTypeEnum.adTypeText
        adodbstream.Charset = "US-ASCII"
        adodbstream.Open()
        iMessage.DataSource.SaveToObject(adodbstream, "_Stream")
        adodbstream.SaveToFile(FilePath, ADODB.SaveOptionsEnum.adSaveCreateOverWrite)
    End Sub
    Private Sub Resultsbox_SelectedIndexChanged(sender As Object, e As EventArgs) Handles Resultsbox.SelectedIndexChanged

    End Sub
    Private Sub Servicebox_SelectedIndexChanged(sender As Object, e As EventArgs) Handles Servicebox.SelectedIndexChanged

    End Sub
End Class


Was This Post Helpful? 0
  • +
  • -

#52 lar3ry  Icon User is offline

  • Coding Geezer
  • member icon

Reputation: 310
  • View blog
  • Posts: 1,290
  • Joined: 12-September 12

Re: Making a "googler" program.

Posted 26 February 2013 - 12:55 PM

Quote

Ps.Instead of using Debug.Print() I useMessageBox.Show, that is also fine, right?

Absolutely! I mostly use debug.print, but will often make an extra TextBox and use that. A MessageBox is as good a way as any. Of course, you stall the program until you dismiss the MessageBox, but if that doesn't matter, go for it.

The important thing is to show what's happening at a particular place in the code. If I have more than one place I want to display data from, I'll use something to identify it, like prefacing it with "--" or "==", etc. As well, don't forget that if you are looping, it helps to put the loop counter (if any) out there too, so your output might look like:

0 Andy
1 Bernardo
2 Ching
Was This Post Helpful? 1
  • +
  • -

#53 Raminator  Icon User is offline

  • D.I.C Regular

Reputation: 0
  • View blog
  • Posts: 283
  • Joined: 16-July 12

Re: Making a "googler" program.

Posted 26 February 2013 - 12:58 PM

View Postlar3ry, on 26 February 2013 - 07:55 PM, said:

Quote

Ps.Instead of using Debug.Print() I useMessageBox.Show, that is also fine, right?

Absolutely! I mostly use debug.print, but will often make an extra TextBox and use that. A MessageBox is as good a way as any. Of course, you stall the program until you dismiss the MessageBox, but if that doesn't matter, go for it.

The important thing is to show what's happening at a particular place in the code. If I have more than one place I want to display data from, I'll use something to identify it, like prefacing it with "--" or "==", etc. As well, don't forget that if you are looping, it helps to put the loop counter (if any) out there too, so your output might look like:

0 Andy
1 Bernardo
2 Ching

Thanks lar3ry!
Was This Post Helpful? 0
  • +
  • -

#54 Raminator  Icon User is offline

  • D.I.C Regular

Reputation: 0
  • View blog
  • Posts: 283
  • Joined: 16-July 12

Re: Making a "googler" program.

Posted 28 February 2013 - 10:11 AM

I have been thinking to change the webpage format from MHT to HTM, because more browsers support HTM and it displays webpages MUCH better, does anyone know how to do this and can help me out? (changing the extention to .htm doesn't work btw)
Full Code:
Imports Google.API.Search
Imports System.IO
Imports System.Net
Imports System.Drawing
Imports ADODB
Imports CDO
Public Class GSearchPro
    Dim surl(64) As String
    Dim sname(64) As String
    Dim Counter As Integer = 0
    Dim count As Integer
    Private Sub GSearchPro_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        Servicebox.Items.Add("Google Web")
        Servicebox.Items.Add("Google Images")
        Servicebox.SelectedItem = "Google Web"
        For count = 1 To 64
            Numberbox.Items.Add(count)
        Next
        Numberbox.SelectedItem = 4
    End Sub
    Private Sub Searchbutton_Click(sender As Object, e As EventArgs) Handles Searchbutton.Click
        Servicebox.Enabled = False
        Numberbox.Enabled = False
        If Servicebox.SelectedItem = "Google Web" Then
            Dim search As New GwebSearchClient("")
            Dim results As IList(Of IWebResult) = search.Search(Searchtextbox.Text, Numberbox.SelectedItem, Nothing, Nothing, Nothing, Nothing, Nothing)
            Resultsbox.Items.Clear()
            Counter = 0
            For Each result As IWebResult In results
                Resultsbox.Items.Add(result.Title)
                surl(Counter) = result.Url
                sname(Counter) = result.Title
                Counter += 1
            Next
        Else
            Dim search As New GimageSearchClient("")
            Dim results As IList(Of IImageResult) = search.Search(Searchtextbox.Text, Numberbox.SelectedItem, Nothing, Nothing, Nothing, Nothing, Nothing)
            Resultsbox.Items.Clear()
            Counter = 0
            For Each result As IImageResult In results
                Resultsbox.Items.Add(result.Title)
                surl(Counter) = result.Url
                sname(Counter) = result.Title
                Counter += 1
                If Counter = Numberbox.SelectedItem Then Exit For
            Next
        End If
    End Sub
    Private Sub Savebutton_Click(sender As Object, e As EventArgs) Handles Savebutton.Click
        checkfolder()
        If Servicebox.SelectedItem = "Google Web" Then
            For c As Integer = 0 To Numberbox.SelectedItem
                If (String.IsNullOrEmpty(surl(c))) Then
                    ' MessageBox.Show("Empty surl(c) String")
                    Exit For
                End If
                SavePage(surl(c), String.Format("C:/GSearchPro/{0}.mht", sname(c)))
            Next
        Else
            Dim MyImage As System.Drawing.Image
            Dim FileName As String, URLpieces As String()
            Dim location As String = "C:/GSearchPro/"
            For c2 As Integer = 0 To Numberbox.SelectedItem
                URLpieces = Split(surl(c2), "/")
                FileName = location & URLpieces.GetValue(UBound(URLpieces))
                If (String.IsNullOrEmpty(surl(c2))) Then
                    ' MessageBox.Show("Empty surl(c2) String")
                    Exit For
                End If
                MyImage = GetImage(surl(c2))
                MyImage.Save(FileName)
                MyImage = Nothing
                If c2 = Numberbox.SelectedItem Then Exit For
            Next
        End If
        Servicebox.Enabled = True
        Numberbox.Enabled = True
    End Sub
    Private Sub checkfolder()
        If Not Directory.Exists("C:/GSearchPro") Then
            Directory.CreateDirectory("C:/GSearchPro")
        Else
        End If
    End Sub
    Function GetImage(ByVal URL As String) As System.Drawing.Image
        Dim Request As System.Net.HttpWebRequest
        Dim Response As System.Net.HttpWebResponse
        Try
            Request = System.Net.WebRequest.Create(URL)
            Response = CType(Request.GetResponse, System.Net.WebResponse)
            If Request.HaveResponse Then
                If Response.StatusCode = Net.HttpStatusCode.OK Then
                    GetImage = System.Drawing.Image.FromStream(Response.GetResponseStream)
                End If
            End If
        Catch e As System.Net.WebException
            MsgBox("A web exception has occured [" & URL & "]." & vbCrLf & " System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        Catch e As System.Net.ProtocolViolationException
            MsgBox("A protocol violation has occured [" & URL & "]." & vbCrLf & "  System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        Catch e As System.Net.Sockets.SocketException
            MsgBox("Socket error [" & URL & "]." & vbCrLf & "  System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        Catch e As System.IO.EndOfStreamException
            MsgBox("An IO stream exception has occured. System returned: " & e.Message, MsgBoxStyle.Exclamation, "Error!")
        End Try
    End Function
    Private Sub SavePage(ByVal Url As String, ByVal FilePath As String)
        Dim iMessage As CDO.Message = New CDO.Message
        iMessage.CreateMHTMLBody(Url, CDO.CdoMHTMLFlags.cdoSuppressNone, "", "")
        Dim adodbstream As ADODB.Stream = New ADODB.Stream
        adodbstream.Type = ADODB.StreamTypeEnum.adTypeText
        adodbstream.Charset = "US-ASCII"
        adodbstream.Open()
        iMessage.DataSource.SaveToObject(adodbstream, "_Stream")
        adodbstream.SaveToFile(FilePath, ADODB.SaveOptionsEnum.adSaveCreateOverWrite)
    End Sub
    Private Sub Resultsbox_SelectedIndexChanged(sender As Object, e As EventArgs) Handles Resultsbox.SelectedIndexChanged

    End Sub
    Private Sub Servicebox_SelectedIndexChanged(sender As Object, e As EventArgs) Handles Servicebox.SelectedIndexChanged

    End Sub
End Class


Was This Post Helpful? 0
  • +
  • -

  • (4 Pages)
  • +
  • « First
  • 2
  • 3
  • 4