
Load a random http proxy from .txt file
Page 1 of 114 Replies - 1328162 Views - Last Post: 25 November 2009 - 08:28 PM
#1
Load a random http proxy from .txt file
Posted 24 May 2009 - 10:46 AM

Replies To: Load a random http proxy from .txt file
#2
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 11:07 AM
#3
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 11:18 AM

#4
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 11:37 AM
#5
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 11:42 AM
#6
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 11:44 AM
Imports System.Runtime.InteropServices
<Runtime.InteropServices.DllImport("wininet.dll", SetLastError:=True)> _ Private Shared Function InternetSetOption(ByVal hInternet As IntPtr, ByVal dwOption As Integer, ByVal lpBuffer As IntPtr, ByVal lpdwBufferLength As Integer) As Boolean End Function Public Structure Struct_INTERNET_PROXY_INFO Public dwAccessType As Integer Public proxy As IntPtr Public proxyBypass As IntPtr End Structure
Private Sub RefreshIESettings(ByVal strProxy As String) Const INTERNET_OPTION_PROXY As Integer = 38 Const INTERNET_OPEN_TYPE_PROXY As Integer = 3 Dim struct_IPI As Struct_INTERNET_PROXY_INFO ' Filling in structure struct_IPI.dwAccessType = INTERNET_OPEN_TYPE_PROXY struct_IPI.proxy = Marshal.StringToHGlobalAnsi(strProxy) struct_IPI.proxyBypass = Marshal.StringToHGlobalAnsi("local") ' Allocating memory Dim intptrStruct As IntPtr = Marshal.AllocCoTaskMem(Marshal.SizeOf(struct_IPI)) ' Converting structure to IntPtr Marshal.StructureToPtr(struct_IPI, intptrStruct, True) Dim iReturn As Boolean = InternetSetOption(IntPtr.Zero, INTERNET_OPTION_PROXY, intptrStruct, System.Runtime.InteropServices.Marshal.SizeOf(struct_IPI)) End Sub
Make a button that changes the proxy?
RefreshIESettings("Proxy url")
This post has been edited by Joshdw: 24 May 2009 - 11:45 AM
#7
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 12:03 PM
#8
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 12:43 PM
Imports System.Runtime.InteropServices
Goes Above the Private Class form, top of everything
The
<Runtime.InteropServices.DllImport("wininet.dll", SetLastError:=True)> _ Private Shared Function InternetSetOption(ByVal hInternet As IntPtr, ByVal dwOption As Integer, ByVal lpBuffer As IntPtr, ByVal lpdwBufferLength As Integer) As Boolean End Function Public Structure Struct_INTERNET_PROXY_INFO Public dwAccessType As Integer Public proxy As IntPtr Public proxyBypass As IntPtr End Structure
Private Sub RefreshIESettings(ByVal strProxy As String) Const INTERNET_OPTION_PROXY As Integer = 38 Const INTERNET_OPEN_TYPE_PROXY As Integer = 3 Dim struct_IPI As Struct_INTERNET_PROXY_INFO ' Filling in structure struct_IPI.dwAccessType = INTERNET_OPEN_TYPE_PROXY struct_IPI.proxy = Marshal.StringToHGlobalAnsi(strProxy) struct_IPI.proxyBypass = Marshal.StringToHGlobalAnsi("local") ' Allocating memory Dim intptrStruct As IntPtr = Marshal.AllocCoTaskMem(Marshal.SizeOf(struct_IPI)) ' Converting structure to IntPtr Marshal.StructureToPtr(struct_IPI, intptrStruct, True) Dim iReturn As Boolean = InternetSetOption(IntPtr.Zero, INTERNET_OPTION_PROXY, intptrStruct, System.Runtime.InteropServices.Marshal.SizeOf(struct_IPI)) End Sub
Will go undernieth Private Class Form..
And then, you could do,
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load RefreshIESettings("Proxy url") End Sub
The RefreshIESettings is the part where it sets the proxy to the link you give it, so basiclly you could do a Select Case, and it could select a random proxy

Anyway hopefully that helps?
The webbrowser, it doesnt matter where you put it
#9
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 01:45 PM
#10
Re: Load a random http proxy from .txt file
Posted 24 May 2009 - 01:46 PM

#11
Re: Load a random http proxy from .txt file
Posted 25 November 2009 - 02:47 AM
Joshdw, on 24 May, 2009 - 12:46 PM, said:

I've been looking my self on how to do this. But all the codes are vb6 code

So there is no vb.net pointers and hence all the marshal thingy.
Let me try to understand.
In vb6 there is no pointer either, but there is a reference, which is a pointer automatically dereferenced. So by putting the variable to the API it's as good as inputting the address of the pointer.
Hmm.... Why can't we do the same thing for vb.net and change "as any" to "as object"
#12
Re: Load a random http proxy from .txt file
Posted 25 November 2009 - 07:28 AM
.Net has a whole bunch of types that are Blittable - it can whack them from .Net memory to C memory and back without much fuss.
Other types, such as C structures, don't exist in .Net and so you have to define a .Net version and marshal them with a few extra parameters.
The above code is pretty verbose - it is copying strings to memory and back - but Strings are blittable, so it can do those automatically - but you may need to tell it which C type it is. Also, it uses AllocCoTaskMem which is allocating memory for COM interop, but this is platform interop, AllocHGlobal is more suitable. And you must always free the memory that you allocate. That means calling Marshal.FreeCoTaskMem or Marshal.FreeHGlobal after use (preferably in a finally block).
The structure is also copied into memory for the call, but that isn't necessary. If the structure is declared correctly, it is blittable - you just have to pass it ByRef.
Option Strict On Option Explicit On Imports System.Runtime.InteropServices Public Class Form1 ' This could be in an enum, but there are 90ish InternetOptions, and we are not interested in them all.. ' Const INTERNET_OPTION_PROXY As Integer = 38 Friend WithEvents Browser As New WebBrowser With {.Size = New Size(200, 200), .Location = New Point(5, 5)} Friend WithEvents SetProxyButon As New Button With {.Text = "Proxy"} Friend WithEvents NavigateButton As New Button With {.Text = "Navigate"} Friend ProxyLabel As New Label With {.AutoSize = True, .Text = "proxy"} Friend NavigateLabel As New Label With {.AutoSize = True, .Text = "url"} Private proxies() As String = {"148.233.239.24:80", "193.174.155.27:3124", "141.24.249.130:3127"} ' Pulled these off some internet page... ' Private currentProxyIndex As Integer = 0 Private urls() As String = {"www.bbc.co.uk", "www.google.com", "www.wikipedia.com"} Private currentUrlsIndex As Integer = 0 ' Usually it is best to put all the possible values for a flag or option into an enum: ' Enum AccessType INTERNET_OPEN_TYPE_PRECONFIG = 0 INTERNET_OPEN_TYPE_DIRECT = 1 INTERNET_OPEN_TYPE_PROXY = 3 End Enum <DllImport("wininet.dll", SetLastError:=True)> _ Private Shared Function InternetSetOption( _ ByVal hInternet As IntPtr, _ ByVal accessOption As Integer, _ ByRef lpBuffer As INTERNET_PROXY_INFO, _ ByVal lpdwBufferLength As Integer) As Boolean End Function <DllImport("wininet.dll", SetLastError:=True)> _ Private Shared Function InternetQueryOption( _ ByVal hInternet As IntPtr, _ ByVal accessOption As Integer, _ ByVal pInfo As IntPtr, _ ByRef lpdwBufferLength As Integer) As Boolean End Function <StructLayout(LayoutKind.Sequential)> _ Public Structure INTERNET_PROXY_INFO Public dwAccessType As AccessType Public proxy As String Public proxyBypass As String ' A constructor tidied up the interop code a bit, if the fields are few and simple. ' Public Sub New(ByVal accessType As AccessType, ByVal proxy As String, ByVal proxyByPass As String) Me.dwAccessType = accessType Me.proxy = proxy Me.proxyBypass = proxyByPass End Sub ' Sticking a function on for size makes the interop call more readable. ' Public Function Size() As Integer Return Marshal.SizeOf(Me) End Function End Structure Sub New() InitializeComponent() SetProxyButon.Location = New Point(5, Browser.Bottom + 5) NavigateButton.Location = New Point(5, SetProxyButon.Bottom + 5) ProxyLabel.Location = New Point(SetProxyButon.Right + 5, SetProxyButon.Top) NavigateLabel.Location = New Point(NavigateButton.Right + 5, NavigateButton.Top) Me.Controls.AddRange(New Control() {Browser, SetProxyButon, NavigateButton, ProxyLabel, NavigateLabel}) TrySetIEProxy(proxies(currentProxyIndex)) ProxyLabel.Text = GetIEProxyInfo.proxy Browser.Navigate(urls(0)) NavigateLabel.Text = urls(0) End Sub Private Function TrySetIEProxy(ByVal proxy As String) As Boolean Dim info As New INTERNET_PROXY_INFO(AccessType.INTERNET_OPEN_TYPE_PROXY, proxy, "local") Return InternetSetOption(Nothing, INTERNET_OPTION_PROXY, info, info.Size) End Function ' This is a bit more complicated - we dont know the size of the strings... ' Private Function GetIEProxyInfo() As INTERNET_PROXY_INFO Dim size As Integer = 0 ' What is the total size of the structure? ' InternetQueryOption(Nothing, INTERNET_OPTION_PROXY, Nothing, size) Dim pInfo As IntPtr Dim info As INTERNET_PROXY_INFO Try ' Allocate enough room. ' pInfo = Marshal.AllocHGlobal(size) ' Get the structure in memory. ' InternetQueryOption(Nothing, INTERNET_OPTION_PROXY, pInfo, size) ' Cast it back to the structure type. ' info = DirectCast(Marshal.PtrToStructure(pInfo, GetType(INTERNET_PROXY_INFO)), INTERNET_PROXY_INFO) Finally Marshal.FreeHGlobal(pInfo) End Try Return info End Function Private Sub NavigateButton_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles NavigateButton.Click currentUrlsIndex = (currentUrlsIndex + 1) Mod urls.Length Browser.Navigate(urls(currentUrlsIndex)) NavigateLabel.Text = urls(currentUrlsIndex) End Sub Private Sub SetProxyButon_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles SetProxyButon.Click currentProxyIndex = (currentProxyIndex + 1) Mod proxies.Length TrySetIEProxy(proxies(currentProxyIndex)) ProxyLabel.Text = GetIEProxyInfo.proxy End Sub End Class
This post has been edited by crepitus: 25 November 2009 - 07:29 AM
#13
Re: Load a random http proxy from .txt file
Posted 25 November 2009 - 01:34 PM
So you declare the internetsetoption and change the void* thingy straight into INTERNET_PROXY_INFO
Also you set the layout to be sequential to make the struct blitable. Now that make sense. It works.
The code works.
So I changed that a little bit. Rather than just having a webbrowser control, I added shdocvw.internetexplorer control. That's because my whole program automate shdocvw.internetexplorer
I see that the proxy is indeed used for your webbrowser control. However, the proxy is not used in shdocvw.internetexplorer
How do I arrange that shdocvw.internetexplorer object use a particular proxy?
How do the code knows that you need to change the proxy for the webbrowser control?
I tested your code and change google.com to formyip.com. The webbrowser control use proxy. However, all the other internet explorer windows doesn't.
Moreover, As far as I know, function InternetSetOption doesn't exist in wininet.dll . What exist is either InternetSetOptionA or InternetSetOptionW. So how do vb knows which functions to call?
Looks like I am going to declare InternetSetOption several times. One for INTERNET_PROXY_INFO another for system.string so I will use a different style of declaration.l
crepitus, on 25 Nov, 2009 - 06:28 AM, said:
.Net has a whole bunch of types that are Blittable - it can whack them from .Net memory to C memory and back without much fuss.
Other types, such as C structures, don't exist in .Net and so you have to define a .Net version and marshal them with a few extra parameters.
The above code is pretty verbose - it is copying strings to memory and back - but Strings are blittable, so it can do those automatically - but you may need to tell it which C type it is. Also, it uses AllocCoTaskMem which is allocating memory for COM interop, but this is platform interop, AllocHGlobal is more suitable. And you must always free the memory that you allocate. That means calling Marshal.FreeCoTaskMem or Marshal.FreeHGlobal after use (preferably in a finally block).
The structure is also copied into memory for the call, but that isn't necessary. If the structure is declared correctly, it is blittable - you just have to pass it ByRef.
Option Strict On Option Explicit On Imports System.Runtime.InteropServices Public Class Form1 Dim ie As SHDocVw.InternetExplorer ' This could be in an enum, but there are 90ish InternetOptions, and we are not interested in them all.. ' Const INTERNET_OPTION_PROXY As Integer = 38 Friend WithEvents Browser As New WebBrowser With {.Size = New Size(200, 200), .Location = New Point(5, 5)} Friend WithEvents SetProxyButon As New Button With {.Text = "Proxy"} Friend WithEvents NavigateButton As New Button With {.Text = "Navigate"} Friend ProxyLabel As New Label With {.AutoSize = True, .Text = "proxy"} Friend NavigateLabel As New Label With {.AutoSize = True, .Text = "url"} Private proxies() As String = {"148.233.239.24:80", "193.174.155.27:3124", "141.24.249.130:3127"} ' Pulled these off some internet page... ' Private currentProxyIndex As Integer = 0 Private urls() As String = {"http://formyip.com"} Private currentUrlsIndex As Integer = 0 ' Usually it is best to put all the possible values for a flag or option into an enum: ' Enum AccessType INTERNET_OPEN_TYPE_PRECONFIG = 0 INTERNET_OPEN_TYPE_DIRECT = 1 INTERNET_OPEN_TYPE_PROXY = 3 End Enum <DllImport("wininet.dll", SetLastError:=True)> _ Private Shared Function InternetSetOption( _ ByVal hInternet As IntPtr, _ ByVal accessOption As Integer, _ ByRef lpBuffer As INTERNET_PROXY_INFO, _ ByVal lpdwBufferLength As Integer) As Boolean End Function <DllImport("wininet.dll", SetLastError:=True)> _ Private Shared Function InternetQueryOption( _ ByVal hInternet As IntPtr, _ ByVal accessOption As Integer, _ ByVal pInfo As IntPtr, _ ByRef lpdwBufferLength As Integer) As Boolean End Function <StructLayout(LayoutKind.Sequential)> _ Public Structure INTERNET_PROXY_INFO Public dwAccessType As AccessType Public proxy As String Public proxyBypass As String ' A constructor tidied up the interop code a bit, if the fields are few and simple. ' Public Sub New(ByVal accessType As AccessType, ByVal proxy As String, ByVal proxyByPass As String) Me.dwAccessType = accessType Me.proxy = proxy Me.proxyBypass = proxyByPass End Sub ' Sticking a function on for size makes the interop call more readable. ' Public Function Size() As Integer Return Marshal.SizeOf(Me) End Function End Structure Sub New() InitializeComponent() SetProxyButon.Location = New Point(5, Browser.Bottom + 5) NavigateButton.Location = New Point(5, SetProxyButon.Bottom + 5) ProxyLabel.Location = New Point(SetProxyButon.Right + 5, SetProxyButon.Top) NavigateLabel.Location = New Point(NavigateButton.Right + 5, NavigateButton.Top) Me.Controls.AddRange(New Control() {Browser, SetProxyButon, NavigateButton, ProxyLabel, NavigateLabel}) TrySetIEProxy(proxies(currentProxyIndex)) ProxyLabel.Text = GetIEProxyInfo.proxy Browser.Navigate(urls(0)) NavigateLabel.Text = urls(0) ie = New SHDocVw.InternetExplorer ie.Visible = True End Sub Private Function TrySetIEProxy(ByVal proxy As String) As Boolean Dim info As New INTERNET_PROXY_INFO(AccessType.INTERNET_OPEN_TYPE_PROXY, proxy, "local") Return InternetSetOption(Nothing, INTERNET_OPTION_PROXY, info, info.Size) End Function ' This is a bit more complicated - we dont know the size of the strings... ' Private Function GetIEProxyInfo() As INTERNET_PROXY_INFO Dim size As Integer = 0 ' What is the total size of the structure? ' InternetQueryOption(Nothing, INTERNET_OPTION_PROXY, Nothing, size) Dim pInfo As IntPtr Dim info As INTERNET_PROXY_INFO Try ' Allocate enough room. ' pInfo = Marshal.AllocHGlobal(size) ' Get the structure in memory. ' InternetQueryOption(Nothing, INTERNET_OPTION_PROXY, pInfo, size) ' Cast it back to the structure type. ' info = DirectCast(Marshal.PtrToStructure(pInfo, GetType(INTERNET_PROXY_INFO)), INTERNET_PROXY_INFO) Finally Marshal.FreeHGlobal(pInfo) End Try Return info End Function Private Sub NavigateButton_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles NavigateButton.Click currentUrlsIndex = (currentUrlsIndex + 1) Mod urls.Length Browser.Navigate(urls(currentUrlsIndex)) ie.Navigate(urls(currentUrlsIndex)) NavigateLabel.Text = urls(currentUrlsIndex) End Sub Private Sub SetProxyButon_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles SetProxyButon.Click currentProxyIndex = (currentProxyIndex + 1) Mod proxies.Length TrySetIEProxy(proxies(currentProxyIndex)) ProxyLabel.Text = GetIEProxyInfo.proxy End Sub Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load End Sub End Class
This post has been edited by teguh123: 25 November 2009 - 02:01 PM
#14
Re: Load a random http proxy from .txt file
Posted 25 November 2009 - 02:20 PM
teguh123, on 25 Nov, 2009 - 12:34 PM, said:
How do I arrange that shdocvw.internetexplorer object use a particular proxy?
How do the code knows that you need to change the proxy for the webbrowser control?
It works for the current process, it's in the documentation on the msdn. If you are automating an external process, then it's not going to have an effect there. I'm not sure how you would do that - presumably you want to change the proxy for just the target process rather than globally change it.
Quote
That's pretty simple. The two functions are for (A)nsi strings and unicode or (W)ide strings. You can specify for a signature to call one or the other verbosely by setting the CharSet option on a function:
<DllImport("blah.dll", CharSet:=CharSet.Ansi)> ...
By default it uses CharSet.Auto, I think it prefers the Unicode version of a function if there is a choice.
Quote
It's ok to overload the .Net signatures.
msdn: Interop Marshaling
#15
Re: Load a random http proxy from .txt file
Posted 25 November 2009 - 08:28 PM
I am sorry if I didn't make my self clear. Please look at this code
Declare Function GetWindowTextLength Lib "user32" Alias "GetWindowTextLengthA"(ByVal hwnd As Integer) As Integer Declare Function GetWindowText Lib "user32" Alias "GetWindowTextA" (ByVal hwnd As Integer, ByVal lpString As String, ByVal cch As Integer) As Integer Public Function GetCaption(ByVal hwnd As Long) As String Dim sTmp As String Dim lLen As Integer ' get the caption length lLen = GetWindowTextLength(hwnd) + 1 ' prepare the buffer sTmp = Space(lLen) ' fill the buffer GetWindowText(hwnd, sTmp, lLen) ' trim off the null char and return GetCaption = Left(sTmp, Len(sTmp) - 1) End Function
See, we can get information to a string without marshalling. So I think we should be able to do so for internetqueryoptions too somehow. I'll look further into this.
All your samples are great. If only it can work for internet explorer object that's not withing current thread.
---
I can try to find the session handle of the current internet explorer object using internetopen but I can't find a sample in the web.
Curiously, why do you declare
Private Shared Function InternetQueryOption( _ ByVal hInternet As IntPtr, _ ByVal accessOption As Integer, _ ByVal pInfo As IntPtr, _ ByRef lpdwBufferLength As Integer) As Boolean End Function
Why not replace byval pInfo as IntPtr with byref info as INTERNET_PROXY_INFO like usual? If the issue is space allocation, wouldn't specifying byref info as INTERNET_PROXY_INFO solve the problem?
Well, I learned a lot already. Another way to change ie proxy option is to manipulate registry straight. However, my proxies have authorization and everytime it change proxys and restart ie I have to press a stupid ok button to effectively say, yes IE, use that same username and password again. I suppose I can simply automate pressing that stupid button with findwindow api (or some even better .net equivalent)
Hmm.... I suppose I can execute the code in before_navigate2 event in the ie object so it should execute at the same "process" with the ie object. That fails

The thing is my whole program is created to automate internet explorer object. Internet explorer object is easier to debug. For example, if I break my program, I can still go to the ie window and see what's showing up. However If I add an event handler to ie object, ie lose this capability too. So I like to avoid that.
In ancient time, during vb6 era, webbrowser control and ie is used interchangeably. Now they no longer share the same common interface. Things are a little different for each.
crepitus, on 25 Nov, 2009 - 01:20 PM, said:
teguh123, on 25 Nov, 2009 - 12:34 PM, said:
How do I arrange that shdocvw.internetexplorer object use a particular proxy?
How do the code knows that you need to change the proxy for the webbrowser control?
It works for the current process, it's in the documentation on the msdn. If you are automating an external process, then it's not going to have an effect there. I'm not sure how you would do that - presumably you want to change the proxy for just the target process rather than globally change it.
Quote
That's pretty simple. The two functions are for (A)nsi strings and unicode or (W)ide strings. You can specify for a signature to call one or the other verbosely by setting the CharSet option on a function:
<DllImport("blah.dll", CharSet:=CharSet.Ansi)> ...
By default it uses CharSet.Auto, I think it prefers the Unicode version of a function if there is a choice.
Quote
It's ok to overload the .Net signatures.
msdn: Interop Marshaling
This post has been edited by teguh123: 26 November 2009 - 07:28 AM