Hello,

I've wanted to create a web filter of my own for a long time. I feel like there are so much bad things that a child could be exposed to that I wanted to do something to help out.

My problem is a really don't know where to start. Unfortunately, I have a handle of the basics of languages like C, C++, VB, HTML, and PHP, but I don't really know how to do anything advanced.

I've built websites and stuff and done basics executable programs but nothing this intense.

I would really appreciate it if any of you could help me get started?

What language would be preferred for this software?
Could I create an online app version of this?

Any and all help would be very much appreciated.

Skinny

This post has been edited by Skinny: 16 Nov, 2007 - 07:08 PM

<snip>


Hey Skinny-

Internet filters are great. I use one here in the house to not only block porn, but to block ad websites, and sites that could be harmful. I run a squid proxy server on my Debian based router. It's been quite convenient. All the code for Squid was written in C. However, there are some things I think you should consider first.

I think you should probably read up on the OSI model, and learn how a network works. You could implement a proxy at many different levels, including the Application level, which would apply to your browser. There's a Firefox extension called AdBlock that does just this. I can think of a few ways to implement a proxy through plain ol' javascript. My squid proxy is a layer 3/4 proxy (depending on where TCP/IP fits, which is debatable). When you decide which layer you'd like implement your filter, than you can start considering languages to write it in.

The higher level your proxy is, the easier it is to bypass, and the higher level language you can use. The lower level it is, the lower level the language is, and the harder it is to bypass. My squid proxy has firewall rules set up that pretty much FORCE you to go through the proxy or nothing at all.

rockstar

Thanks for responding.

I'm familiar with the OSI model (I'm taking Electronics and I studied briefly Networking which discussed the OSI model).

I will read into that more.

So do these filters operate by reading packets that are coming in through the internet connection or do they operate by checking to see if a URL entered isn't harmful (by checking a database) or a combo of both?

Skinny

This post has been edited by Skinny: 17 Nov, 2007 - 09:49 AM

The most common implementation is a basic proxy server with a url blacklist. These used to work before kids stared getting smart. The same mechanism is still used, but the application of the blacklist is more complex.

The best blockers are actually intrusive routers. They open up http packets, make sure the http header isn't bogus, and then, just to be safe, rewrap the pay load and send it on.

We use an interesting one at work, a combination of a firewall and a content filter. The firewall has a rule for http traffic that has it ask the filter. The filter is stateful and either asks the user for validation or uses current authentication to figure out identity. Either way it creates a session to track the user. It then checks the user against the blacklist and ultimately tells the firewall whether to allow the packet through or not.

Note, a proxy server is little more than an http server. It passes the requests to the internet and passes back the responses. A filtering proxy, because of it's position in the communication channel, just chooses never to pass on some requests.

I mention this because http servers are surprisingly easy to write.

Hope this helps.

More importantly, which OS are you looking to code this for? There are lots of firewall programs for *nix, IPFW, IPChains, IPTables, ect. However for Windows, most people just assume that they are protected. Plus you'll have to fight with turning off the Windows firewall, if this is the direction that you take. You'll also want to have a look at the registry entry that controls the order of which domain names are resolved.

[offtopic]
Do you really think these filters will ever work?
If I think of my childhood I am not so optimistic. We could always circumvent whatever filters or limitations were in question (I'm not just talking about the internet here). I think some of the problems could be dealt with education (when your kid searches for these things - because (s)he will get it, at least you won't know about it, that is not a better option) and others by going and hitting the perv on the head with a shovel (when the content finds him/her - via msn, myspace, etc.) - with the shovel I meant the law, of course.
[/offtopic - if you want to reply to this, continue in the lounge, please]

Anyway, you can simply block websites by redirecting them to localhost in the etc/hosts file (both on *nix and windows systems) - maybe with some write protection on the file. You can't do serious filtering with this, only blocking whole domains is possible, and accessing them by ip addresses is still possible I think.

No. They work the way all protection works, they prevent the casual user, but not the truly determined.

The trick is actually not the filtering itself; if it's in the blacklist, block it! The problem is a function of scale. Sure, you can block playboy.com, but what about the random site that just mirrors some of their favorite porn and only about ten people know about it? And a new proxy comes up every day. It's computer whack-a-mole and a losing proposition.

I believe, rather than blocking, auditing should be the norm for those who would control such behavior. Never block, but sometimes fire off a message that says something like, "This is on the restricted list and your actions have been logged! You will not always see this message, but be aware that all request are recorded."

Then, if it's bad, come to the infringer with an activity log and talk to them about it. Have consequences beyond merely, oops, you can't do that. This has a chance of changing behavior. Simply blocking is merely a challenge to be overcome.