[O9Tacoma4x4]

I've tried google. I've downloaded a couple ebooks on entry level programming languages. PERL, PHP, Ruby. What the goal of my project is, is to take existing data from Our States prison site, " http://www.drc.ohio.gov/ " which shows, County committed, inmate number, current prison housed in, inmate number, first/last name as well as a few other things. And archive them so that Users of my site would be able to search inhabitants of a particular facility. I'd eventually like to add an interactive map similar to " http://www.bing.com/map " I'm not sure what it would do. Probably point out whos where by commitment County. But first things first.

What I'm asking is, what exactly do I need physically? Obviously a server, and I'd prefer to use opensource software through out. So I'd probably use Ubuntu Server. And what programming languages in particular do I need to study aggressively (the most) as well as semi aggressive (Will need information from, but not so much as to rely on them).

Any help at all is appreciated. And any questions asked for more particular goals, I'll try to answer in a timely manner. Like I said I'm new, I dont know what specifics need to be answered to get a definitive answer.

[macosxnerd101]

Moved to Web Development.

You will want to use a SQL Database for this. MySQL is commonly used, and is free. Also, there are a few common web development languages used today, including PHP, ColdFusion, and ASP.NET. I personally use PHP, but I'm sure the other two are just fine as well. Also, take a look at technologies like WAMP (Windows, Apache, MySQL and PHP) or LAMP (Linux, Apache, MySQL, and PHP).

[Oler1s]

An important question: how are you getting the inmate data, and are you authorized to host and serve that data yourself?

That's an important question with both technical and legal ramifications.

[Lemur]

This type of project will be far over your head for at least 2-3 years of solid programming experience. I would hire someone because legally you could get screwed with any form of exploit that is discovered.

[O9Tacoma4x4]

I dont think legality should be an issue. It's public records. All I'm attempting to do is archive existing data into a search form not available from the original source. It's exactly the same thing google does, but more specific and less advanced. The only money I'd want from Users would be to support sponsors. And even if I charged a subscription or per use fee, I'm pretty sure that's acceptable as well. Theres a number of public records search Companies out there. And they dont get the press lifelock does. The only income I intend to derive from it would be in a couple well choreographed, non-animated ads.

Hiring someone is not an option. I was trying to do a public service while learning. Outsourcing certain elements may be an option, but to just buy a project is self defeating. I'm not in a hurry to get it built. I just got the idea the other day. And even if Ohio puts a Facility inhabitants feature on their site, there are several States without I could re-gear the project for.

Thanks for input and suggestions. I appreciate it greatly.

[eTech1]

I going to recommend this combination because they are often used together and they will run on linux. Learn all of them and in this order:

XHTML
CSS
Javascript
PHP
MySQL

I'd use the tutorials at w3schools and follow each tutorial up with reading through some examples on the site. Then follow that up with some practice projects of your own. At some point you'll need to be learning PHP and MySQL at the same time because working with a database is a major part of working with a server side language such as PHP.

*EDIT* and about the legality it sounds safe to me but you could always apply for an LLC and have that LLC pay you. That way if the site gets sued they can only take the company and not all the profit you've made.

This post has been edited by eTech1: 04 October 2010 - 01:13 PM

[Oler1s]

Quote

I dont think legality should be an issue.

Ok, fair enough. I am no lawyer, and it's just a concern one brings up in relation to these types of applications.

But you haven't answered my question about how you have access to the data. It's important because it affects what programming work you have to do, and consequently, what languages you may learn. Do you have a raw dump of all the inmate data? Does the website provide a machine friendly way to access inmate data? This is the first issue to be tackled in this kind of project!

Also, to address what you physically need. I don't think you want to host the website yourself. You want to host it properly. That means server hosts who have proper and expensive hardware, with good backup and systems, have fat pipes to the internet, and are few hops away from the internet backbone.

For development purposes, you just need a regular computer. You don't need to install Linux. Exactly what software you install depends on what languages you pick.

I'd like to know the answer to the data source question, before I throw any input on what languages you need to study.

[O9Tacoma4x4]

Oler1s, on 04 October 2010 - 02:04 PM, said:

Quote

I dont think legality should be an issue.

Ok, fair enough. I am no lawyer, and it's just a concern one brings up in relation to these types of applications.

But you haven't answered my question about how you have access to the data. It's important because it affects what programming work you have to do, and consequently, what languages you may learn. Do you have a raw dump of all the inmate data? Does the website provide a machine friendly way to access inmate data? This is the first issue to be tackled in this kind of project!

Also, to address what you physically need. I don't think you want to host the website yourself. You want to host it properly. That means server hosts who have proper and expensive hardware, with good backup and systems, have fat pipes to the internet, and are few hops away from the internet backbone.

For development purposes, you just need a regular computer. You don't need to install Linux. Exactly what software you install depends on what languages you pick.

I'd like to know the answer to the data source question, before I throw any input on what languages you need to study.

The information would be gathered from http://www.drc.ohio....rch/Search.aspx .gov means government obviously and everything viewed is under public records. The only real problem I think I'd come into is "Bubba" being disappointed His picture is now in two places rather then one. The DRC site does not even have a terms of use or "condition of entry" pop up. Theres another website that's a publically traded Company I believe that uses data gathered from the National sex offenders registry and makes it a fancy GUI. I've never read about them catching flack.

The HTML is pretty setforth when I use firebug to look @ the code. Nothing seems "hidden". In fact they make the inmate # the picture file name so that would give me a route for back up.

I agree with you about using a hosting Company. If I had a generator and better access to a T3 connection I'd undoubtedly host it myself. I just kind of like the idea of having a server in the house to goof around with. Obviously I'd like to have 100% uptime. I'm practical thou and understand things will happen. And undoubtedly once the State Network Guys find out about my project they'll recode it so I'll have to some work.

[eTech1]

If you want a server in your house to mess around with and learn then just put one on virtual pc, don't use a server at your house to host.

For this site to stay up to date you'll basically have to connect your search to theirs and parse the html file that their site spits out. It's possible but if they ever change the output of there search pages then you'd be down until you rewrite your parsing algorithm.

Another way would be do a search of each county, say once a day, and parse from there. Then your page would still be up to date every morning and you wouldn't be directly connecting to their site each time you get a visitor.

You mentioned images. My guess is public records only includes the information. I would highly recommend you ask first if you intend to use their images or even plan on hot-linking to them.

I would still say go with PHP and MySQL because they are going to be cheaper, there's more information about them on the web, they can get this job done and you get to choose your development environment.

This post has been edited by eTech1: 05 October 2010 - 08:45 AM

[Oler1s]

Quote

The information would be gathered from http://www.drc.ohio....rch/Search.aspx

Well, it looks like you can scrape the inmate data by searching on each county of commitment or county of residence. You can't get a complete listing from that form.

Quote

The HTML is pretty setforth when I use firebug to look @ the code. Nothing seems "hidden". In fact they make the inmate # the picture file name so that would give me a route for back up.

Yes, there are some conveniences on that site. It's not a clean search api, but it's better than some of what is out there on the net.

You would want to write an automated scraper for this, of course. It's just a script or two that needs to be called periodically to update your own databases of inmate data. Python and Ruby (and Perl) work well for this need. You could also use C# or Java if you like, but this kind of high level scripting is done fastest in Python or Ruby. Pick your poison.

What you're going to do is create an HTTP client (like your browser). A form submission is nothing more than an HTTP request. You need to craft the appropriate request through code, and handle the response from the server. This response will be the HTML document.

So this is your starting point. To be able to actually write some code in one of those languages I mentioned. Pick one, and go through some basic language tutorials and practice a bit until you can write code.

[O9Tacoma4x4]

eTech1, on 05 October 2010 - 07:43 AM, said:

If you want a server in your house to mess around with and learn then just put one on virtual pc, don't use a server at your house to host.

For this site to stay up to date you'll basically have to connect your search to theirs and parse the html file that their site spits out. It's possible but if they ever change the output of there search pages then you'd be down until you rewrite your parsing algorithm.

I was hoping I'd be able to just run the update in the background from like 03:00- whatevers a reasonable amount of time to fetch the DRC changes. I was thinking sort of like googles spiders. Where they dont archive every change individually, only modify.

eTech1, on 05 October 2010 - 07:43 AM, said:

You mentioned images. My guess is public records only includes the information. I would highly recommend you ask first if you intend to use their images or even plan on hot-linking to them.

I'm thinking I will play it safe now. Several people have mentioned it. And since I'm not in a huge hurry, I wouldnt mind the 3 week wait it takes the State to figure out where it placed it's pencil. I think the only thing that will piss me off about the project is if after I complete it, the web administrator strokes a couple keys and makes an even better version of the inmate tracker.

Oler1s, on 05 October 2010 - 03:23 PM, said:

Quote

The information would be gathered from http://www.drc.ohio....rch/Search.aspx

Well, it looks like you can scrape the inmate data by searching on each county of commitment or county of residence. You can't get a complete listing from that form.

Quote

The HTML is pretty setforth when I use firebug to look @ the code. Nothing seems "hidden". In fact they make the inmate # the picture file name so that would give me a route for back up.

Yes, there are some conveniences on that site. It's not a clean search api, but it's better than some of what is out there on the net.

You would want to write an automated scraper for this, of course. It's just a script or two that needs to be called periodically to update your own databases of inmate data. Python and Ruby (and Perl) work well for this need. You could also use C# or Java if you like, but this kind of high level scripting is done fastest in Python or Ruby. Pick your poison.

What you're going to do is create an HTTP client (like your browser). A form submission is nothing more than an HTTP request. You need to craft the appropriate request through code, and handle the response from the server. This response will be the HTML document.

So this is your starting point. To be able to actually write some code in one of those languages I mentioned. Pick one, and go through some basic language tutorials and practice a bit until you can write code.

If you wouldnt mind, how long do you think it would take a super proficient programmer, a moderately experienced programmer and a virgin programmer to complete such a program?

[Brewer]

www.W3Schools.com is a great website, no matter what your abilities.