Join 244,117 Programmers for FREE! Get instant access to thousands of experts, tutorials, code snippets, and more! There are 1,380 people online right now. Registration is fast and FREE... Join Now!
Dream.In.Code is currently running on a single server that both the database, email, and web server reside on.
I'm thinking of going to a multi-server configuration, but wanted to hear what some of you have setup for large sites.
What I'm thinking is just a simple database server and web server, but that may grow in to multiple load balanced web servers and seperate email servers.
The MySQL database on DIC currently gets over 220 queries/second during peak hours and that pushes the server pretty hard, but isn't hurting it yet, so I'm just doing some pre-planning.
I'm on a shared server with a hosting company right now, so I can't really help you ;-)
But one thing I've noticed about your site that should help, at least a bit, is your "who's here" section, particularly the online stats --- perhaps that could be reverse proxy cached and update every five minutes.... that'd be one querey per five minutes (through a cron job?), rather than one more with each page view.
Sorry if I'm totally off on this, or what reverse proxies are, I just remember seeing a small snippet about them somewhere.
Right now all of our services have one server to themselves.
And thats taking care of everything so far. We have to start clustering email soon, our volume has increased 5 fold in the last 2 years. We're holding 23 gigs of spam from the last month.
The main application that we develop right now is usually deployed on a server that runs an application server (however sometimes it runs other applications as well), and the database is on a different server. It works quite well. A friend of mine was working on a really big social networking site, and there they had all kind of perv setup, I think they had a single DB, some squid proxies, and a bunch of web front ends, and also some fileservers for the pictures - I might have wrote some bs here, as he was always ranting about it during binge drinking, so although I found it interesting, I am not sure if I remember correctly.
The company I now work for has 15 servers total, 10 web, 1 email, 3 database (MSSQL), and we use a load balancing software, but this software I believe is for Windows servers so I'm not sure if this exact piece of software will work for Apache servers (but don't quote me on that). Ill get the name of the software for when I VPN in today
I'm on a shared server with a hosting company right now, so I can't really help you ;-)
But one thing I've noticed about your site that should help, at least a bit, is your "who's here" section, particularly the online stats --- perhaps that could be reverse proxy cached and update every five minutes.... that'd be one querey per five minutes (through a cron job?), rather than one more with each page view.
Sorry if I'm totally off on this, or what reverse proxies are, I just remember seeing a small snippet about them somewhere.
Thanks for the suggestion spearfish, the number of queries really isn't a problem, Dream.in.code is just growing very rapidly.
I've actually optimized a lot of the site already. The who's online box is stored in a HEAP which is all memory so the queries to that are extremely fast.
That's another question I have. For those of you that have gone from a single server config to a web + database config, did you see a performance hit due to latency?
Not really, but it depends on your queries. If you do a lot of filtering in your application and not in the query, then it can cause latency problems because a lot of data has to be transmitted through the wire.
Our database server is separate from the web servers and I notice no lag at all. I believe it mostly depends on how you have it set up and the connection between the 2 as Nova pointed out.
I would say it depends, if it is a dedicated line for just your application, it might be the case, but if it sits on a network, then other traffic might change the picture.
The datacenter that DIC is at has 2 networks, a public network (what you're viewing DIC on right now) and a private network (called the softlayer), the private network is a full gigabit network between servers. So hopefully if I go that route I'll have full gig between boxes.
I'm thinking of going to a multi-server configuration, but wanted to hear what some of you have setup for large sites.
By no means is my network pushing large traffic, but my network/web hosting currently runs on 3 servers.
1.) Web Server / MySql : Dell PowerEdge 350 w/ Sun D1000 attached Storage 2.) email server : Sun X1 500mhz 3.) XRMS Ticketing Software & internal test environment : Sun X1 500mhz
When any software on the 2 Sun Boxes needs to connect to the MySql database, I just issue the -h option, works flawlessly. & anytime a web app needs to use email, I follow this setup.
I have no idea if there are any advantages or disadvantages to this setup as far as speed. However, I have a few double-check measure in place so if security should get circumvented, or a server should experience hardware failure, the site(s) doesn't just puke.
Forum Index:
Programming Help |
C and C++ |
VB6 |
Java |
VB.NET |
C# |
ASP.NET |
PHP |
ColdFusion |
Perl and Python |
Ruby |
Databases |
Other Languages |
Game Programming |
Mobile Development |
Software Development |
Computer Science |
Industry News |
Programming Tutorials |
C++ Tutorials |
Visual Basic Tutorials |
Java Tutorials |
VB.NET Tutorials |
C# Tutorials |
Linux Tutorials |
PHP Tutorials |
ColdFusion Tutorials |
Windows Tutorials |
HTML/JavaScript Tutorials |
CSS Tutorials |
Flash Tutorials |
Web Promotion Tutorials |
Graphic Design & Photoshop Tutorials |
Software Development Tutorials |
Database Tutorials |
Other Language Tutorials |
ASP.NET Tutorials |
Game Programming Tutorials |
12 Mar, 2008 - 05:37 AM
| 
