Join 85,023 Programmers. There are 1,295 online right now! Ask your question and get quick answers from Dream.In.Code experts. Join the #1 programming help community on the internet! Registration is fast and FREE... Join Now!
Dream.In.Code is currently running on a single server that both the database, email, and web server reside on.
I'm thinking of going to a multi-server configuration, but wanted to hear what some of you have setup for large sites.
What I'm thinking is just a simple database server and web server, but that may grow in to multiple load balanced web servers and seperate email servers.
The MySQL database on DIC currently gets over 220 queries/second during peak hours and that pushes the server pretty hard, but isn't hurting it yet, so I'm just doing some pre-planning.
I'm on a shared server with a hosting company right now, so I can't really help you ;-)
But one thing I've noticed about your site that should help, at least a bit, is your "who's here" section, particularly the online stats --- perhaps that could be reverse proxy cached and update every five minutes.... that'd be one querey per five minutes (through a cron job?), rather than one more with each page view.
Sorry if I'm totally off on this, or what reverse proxies are, I just remember seeing a small snippet about them somewhere.
Right now all of our services have one server to themselves.
And thats taking care of everything so far. We have to start clustering email soon, our volume has increased 5 fold in the last 2 years. We're holding 23 gigs of spam from the last month.
The main application that we develop right now is usually deployed on a server that runs an application server (however sometimes it runs other applications as well), and the database is on a different server. It works quite well. A friend of mine was working on a really big social networking site, and there they had all kind of perv setup, I think they had a single DB, some squid proxies, and a bunch of web front ends, and also some fileservers for the pictures - I might have wrote some bs here, as he was always ranting about it during binge drinking, so although I found it interesting, I am not sure if I remember correctly.
The company I now work for has 15 servers total, 10 web, 1 email, 3 database (MSSQL), and we use a load balancing software, but this software I believe is for Windows servers so I'm not sure if this exact piece of software will work for Apache servers (but don't quote me on that). Ill get the name of the software for when I VPN in today
I'm on a shared server with a hosting company right now, so I can't really help you ;-)
But one thing I've noticed about your site that should help, at least a bit, is your "who's here" section, particularly the online stats --- perhaps that could be reverse proxy cached and update every five minutes.... that'd be one querey per five minutes (through a cron job?), rather than one more with each page view.
Sorry if I'm totally off on this, or what reverse proxies are, I just remember seeing a small snippet about them somewhere.
Thanks for the suggestion spearfish, the number of queries really isn't a problem, Dream.in.code is just growing very rapidly.
I've actually optimized a lot of the site already. The who's online box is stored in a HEAP which is all memory so the queries to that are extremely fast.
That's another question I have. For those of you that have gone from a single server config to a web + database config, did you see a performance hit due to latency?
Not really, but it depends on your queries. If you do a lot of filtering in your application and not in the query, then it can cause latency problems because a lot of data has to be transmitted through the wire.
Our database server is separate from the web servers and I notice no lag at all. I believe it mostly depends on how you have it set up and the connection between the 2 as Nova pointed out.
I would say it depends, if it is a dedicated line for just your application, it might be the case, but if it sits on a network, then other traffic might change the picture.
The datacenter that DIC is at has 2 networks, a public network (what you're viewing DIC on right now) and a private network (called the softlayer), the private network is a full gigabit network between servers. So hopefully if I go that route I'll have full gig between boxes.
12 Mar, 2008 - 06:37 AM
| 