Site Backup Problem

Page 1 of 1

7 Replies - 473 Views - Last Post: 22 January 2011 - 11:20 AM

#1 Martyr2  Icon User is offline

  • Programming Theoretician
  • member icon

Reputation: 4364
  • View blog
  • Posts: 12,182
  • Joined: 18-April 07

Site Backup Problem

Posted 18 January 2011 - 11:12 AM

I know I am going against a bit of the tide here by not mentioning feedback like changing nickname colors or adding some kind of social flair... but I thought I would bring up the recurring problem of midnight backups (12am New Mexico time.... 11pm PST). I am often on around 11pm and right at the stroke of 11pm this site pretty much lags to all hell and even appears to go offline. Normally it is for 2 minutes or so, but the other night it was 16 minutes. I was right in the middle of answering a question when suddenly the site was unreachable.

As this site grows into a site that never sleeps, might be advantageous for DIC to come up with a way to solve the problem or it will just get worse. :)

Is This A Good Question/Topic? 4
  • +

Replies To: Site Backup Problem

#2 skyhawk133  Icon User is offline

  • Head DIC Head
  • member icon

Reputation: 1876
  • View blog
  • Posts: 20,284
  • Joined: 17-March 01

Re: Site Backup Problem

Posted 18 January 2011 - 11:37 AM

Yeh, I got the server down notice just after midnight. Our server management folks sent a reboot request to the NOC and it didn't come back up on the first try. Not sure what happened.

I'll contact them about making those backups smoother. Maybe they've got some ideas. For a while we weren't having problems, so I'm not sure what's causing it now. Especially having the database server on it's own dedicated box.

It's something that's definitely on my radar as it wakes me up every night the server monitor goes off.
Was This Post Helpful? 0
  • +
  • -

#3 Martyr2  Icon User is offline

  • Programming Theoretician
  • member icon

Reputation: 4364
  • View blog
  • Posts: 12,182
  • Joined: 18-April 07

Re: Site Backup Problem

Posted 19 January 2011 - 12:14 AM

11pm again and another incident with the site being laggy and being a bit unresponsive. Below is a screenshot of what I ended up getting as an error at 11:01 pm PST...

Attached Image

This is what I ran into when trying to post an answer right as the 11:00 backup thing started going on. Other times I don't even get an error, it just hangs and finally times out.

Hope this helps! :)

This post has been edited by Martyr2: 19 January 2011 - 12:15 AM

Was This Post Helpful? 0
  • +
  • -

#4 skyhawk133  Icon User is offline

  • Head DIC Head
  • member icon

Reputation: 1876
  • View blog
  • Posts: 20,284
  • Joined: 17-March 01

Re: Site Backup Problem

Posted 19 January 2011 - 08:35 AM

Thanks Martyr2! I saw it go down and come back up within a few minutes.
Was This Post Helpful? 0
  • +
  • -

#5 Janitor  Icon User is offline

  • D.I.C Head

Reputation: 6
  • View blog
  • Posts: 56
  • Joined: 30-July 10

Re: Site Backup Problem

Posted 19 January 2011 - 12:40 PM

parallel/mirrored servers, mimicking your current database server, would remove the need for backups, as well as not affecting performance...
Was This Post Helpful? 0
  • +
  • -

#6 skyhawk133  Icon User is offline

  • Head DIC Head
  • member icon

Reputation: 1876
  • View blog
  • Posts: 20,284
  • Joined: 17-March 01

Re: Site Backup Problem

Posted 19 January 2011 - 02:29 PM

Yeh... I'm already in for $1400+/mo right now on servers, so that's not gonna happen, but we're exploring some solutions.
Was This Post Helpful? 1
  • +
  • -

#7 skyhawk133  Icon User is offline

  • Head DIC Head
  • member icon

Reputation: 1876
  • View blog
  • Posts: 20,284
  • Joined: 17-March 01

Re: Site Backup Problem

Posted 20 January 2011 - 12:43 AM

Serever management team made some changes that reduced the backup priority so it didn't take things offline. I'll have to verify they actually ran in the morning, but it doesn't look like it took the server offline tonight.

Let me know if you notice any other issues (slowness, etc.) as we deployed several changes to the configuration of the web server today.
Was This Post Helpful? 0
  • +
  • -

#8 skyhawk133  Icon User is offline

  • Head DIC Head
  • member icon

Reputation: 1876
  • View blog
  • Posts: 20,284
  • Joined: 17-March 01

Re: Site Backup Problem

Posted 22 January 2011 - 11:20 AM

Backups ran last night with little problem it looks like. Continuing to monitor the situation. We did have a little hiccup last night that was code related, but shouldn't have affected logged in users.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1