ATP is live again, after an 18 hour breakdown

Posted on March 20, 2008
Filed Under >Adil Najam, About ATP, Science and Technology
9 Comments
Total Views: 33557

ATP Team

Pakistan computersAs our regular readers know, All Things Pakistan (ATP) was down for nearly 18 hours between 3AM and 9PM (Eastern US Time) today (20 March, 2008). Our thanks to the many many readers who sent us emails alerting us of this and providing suggestions. Since early morning we have been working frantically, but mostly unsuccessfully, in trying to resolve the problem.

This, of course, was not our first scare. But it was clearly teh worst and longest breakdown we have had. Although it is still not clear if the problem has been fully resolved, the site is now stable enough once again for us to post this message and open comments on existing posts.

We hope that things will remain stable and we will figure out the root problem very soon, but this is also to alert you to please understand and be patient if problems persist.

Since we did our last redesign in July 2007 (ATP 3.0) we have tried to keep a fairly close eye on the technical stuff but not tried to mess much with the design or the functionality. Our new host – mediatemple – has turned out to be decent but not problem-free. The management of the MySQL database has remained a major challenge and caused many heartaches for us as our traffic as well as comments have increased.

Since morning as we did all sort of trouble-shooting, the site did come back live in spurts a few time but was never stable enough for long enough. To be honest, we are still not sure exactly who went wrong, why or how. But the visitor count shows that around 3AM last night the site went down. By around 6PM we had at least figured out a way to get the site back up in a stable fashion. Between then and now we have tweaked things, done various tests and generally tried to ensure general stability.

The mix of what went wrong had to do with some plug-ins (particularly WP-cache and possibly the wordpress plugin “Brian’s Latest Comments”). More importantly, the MySQL database which now seems to have a very large and unweildly table of tens of thousands of comments on all past posts seems to be getting out of control. It is the later that worries us more right now and we will be immediately turn our attention to fixing things there.

Enough technical details for now. We are glad to be back and we hope that things will remain that way.

9 responses to “ATP is live again, after an 18 hour breakdown”

  1. Pakistaniat says:

    Dear readers, it seems that the spam control feature (adding numbers) we had installed was not working for some of you. We apologize for this. We are trying to figure out why it is not working for some people, some time (we are told it may be because of cache functions or with cookies). As we work through this we have replaced the feature with a similar but different plugin and hopefully that might work better. We will will continue trying to figure out what the problem and its solution might be.

    Meanwhile, if you do face this problem could we please request that you send us an email at pakistaniat@gmail.com; please also do mention what browser you were using when the problem occurred.

    Thanks.

  2. Rizwan says:

    so good to get our ATP fix back!!!!!

  3. SMM says:

    u gave us a scare…. I thought there was some hanky lanky going on…. Good to have ATP back


  4. More importantly, the MySQL database which now seems to have a very large and unweildly table of tens of thousands of comments on all past posts seems to be getting out of control.

    MySQL is a very stable DBMS. Much stable and responsive than Oracle and SQL Server and it has capability to handle millions of records. People have been doing it. All is required performance tuning which should be practised with every database.

    The common practice is to store archived data in separate db which would actually be clone of current db. if a user requests a post(via search) which was published 2 years ago then the system should actually query archive db rather current. In this way current db would always have lesser records and system would not take much time in case of complex queries. If still problem persist, get rid of this damn server and buy some reliable one.

Leave a Reply

Your email address will not be published. Required fields are marked *

*