{"id":43645,"date":"2024-09-21T10:05:19","date_gmt":"2024-09-21T14:05:19","guid":{"rendered":"https:\/\/angry.net\/blog2\/?p=43645"},"modified":"2024-09-21T10:05:19","modified_gmt":"2024-09-21T14:05:19","slug":"another-outage-and-the-rca","status":"publish","type":"post","link":"https:\/\/angry.net\/blog2\/?p=43645","title":{"rendered":"Another outage and the RCA"},"content":{"rendered":"<p>Greetings all, this is the Angry Systems Administrator. If you hadn&#8217;t noticed, the Musing of the Angry Webmaster, along with our old rant site went offline around 4am Eastern United States time.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium\" src=\"https:\/\/i0.wp.com\/i.pinimg.com\/originals\/57\/34\/84\/573484233cd43805189dd9cbe1ab7c65.png?resize=375%2C436&#038;ssl=1\" width=\"375\" height=\"436\" \/><\/p>\n<p><!--more--><\/p>\n<p>I noticed the alerts when I woke up and did a quick check. Everything was down hard. I contacted our hosting service and they have restored the system. The reason for this outage?<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium\" src=\"https:\/\/i0.wp.com\/i.pinimg.com\/originals\/ea\/23\/fc\/ea23fc317950a351b6133aa54935691a.jpg?resize=400%2C264&#038;ssl=1\" width=\"400\" height=\"264\" \/><\/p>\n<p>A network card had failed. I suspect it&#8217;s been on it&#8217;s way out for a while now considering how I have been seeing outages lasting a few minutes a few times a week. The hosting service has replaced the bad network card and while they had the server offline, added additional RAM to it. (things had seemed a bit sluggish the last few weeks or so)<\/p>\n<p>Obviously, we&#8217;re back online and it does look like the added RAM is actually helping with things. We&#8217;ve recently had some issues with bad plugins and I&#8217;m now wondering if the fault network adaptor might not have been part of the issue. (I&#8217;ll be testing things and the site might drop offline for a few minutes)<\/p>\n<p>Thank you for your patience and understanding. Now if you will excuse me, I need to fire a trank dart into the Angry Webmaster before he sees what happened.<\/p>\n<p>Thank you<\/p>\n<span id=\"post-ratings-43645\" class=\"post-ratings\" data-nonce=\"8a9a80fe58\"><img data-recalc-dims=\"1\" decoding=\"async\" id=\"rating_43645_1\" src=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/plugins\/wp-postratings\/images\/stars_crystal\/rating_on.gif?w=640&#038;ssl=1\" alt=\"1 Star\" title=\"1 Star\" onmouseover=\"current_rating(43645, 1, '1 Star');\" onmouseout=\"ratings_off(5, 0, 0);\" onclick=\"rate_post();\" onkeypress=\"rate_post();\" style=\"cursor: pointer; border: 0px;\" \/><img data-recalc-dims=\"1\" decoding=\"async\" id=\"rating_43645_2\" src=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/plugins\/wp-postratings\/images\/stars_crystal\/rating_on.gif?w=640&#038;ssl=1\" alt=\"2 Stars\" title=\"2 Stars\" onmouseover=\"current_rating(43645, 2, '2 Stars');\" onmouseout=\"ratings_off(5, 0, 0);\" onclick=\"rate_post();\" onkeypress=\"rate_post();\" style=\"cursor: pointer; border: 0px;\" \/><img data-recalc-dims=\"1\" decoding=\"async\" id=\"rating_43645_3\" src=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/plugins\/wp-postratings\/images\/stars_crystal\/rating_on.gif?w=640&#038;ssl=1\" alt=\"3 Stars\" title=\"3 Stars\" onmouseover=\"current_rating(43645, 3, '3 Stars');\" onmouseout=\"ratings_off(5, 0, 0);\" onclick=\"rate_post();\" onkeypress=\"rate_post();\" style=\"cursor: pointer; border: 0px;\" \/><img data-recalc-dims=\"1\" decoding=\"async\" id=\"rating_43645_4\" src=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/plugins\/wp-postratings\/images\/stars_crystal\/rating_on.gif?w=640&#038;ssl=1\" alt=\"4 Stars\" title=\"4 Stars\" onmouseover=\"current_rating(43645, 4, '4 Stars');\" onmouseout=\"ratings_off(5, 0, 0);\" onclick=\"rate_post();\" onkeypress=\"rate_post();\" style=\"cursor: pointer; border: 0px;\" \/><img data-recalc-dims=\"1\" decoding=\"async\" id=\"rating_43645_5\" src=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/plugins\/wp-postratings\/images\/stars_crystal\/rating_on.gif?w=640&#038;ssl=1\" alt=\"5 Stars\" title=\"5 Stars\" onmouseover=\"current_rating(43645, 5, '5 Stars');\" onmouseout=\"ratings_off(5, 0, 0);\" onclick=\"rate_post();\" onkeypress=\"rate_post();\" style=\"cursor: pointer; border: 0px;\" \/> (<strong>4<\/strong> votes, average: <strong>5.00<\/strong> out of 5)<br \/><span class=\"post-ratings-text\" id=\"ratings_43645_text\"><\/span><\/span><span id=\"post-ratings-43645-loading\" class=\"post-ratings-loading\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/plugins\/wp-postratings\/images\/loading.gif?resize=16%2C16&#038;ssl=1\" width=\"16\" height=\"16\" class=\"post-ratings-image\" \/>Loading...<\/span>\n<p><em><strong>~The Angry Systems Administrator~<\/strong><\/em><\/p>\n<p><a href=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2013\/12\/dr_strangelove_1ed07.jpg?ssl=1\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium wp-image-13627\" src=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2013\/12\/dr_strangelove_1ed07.jpg?resize=300%2C225&#038;ssl=1\" alt=\"\" width=\"300\" height=\"225\" srcset=\"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2013\/12\/dr_strangelove_1ed07.jpg?resize=300%2C225&amp;ssl=1 300w, https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2013\/12\/dr_strangelove_1ed07.jpg?resize=150%2C112&amp;ssl=1 150w, https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2013\/12\/dr_strangelove_1ed07.jpg?resize=400%2C300&amp;ssl=1 400w, https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2013\/12\/dr_strangelove_1ed07.jpg?w=1024&amp;ssl=1 1024w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Greetings all, this is the Angry Systems Administrator. If you hadn&#8217;t noticed, the Musing of the Angry Webmaster, along with our old rant site went offline around 4am Eastern United States time.<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[32,71],"tags":[125,1742,1743,57],"class_list":["post-43645","post","type-post","status-publish","format-standard","hentry","category-general","category-news-of-the-day","tag-epic-fail","tag-rca","tag-root-cause-analysis","tag-technology"],"views":157,"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":30313,"url":"https:\/\/angry.net\/blog2\/?p=30313","url_meta":{"origin":43645,"position":0},"title":"Anger Central Outage","author":"Angry Systems Administrator","date":"February 9, 2020","format":false,"excerpt":"Good day all, this is the Angry Systems Administrator. You may have noticed that Anger Central, both the main site and the blog were not reachable. We noticed this around 6 AM Eastern Standard Time, and tried to reach out to our Hosting company. However, their site was offline as\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/angry.net\/blog2\/?cat=32"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2016\/01\/train-wreck-250x300.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":43154,"url":"https:\/\/angry.net\/blog2\/?p=43154","url_meta":{"origin":43645,"position":1},"title":"Site down July 7th","author":"Angry Systems Administrator","date":"July 7, 2024","format":false,"excerpt":"Good day all, this is the Angry Systems Administrator. You may have noticed that our teensy little web site was offline for a good five hours. It's just come back online thankfully. We don't have the details on what the issue was. We're waiting to hear back from our hosting\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/angry.net\/blog2\/?cat=32"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2014\/04\/snoopy-happy-dance.jpeg-236x300.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":42756,"url":"https:\/\/angry.net\/blog2\/?p=42756","url_meta":{"origin":43645,"position":2},"title":"Yet another server update and RCA","author":"Angry Systems Administrator","date":"May 13, 2024","format":false,"excerpt":"Greetings all you nice people and 1400 bots now hitting the site. As you probably didn't notice, since we rarely see a lot of visitors, we had a major outage last night. The reason for this is simple and hopefully will improve things. As you no doubt noticed, we have\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/angry.net\/blog2\/?cat=32"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/i.pinimg.com\/originals\/57\/34\/84\/573484233cd43805189dd9cbe1ab7c65.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":43374,"url":"https:\/\/angry.net\/blog2\/?p=43374","url_meta":{"origin":43645,"position":3},"title":"Site outage August 8","author":"Angry Systems Administrator","date":"August 8, 2024","format":false,"excerpt":"Greetings everyone. As some of you may have noticed, our site went down around 02:00 Eastern Time. This was due to a migration of our sites from antiquated hardware to new equipment by the hosting service. As you can see now, the site is back online. The issue was due\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/angry.net\/blog2\/?cat=32"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2014\/06\/dog-ate-computer-300x223.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":5092,"url":"https:\/\/angry.net\/blog2\/?p=5092","url_meta":{"origin":43645,"position":4},"title":"System outage","author":"Angry Systems Administrator","date":"October 8, 2011","format":false,"excerpt":"Good day all. We had a major outage this morning around 10am Eastern Time. I contacted our hosting service and it appears that a process ran away on the server we use.","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/angry.net\/blog2\/?cat=32"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":14699,"url":"https:\/\/angry.net\/blog2\/?p=14699","url_meta":{"origin":43645,"position":5},"title":"Site issues","author":"Angry Systems Administrator","date":"April 18, 2014","format":false,"excerpt":"Good afternoon all, this is the Angry Systems Administrator. Shortly after the Angry Webmaster's last posting, the site went down. We were offline for about 30 minutes. When I tried to access the other domains we manage, (Which we won't tell you about), I found they were offline as well.\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/angry.net\/blog2\/?cat=32"},"img":{"alt_text":"The Angry Systems Administrator","src":"https:\/\/i0.wp.com\/angry.net\/blog2\/wp-content\/uploads\/2013\/12\/dr_strangelove_1ed07-150x150.jpg?resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=\/wp\/v2\/posts\/43645","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=43645"}],"version-history":[{"count":1,"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=\/wp\/v2\/posts\/43645\/revisions"}],"predecessor-version":[{"id":43646,"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=\/wp\/v2\/posts\/43645\/revisions\/43646"}],"wp:attachment":[{"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=43645"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=43645"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/angry.net\/blog2\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=43645"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}