Ticket Tool Status

Postmortem: After an hour of troubleshooting and debugging the issue was located and found to be a hardware failure, A short window of time where I attempted to resolve the issue myself failed and I moved into resolving the issue by spinning up a new server to host the API & Redis cache that the dead server was responsible for.

After another hour I managed to get it all setup, DNS updated and IP addresses pointing to the new server we had the api back online.. about 10 minutes after that I had the premium bot back online..

We monitored the bot for about 15 minutes to make sure the migration didn’t have any unforeseen issues and proceeded to spin up the primary bot.

It all was solid and we’re back online.

Resolved: This incident has been resolved.

Monitoring: Primary bot is coming back online

Monitoring: The server has been replaced, we have started up the premium bot and are monitoring it for stability issues on the new server.. The primary bot will follow soon.

Identified: We have located the issue as a hardware failure and are looking to get this fixed or replaced.

Investigating: At this time Ticket Tool's Bot is experiencing outages in all servers. The Development Team has been alerted and is looking into it.

Postmortem: After an hour of troubleshooting and debugging the issue was located and found to be a hardware failure, A short window of time where I attempted to resolve the issue myself failed and I moved into resolving the issue by spinning up a new server to host the API & Redis cache that the dead server was responsible for.

After another hour I managed to get it all setup, DNS updated and IP addresses pointing to the new server we had the api back online.. about 10 minutes after that I had the premium bot back online..

We monitored the bot for about 15 minutes to make sure the migration didn’t have any unforeseen issues and proceeded to spin up the primary bot.

It all was solid and we’re back online.

Resolved: This incident has been resolved.

Monitoring: Primary bot is coming back online

Monitoring: The server has been replaced, we have started up the premium bot and are monitoring it for stability issues on the new server.. The primary bot will follow soon.

Identified: We have located the issue as a hardware failure and are looking to get this fixed or replaced.

Investigating: At this time Ticket Tool's Bot is experiencing outages in all servers. The Development Team has been alerted and is looking into it.

Postmortem: After an hour of troubleshooting and debugging the issue was located and found to be a hardware failure, A short window of time where I attempted to resolve the issue myself failed and I moved into resolving the issue by spinning up a new server to host the API & Redis cache that the dead server was responsible for.

After another hour I managed to get it all setup, DNS updated and IP addresses pointing to the new server we had the api back online.. about 10 minutes after that I had the premium bot back online..

We monitored the bot for about 15 minutes to make sure the migration didn’t have any unforeseen issues and proceeded to spin up the primary bot.

It all was solid and we’re back online.

Resolved: This incident has been resolved.

Monitoring: Primary bot is coming back online

Monitoring: The server has been replaced, we have started up the premium bot and are monitoring it for stability issues on the new server.. The primary bot will follow soon.

Identified: We have located the issue as a hardware failure and are looking to get this fixed or replaced.

Investigating: At this time Ticket Tool's Bot is experiencing outages in all servers. The Development Team has been alerted and is looking into it.

Postmortem: After an hour of troubleshooting and debugging the issue was located and found to be a hardware failure, A short window of time where I attempted to resolve the issue myself failed and I moved into resolving the issue by spinning up a new server to host the API & Redis cache that the dead server was responsible for.

After another hour I managed to get it all setup, DNS updated and IP addresses pointing to the new server we had the api back online.. about 10 minutes after that I had the premium bot back online..

We monitored the bot for about 15 minutes to make sure the migration didn’t have any unforeseen issues and proceeded to spin up the primary bot.

It all was solid and we’re back online.

Resolved: This incident has been resolved.

Monitoring: Primary bot is coming back online

Monitoring: The server has been replaced, we have started up the premium bot and are monitoring it for stability issues on the new server.. The primary bot will follow soon.

Identified: We have located the issue as a hardware failure and are looking to get this fixed or replaced.

Investigating: At this time Ticket Tool's Bot is experiencing outages in all servers. The Development Team has been alerted and is looking into it.