Brick River slowness - vms decided to reboot
Incident Report for Brick River
Resolved
All the virtual machines on Azure are back up.

Sorry about this. We don't know why Azure sometimes decides to reboot the VMs.It might be them pushing an update or might be something in our application. We're looking at Azure's "Application Insights" tool to see if anything jumps out.
We're going to be looking at some deeper level restructuring of the Brick River system in the upcoming months to try to make it faster on the whole.
- Paul Schneider
Posted Nov 10, 2020 - 10:27 EST
Update
The instances are coming back online. Here's the status of this minute for 6 virtual machines...
You'll see that two are out and 4 are on. The two that are out are rebooting. That means our system is 2/3 up...
EngineWeb_EngineWeb_IN_4 10.218.0.8 Timeout
EngineWeb_EngineWeb_IN_0 10.218.0.4 237 200
EngineWeb_EngineWeb_IN_1 10.218.0.5 231 200
EngineWeb_EngineWeb_IN_2 10.218.0.6 391 200
EngineWeb_EngineWeb_IN_3 10.218.0.7 155 200
EngineWeb_EngineWeb_IN_5 10.218.0.9 Timeout
Posted Nov 10, 2020 - 10:11 EST
Monitoring
Azure decided to reboot our instances. We noticed they were going down and restarting one by one. So, we're doing something now to fix the issue. This should only be a few more minutes of slowness.
Posted Nov 10, 2020 - 10:05 EST
This incident affected: Brick River Websites, Brick River Web Console, Brick River Reg System, and Brick River Emailer.