Website hangs are a really painful but all-too-common problem for production web applications. There are just so many things that can go wrong: hanging SQL or web service calls, application deadlocks, thread pool exhaustion (in both the IIS and CLR threadpools), CPU overload, the list just goes on. Most of the time, many of these things interact together to create the hang.
Debugging production hangs is not fun
Historically, resolving hangs has always been a developer-intensive task, rather than something that could be tackled as part of production monitoring. This made it very difficult to resolve hangs because they usually happened in production and would be impossible to reproduce in a developer environment.
If you were hardcore, you’d have to break out the WinDbg debugger and try to catch the hang in production, or try to capture it with Debug Diagnostics … and then often spend days looking for clues in the dump. For everyone else, it meant calling Microsoft support. We’ve had difficult hangs take 2 months to resolve, between catching and collecting proper evidence for a hang in production, and waiting for MSFT support to escalate to an engineer that knows what they are doing.
That said, we finally did it!
LeanSentry now automatically detects and diagnoses IIS and ASP.NET hangs!
Whenever a hang happens, LeanSentry uses an array of heuristics to detect it, and notifies you via an alert. The alert page then provides the step-by-step diagnosis of the hang, no debugging needed.
We’ve added many automatic troubleshooters to LeanSentry over the last few months, but this one certainly takes the cake for taking something so complicated as a hang, and managing to break it down simply so that you can understand/resolve it fast.
I am also incredibly psyched about it because website hangs represent 80% of the panicked support requests we get, so this should really help a great number of people.
Read more about how we do it, and get it for yourself here: http://blog.leansentry.com/2013/05/automatically-diagnosing-iis-asp-net-website-hangs/.
Wow, advertising the holy grail is quite an affair. Look forward to checking this out.