Instrumentation, Logging and Analytics

[stevejgordon]

As we get closer to a real deployment of allReady I'm starting to think about things like instrumentation, logging and analytics. I realise that probably make me seem dull! 😄

It would be really useful as we start to have allReady used by real users to have the ability to understand how it's used, how it performs and when it errors. That way we can build up data to drive future issues and enhancements. Something odd about me is I find the idea of tracking and analyzing such data quite interesting (although I've not had experience doing it much so far)

I know we have a AppInsights key in the config, so I presume we might be getting some data in Azure that could be useful? I've not played with AppInsights much so I'm not sure what it might do for us by default. I guess the first question is, can we get access to this data in a readonly form to understand what it tells us?

I hear a lot about tools such as Stackify, New Relic, RayGun etc which do similar but perhaps additional things compared to AppInsights. Perhaps trialing some of those might be useful? Particularly if they could donate the service to HTBox. I'm not sure which ones currently work with .NET Core though.

A simple but useful starting point would be to get a log of any exceptions encountered in the application with enough data to reproduce/debug on development. I'm not sure if we can log those using AppInsights (or if they get logged automatically) but that would be useful data to see. Perhaps the other APM tools do this if AppInsights does not. It may be that we need to do some basic logging in addition to the APM tools to record issues.

Next up would be some performance monitoring. How long are actions / handlers taking? How long are pages taking to load overall. A slow site may result in user drop off so anything we can do to understand areas for improvement would be useful. Again, perhaps we just need some timers and calls into AppInsights? I'm sure we can optimise the number of db calls and shape of the data being returned in some cases. It would be nice to have regular baselines as well so as we merge new features we can understand their impact, one way or another.

Finally, as we roll out it would be useful to have some data about usage. Which screens are being used, what are the user journeys looking like etc. With this we can perhaps focus in on places to simplify things and enhancing the most used areas of the site.

I thought I'd open up this issue to discuss ideas and whether this should be included in the future milestones. We have higher priorities but if we can collect any of this data with little effort during the testing that could be very useful.

cc/ @tonysurma @MisterJames

[mgmccarthy]

@stevejgordon @tonysurma @MisterJames

RE: logging

projects I've worked on in the past have had great success using Elmah (https://github.com/elmah/Elmah) for logging. You really don't have to build anything, just configure it properly, and you the logging of exceptions as well as a web interface into those logs for free.

When we get around to implementing Issue #1257, having a logging mechanism in place like Elmah is going to help us troubleshoot problems as well as let testers give us better information for reporting a problem. Event without doing the work in #1257, we should have a logging mechanism in place for the application.

Right now, it looks like Elmah for .NET Core is still is planned, but no one is really building it yet: elmah/Elmah#407.

[mgmccarthy]

@stevejgordon @tonysurma @MisterJames

RE: logging

Okay, so it looks like I incorrectly linked Elmah to the wrong site! The Elmah I've been using is located here:
https://github.com/elmah/Elmah

NOT here:
https://elmah.io/

Scott Hanslemen's blog initally turned me onto Elmah: (http://www.hanselman.com/blog/ELMAHErrorLoggingModulesAndHandlersForASPNETAndMVCToo.aspx)

[mgmccarthy]

so it appears there is no association between projects. on this About page of elmah.io:

Based on the de-facto standard error component ELMAH, logging errors from your webserver requires nothing more than installing a NuGet package...

there is also this disclaimer at the bottom:

elmah.io is not affiliated with the ELMAH project and none of the nice folks at ELMAH are crafting elmah.io. We contribute to the ELMAH source once in a while and we're lucky enough to have people from ELMAH helping out with the part of elmah.io which is open sourced.

Elmah.io might be worth checking out, but again, there is a fee.

the origina Elmah project (https://github.com/elmah/Elmah) will eventually support .NET Core, but there is no active work begin done on it right now in order to do so.

[stevejgordon]

cc/ @BillWagner per our conversation at NDC

[mgmccarthy]

ping @BillWagner @stevejgordon cc @MisterJames @tonysurma

I've spent about a half-hour this morning playing around with Elmah.IO, which is a cloud-based logging solution built off the code the made the original ELMAH so successful.

It took about five minutes to get up and running and to integrate with the project where any un-caught exceptions are automatically logged, with the full stack trace available.

Having uncaught exceptions automatically logged is a HUGE win for us, b/c we've written so much redundant code around handling situations like this. Issue #1257 talks about a lot of the redundant checks we could get rid of. The user ends up at a generic error page anyhow, it doesn't matter if we check for null then manually return a status code, or allow the app to throw an exception. We want to optimize for the happy path, not write a bunch of code to nicely return status codes for potentially malicious users.

Elmah.IO also integrates with asp.net core's ILogger, which will allow us to explicitly log a message if we wish to do so. I know that Steve and I were talking a while back and it sounds like there is nothing choosen/picked yet for logging in the live environment.

Lastly, Elmah.IO offers a free Small Business subscription to open source project like ours so we can use their logging framework at no cost.

I wasn't at NDC or part of the conversation about logging, but based on my experience as a developer, I've always used ELMAH for logging uncaight exceptions and it has always been a great logging framework. Elmah.IO moves all the good parts of ELMAH to the cloud so we can trouble shoot our live application.

[stevejgordon]

Hey @mgmccarthy - I was thinking on this last night also. I've briefly looked at Elmah.io myself and it looked pretty good. I didn't pull it in or test with it though.

I'm open to whatever gives us the best diagnostic tools and ways to react to exceptions etc that are captured. I was wondering if we should be looking more at AppInsights since this is part of Azure (and we get a lot of Azure credit). I've used it way back and it seemed pretty good but not sure how good it would be for general logging.

By default ASP.NET Core will log to console. I presume that data is available already in Azure against the WebApp. @tonysurma - Can you confirm?

I also heard about SeriLog on .net rocks this morning which got a good review from Scott Allen. Not sure if we need it but it has a sink to AppInsights as well.

[mgmccarthy]

@stevejgordon, unfortunately, I have no experience with AppInsights and how that relates or doesn't relate to Azure. I'm not too sure if some one else wants to pick that up and have a go at it running locally.

I'm not too sure if we would lose some info or other valuable information that AppInsights gives us if were to use Elmah.IO.

Just figured I'd mention Elmah.IO b/c I've been using Elmah for so long.

[stevejgordon]

@mgmccarthy I'm planning to hook up the site to my Azure and have a bit of a play with what we get.

[mgmccarthy]

@stevejgordon @tonysurma @MisterJames @BillWagner

Looks like we would qualify for ELAMH.IO's free Small Business license: https://elmah.io/pricing/ if we decided to go with this logging solution.

and on this page: https://elmah.io/pricing/ at the bottom:

"Free for Open Source
We love open source at elmah.io. Apply for a free Small Business subscription for your project."

I know v1 launch is coming up (not exactly sure when), but it would be good to have a legit logging framework in place to help us trouble shoot issue in production.

[shahiddev]

I've used application insights for some time so happy to chip in. It works well as a sink for dumping logging/telemetry to and using it for proactive alerting however the "issue" with it is that you're primarily using the Azure portal to inspect/diagnose issues and sometimes this isn't ideal for exposing to the relevant audience. The alerts can trigger emails or webhooks so you have quite a few options for downstream alerting too beyond the out of the box app insights.

[stevejgordon]

@shahiddev Sounds like it get us most of what we want. Agree the access element needs consideration and comes down to how the site gets supported post deployment.

@tonysurma
a) Would you be happy with app insights?
b) Do we collect anything in Azure currently?
c) Any thoughts on whether access can be temporarily granted and limited to just the relevant app insights part of the Azure portal?

[tonysurma]

A y
B some of what comes out of box or is easily turned on
C I need to test but I am pretty sure I can create a resource Group that app insights it's inside of and give account access only to that

[mgmccarthy]

@stevejgordon @tonysurma @shahiddev did we have a decision on the logging framework yet? Are we going to go with AppInsights/Azure for logging purposes? If we do, what's the plan on allowing access to the area of Azure that holds the logs for people that need to troubleshoot?

[stevejgordon]

@tonysurma I updated some of the AppInsights libraries in allReady. If a key is set on the allready-d config it hopefully will start reporting data into an AppInsights bucket.

[tonysurma]

@stevejgordon I will try to set that all up/etc. thanks

[mgmccarthy]

@stevejgordon cc @tonysurma

Steve, do the AppInsights libraries offer an implementation in ILogger? If so, what kind of data would AppInsights collect if an unhandled exceptions were to be thrown from the web project? Also, how would people be able to get access to the logs to troubleshoot? Is this something baked into Azure that we're using?

[mgmccarthy]

@stevejgordon @tonysurma any update on this?