With the recent release of the free tier of Azure Data Explorer, I’m finding myself thinking of more ways where Data Explorer might shine. Being part of the team that works on ADX, I use it regularly as a quick and powerful logs analytics system. Not only it allows me to query and analyze huge amounts of logs and traces, but it’s also used for collecting metrics, high profile events, and such.
With that thought in mind, I remembered one of the main challenges I faces in my previous company – working with traces generated from multiple machines, using various logging libraries (we ended up using log4net).
So, I decided to help other developers ingest their traces into their Azure Data Explorer and started with the popular Serilog library.
I’ve developed a Serilog sink that supports Azure Data Explorer, Azure Synapse Data Explorer and Azure Data Explorer Free-Tier. It supports both Queued and Streaming ingestion, custom data mappings, and various authentication methods.
The code is available as an open-source project in GitHub, or as a Nuget package:
Install-Package Serilog.Sinks.AzureDataExplorer
Let me know if you encounter any issues, have any questions, or just want to contribute to the project.
In the near future, I hope to release similar packages for Log4Net and NLog.
What is Azure Data Explorer?
Azure Data Explorer is a fully managed, high-performance, big data analytics platform that makes it easy to analyze high volumes of data in near real time. The Azure Data Explorer toolbox gives you an end-to-end solution for data ingestion, query, visualization, and management. By analyzing structured, semi-structured, and unstructured data across time series, and by using Machine Learning, Azure Data Explorer makes it simple to extract key insights, spot patterns and trends, and create forecasting models. Azure Data Explorer is scalable, secure, robust, and enterprise-ready, and is useful for log analytics, time series analytics, IoT, and general-purpose exploratory analytics.
What is Serliog?
Serilog is a simple logging framework for .NET. It was built with structured logging in mind. It makes it easy to record custom object properties and even output your logs to JSON.
Logging is one of the most basic things that every application needs. It is fundamental to troubleshoot any application problems.
Logging frameworks make it easy to send your logs to different places via simple configurations. Serilog uses what are called sinks to send your logs to a text file, database, or log management solution, or potentially dozens of other places, all without changing your code.