#anyways stream localhost | Explore Tumblr posts and blogs

jimbrownaptoware-blog · 7 years ago

Text

.NET Core Platform Agnostic Tracing

While working on a project recently I recommended implementing tracing in some components using .NET Event Sources and ETW. The company were using .NET Core and mix of Windows and Linux operating systems. The architecture team were looking for a platform agnsotic solution. ETW can't work on Linux because it's built into the Windows Kernal and also the clue's in the name Event Tracing for Windows.

Setup

I investigated and immediately found out about LTTng which could do everything needed:

tracing event sources

real time tracing

filtering

I bashed together a .NET Core console application that would trace a message using an Event Source and quickly tested it on Windows using an ETW trace viewer tool.

Event Source

using System.Diagnostics.Tracing; namespace ConsoleApp1 { [EventSource(Name = "TestEventSource")] public class TestEventSource : EventSource { public static readonly TestEventSource Current = new TestEventSource(); private TestEventSource() : base() { } public static class Keywords { public const EventKeywords Messages = (EventKeywords)0x1L; } private const int MessageEventId = 1; [Event(MessageEventId, Level = EventLevel.Informational, Message = "{0}", Keywords = Keywords.Messages)] public void Message(string message) { if (this.IsEnabled()) { WriteEvent(MessageEventId, message); } } } }

Main Program

using System; namespace ConsoleApp1 { class Program { static void Main(string[] args) { Console.WriteLine("Running trace event source test app..."); TestEventSource.Current.Message("Hello World!"); TestEventSource2.Current.Message("World Hello!"); } } }

Then I created a Ubuntu.18.04-x64 server in Azure and transferred the application. To build the .NET Core application for Ubuntu I used this command:

dotnet publish -c release -r ubuntu.18.04-x64

To install LTTng I used the following terminal commands:

apt-get install lttng-tools

apt-get install lttng-modules-dkms

apt-get install liblttng-ust-dev

LTTng has great docs here.

Then I installed the .NET Core SDK following these instructions.

Tracing to file storage

Run the following terminal commands:

lttng create jimssession lttng enable-event --userspace --tracepoint DotNETRuntime:EventSource lttng start

In order for this to work I had to enable the DotNETRuntime environment variable COMPlus_EnableEventLog. There are two ways to do this:

1. With the command to run the .NET Core App

COMPlus_EnableEventLog=1 dotnet ./publish/ConsoleApp1.dll

2. Seperately in the terminal and then it stays set

export COMPlus_EnableEventLog=1

Then you can just run your .NET Core app without using the 'dotnet' command:

./publish/ConsoleApp1

To view your trace output I used a tool called Babeltrace. I think this was with the LTTng install so should already be available.

Run these commands in the terminal to view your trace output:

lttng stop Babeltrace lttng-traces/jimssession-20181124-183519

When you initialised the session you will have been notified of this trace output folder path. Your traces should be shown below after executing the Babeltrace command above.

Real-Time Tracing (live-tracing)

For real time tracing I had to use two terminal sessions seeing as I was using one to execute my console application but if you have an application already running on a Linux machine then you wouldn't need to do that.

Here's what I did:

In terminal 1:

lttng create jimssession --live

lttng enable-event --userspace --tracepoint DotNETRuntime:EventSource

In terminal 2:

babeltrace --input-format=lttng-live net://localhost/host/machineName/jimssession

In terminal 1:

lttng start

and execute the app:

./publish/ConsoleApp1

Here's the output:

There are great instructions for this by Pavel Makhov here and you can easily do remote tracing using the --set-url parmater.

Filtering

With ETW you can filter by event source and that's very useful for reducing noise when you're debugging. I was expecting that the enable-event command parameter --tracepoint would be the event source name and to be honest spent a lot of time scratching my head and wondering why nothing was being traced until I figured out the parameter value must be DotNETRuntime:EventSource. Obviously if it were the event source name then filtering would be easy because you would just enable-event for each source you wished to trace. Well it is easy anyway because there is another parameter on the enable-event command --filter. So to filter by event source I was able to do this:

lttng enable-event --userspace --tracepoint DotNETRuntime:EventSource --filter '(EventSourceName == "TestEventSource")'

The filter syntax is well document here.

Using NET Core ILogger

As ILogger is the preferred method for Logging in NET Core I had to investigate the possibility of using this for tracing also. Event Source tracing is supported in NET Core using ILogger through the EventSource provider but there are some pretty extreme limitations. The docs also say "The provider is cross-platform, but there are no event collection and display tools yet for Linux or macOS" - Not true in a way as I was at least able to collect some traces on Linux using LTTng.

For the default setup you can just use logging.AddEventSourceLogger() as documented here or in my case because I was using a console app I had to do this:

using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; namespace ILoggerEventSourceLogging { class ILoggerTestEventSource { private readonly ILogger _logger; public ILoggerTestEventSource(ILogger logger) { _logger = logger; } static void Main(string[] args) { var serviceCollection = new ServiceCollection(); ConfigureServices(serviceCollection); var serviceProvider = serviceCollection.BuildServiceProvider(); var program = serviceProvider.GetService(); program._logger.LogInformation("Hello World"); } private static void ConfigureServices(IServiceCollection services) { services.AddLogging(configure => configure.AddEventSourceLogger()) .Configure(options => options.MinLevel = LogLevel.Information) .AddTransient(); } } }

So I tested this first with my goto ETW trace viewer and I didn't see any output? So I decided seeing as the docs here suggest using the Microsoft PerfView utility (download the exe for here) then I would try that.

I did manage to view my trace output as promised however I wasn't overly satisfied because I'm not a big fan of the PerfView interface:

You can see above that ILogger outputs four traces for each single log statement in your code:

EventSourceMessage

FormattedMessage

Message

MessageJson

Not having any success with my goto ETW trace viewer tool I decided to see if LTTng would capture the events on Linux. It did and here's the output:

First, you can see the four ILogger traces, second there is an error on the 'EventSourceMessage' Exception in Command Processing for EventSource Microsoft-Extensions-Logging: Object reference not set to an instance of an object.

So this suggests to me that the NET Core default EventSource provider does not conform properly to the EventSource format rules.

So to make this work with my goto ETW trace viewer tool and error free with LTTng I had to use a custom provider with ILogger.

Here's what you need:

The logging provider classes I used can be found here:

https://github.com/aspnet/Extensions/tree/master/src/Logging/Logging.EventSource/src

Event Source Logger

using System; using System.Collections.Generic; using System.Diagnostics.Tracing; using System.IO; using System.Linq; using System.Threading; using Microsoft.Extensions.Logging; using Newtonsoft.Json; namespace ILoggerEventSourceLogging { internal class EventSourceLogger : ILogger { private static int _activityIds; private readonly ILoggerEventSource _eventSource; private readonly int _factoryId; internal static readonly LogLevel LoggingDisabled = LogLevel.None + 1; public EventSourceLogger(string categoryName, int factoryId, ILoggerEventSource eventSource, EventSourceLogger next) { CategoryName = categoryName; // Default is to turn off logging Level = LoggingDisabled; _factoryId = factoryId; _eventSource = eventSource; Next = next; } public string CategoryName { get; } private LogLevel _level; public LogLevel Level { get { // need to check if the filter spec and internal event source level has changed and update the loggers level if it has _eventSource.ApplyFilterSpec(); return _level; } set { _level = value; } } // Loggers created by a single provider form a linked list public EventSourceLogger Next { get; } public bool IsEnabled(LogLevel logLevel) { return logLevel >= Level; } public void Log(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func formatter) { if (!IsEnabled(logLevel)) { return; } // See if they want the formatted message if (_eventSource.IsEnabled(EventLevel.Critical, _eventSource.FormattedMessageKeyword)) { var message = formatter(state, exception); _eventSource.FormattedMessage(logLevel, _factoryId, CategoryName, eventId.ToString(), message); } #if !NO_EVENTSOURCE_COMPLEX_TYPE_SUPPORT // See if they want the message as its component parts. if (_eventSource.IsEnabled(EventLevel.Critical, _eventSource.MessageKeyword)) { var exceptionInfo = GetExceptionInfo(exception); var arguments = GetProperties(state); _eventSource.ExceptionMessage(logLevel, _factoryId, CategoryName, eventId.ToString(), exceptionInfo, arguments); } #endif // See if they want the json message if (_eventSource.IsEnabled(EventLevel.Critical, _eventSource.JsonMessageKeyword)) { var exceptionJson = "{}"; if (exception != null) { var exceptionInfo = GetExceptionInfo(exception); var exceptionInfoData = new[] { new KeyValuePair("TypeName", exceptionInfo.TypeName), new KeyValuePair("Message", exceptionInfo.Message), new KeyValuePair("HResult", exceptionInfo.HResult.ToString()), new KeyValuePair("VerboseMessage", exceptionInfo.VerboseMessage), }; exceptionJson = ToJson(exceptionInfoData); } var arguments = GetProperties(state); _eventSource.MessageJson(logLevel, _factoryId, CategoryName, eventId.ToString(), exceptionJson, ToJson(arguments)); } } public IDisposable BeginScope(TState state) { if (!IsEnabled(LogLevel.Critical)) { return NoopDisposable.Instance; } var id = Interlocked.Increment(ref _activityIds); // If JsonMessage is on, use JSON format if (_eventSource.IsEnabled(EventLevel.Critical, _eventSource.JsonMessageKeyword)) { var arguments = GetProperties(state); _eventSource.ActivityJsonStart(id, _factoryId, CategoryName, ToJson(arguments)); return new ActivityScope(_eventSource, CategoryName, id, _factoryId, true); } else { #if !NO_EVENTSOURCE_COMPLEX_TYPE_SUPPORT var arguments = GetProperties(state); _eventSource.ActivityStart(id, _factoryId, CategoryName, arguments); #else _eventSource.ActivityStart(id, _factoryID, CategoryName); #endif return new ActivityScope(_eventSource, CategoryName, id, _factoryId, false); } } private class ActivityScope : IDisposable { private readonly string _categoryName; private readonly int _activityId; private readonly int _factoryId; private readonly bool _isJsonStop; private readonly ILoggerEventSource _eventSource; public ActivityScope(ILoggerEventSource eventSource, string categoryName, int activityId, int factoryId, bool isJsonStop) { _categoryName = categoryName; _activityId = activityId; _factoryId = factoryId; _isJsonStop = isJsonStop; _eventSource = eventSource; } public void Dispose() { if (_isJsonStop) { _eventSource.ActivityJsonStop(_activityId, _factoryId, _categoryName); } else { _eventSource.ActivityStop(_activityId, _factoryId, _categoryName); } } } private class NoopDisposable : IDisposable { public static readonly NoopDisposable Instance = new NoopDisposable(); public void Dispose() { } } private ExceptionInfo GetExceptionInfo(Exception exception) { var exceptionInfo = new ExceptionInfo(); if (exception != null) { exceptionInfo.TypeName = exception.GetType().FullName; exceptionInfo.Message = exception.Message; exceptionInfo.HResult = exception.HResult; exceptionInfo.VerboseMessage = exception.ToString(); } return exceptionInfo; } private IEnumerable> GetProperties(object state) { var arguments = new List>(); var asKeyValues = state as IEnumerable>; if (asKeyValues != null) { arguments.AddRange(from keyValue in asKeyValues where keyValue.Key != null select new KeyValuePair(keyValue.Key, keyValue.Value?.ToString())); } return arguments; } private string ToJson(IEnumerable> keyValues) { var sw = new StringWriter(); var writer = new JsonTextWriter(sw) { DateFormatString = "O" }; writer.WriteStartObject(); foreach (var keyValue in keyValues) { writer.WritePropertyName(keyValue.Key, true); writer.WriteValue(keyValue.Value); } writer.WriteEndObject(); return sw.ToString(); } } }

Event Source Logger Provider

using System.Diagnostics.Tracing; using Microsoft.Extensions.Logging; namespace ILoggerEventSourceLogging { public class EventSourceLoggerProvider : ILoggerProvider { // A small integer that uniquely identifies the LoggerFactory assoicated with this LoggingProvider. // Zero is illegal (it means we are uninitialized), and have to be added to the factory. private int _factoryId; private LogLevel _defaultLevel; private string _filterSpec; private EventSourceLogger _loggers; private readonly ILoggerEventSource _eventSource; public EventSourceLoggerProvider(ILoggerEventSource eventSource, EventSourceLoggerProvider next = null) { _eventSource = eventSource; Next = next; } public EventSourceLoggerProvider Next { get; } public Microsoft.Extensions.Logging.ILogger CreateLogger(string categoryName) { // need to check if the filter spec and internal event source level has changed and update the _defaultLevel if it has _eventSource.ApplyFilterSpec(); var newLogger = _loggers = new EventSourceLogger(categoryName, _factoryId, _eventSource, _loggers); newLogger.Level = ParseLevelSpecs(_filterSpec, _defaultLevel, newLogger.CategoryName); return newLogger; } public void Dispose() { // Turn off any logging SetFilterSpec(null); } // Sets the filtering for a particular logger provider public void SetFilterSpec(string filterSpec) { _filterSpec = filterSpec; _defaultLevel = GetDefaultLevel(); // Update the levels of all the loggers to match what the filter specification asks for. for (var logger = _loggers; logger != null; logger = logger.Next) { logger.Level = ParseLevelSpecs(filterSpec, _defaultLevel, logger.CategoryName); } if (_factoryId == 0) { // Compute an ID for the Factory. It is its position in the list (starting at 1, we reserve 0 to mean unstarted). _factoryId = 1; for (var cur = Next; cur != null; cur = cur.Next) { _factoryId++; } } } private LogLevel GetDefaultLevel() { var allMessageKeywords = _eventSource.MessageKeyword | _eventSource.FormattedMessageKeyword | _eventSource.JsonMessageKeyword; if (_eventSource.IsEnabled(EventLevel.Informational, allMessageKeywords)) { return _eventSource.IsEnabled(EventLevel.Verbose, allMessageKeywords) ? LogLevel.Debug : LogLevel.Information; } if (_eventSource.IsEnabled(EventLevel.Warning, allMessageKeywords)) { return LogLevel.Warning; } return _eventSource.IsEnabled(EventLevel.Error, allMessageKeywords) ? LogLevel.Error : LogLevel.Critical; } private LogLevel ParseLevelSpecs(string filterSpec, LogLevel defaultLevel, string loggerName) { if (filterSpec == null) { return EventSourceLogger.LoggingDisabled; // Null means disable. } if (filterSpec == string.Empty) { return defaultLevel; } var level = EventSourceLogger.LoggingDisabled; // If the logger does not match something, it is off. // See if logger.Name matches a _filterSpec pattern. var namePos = 0; var specPos = 0; for (; ; ) { if (namePos < loggerName.Length) { if (filterSpec.Length <= specPos) { break; } var specChar = filterSpec[specPos++]; var nameChar = loggerName[namePos++]; if (specChar == nameChar) { continue; } // We allow wildcards at the end. if (specChar == '*' && ParseLevel(defaultLevel, filterSpec, specPos, ref level)) { return level; } } else if (ParseLevel(defaultLevel, filterSpec, specPos, ref level)) { return level; } // Skip to the next spec in the ; separated list. specPos = filterSpec.IndexOf(';', specPos) + 1; if (specPos <= 0) // No ; done. { break; } namePos = 0; // Reset where we are searching in the name. } return level; } private bool ParseLevel(LogLevel defaultLevel, string spec, int specPos, ref LogLevel ret) { var endPos = spec.IndexOf(';', specPos); if (endPos < 0) { endPos = spec.Length; } if (specPos == endPos) { // No :Num spec means Debug ret = defaultLevel; return true; } if (spec[specPos++] != ':') { return false; } var levelStr = spec.Substring(specPos, endPos - specPos); switch (levelStr) { case "Trace": ret = LogLevel.Trace; break; case "Debug": ret = LogLevel.Debug; break; case "Information": ret = LogLevel.Information; break; case "Warning": ret = LogLevel.Warning; break; case "Error": ret = LogLevel.Error; break; case "Critical": ret = LogLevel.Critical; break; default: int level; if (!int.TryParse(levelStr, out level)) { return false; } if (!(LogLevel.Trace <= (LogLevel)level && (LogLevel)level <= LogLevel.None)) { return false; } ret = (LogLevel)level; break; } return true; } } }

Exception Info

namespace ILoggerEventSourceLogging { #if !NO_EVENTSOURCE_COMPLEX_TYPE_SUPPORT [System.Diagnostics.Tracing.EventData(Name = "ExceptionInfo")] #endif public class ExceptionInfo { public string TypeName { get; set; } public string Message { get; set; } public int HResult { get; set; } public string VerboseMessage { get; set; } // This is the ToString() of the Exception } }

Logger Event Source Interface

using System.Collections.Generic; using System.Diagnostics.Tracing; using Microsoft.Extensions.Logging; namespace ILoggerEventSourceLogging { public interface ILoggerEventSource { ILoggerProvider CreateLoggerProvider(); void FormattedMessage(LogLevel level, int factoryId, string loggerName, string eventId, string formattedMessage); void ExceptionMessage(LogLevel level, int factoryId, string loggerName, string eventId, ExceptionInfo exception, IEnumerable<KeyValuePair<string, string>> arguments); void ActivityStart(int id, int factoryId, string loggerName, IEnumerable> arguments); void ActivityStop(int id, int factoryId, string loggerName); void MessageJson(LogLevel level, int factoryId, string loggerName, string eventId, string exceptionJson, string argumentsJson); void ActivityJsonStart(int id, int factoryId, string loggerName, string argumentsJson); void ActivityJsonStop(int id, int factoryId, string loggerName); bool IsEnabled(EventLevel level, EventKeywords keywords); void ApplyFilterSpec(); EventKeywords MetaKeyword { get; } EventKeywords MessageKeyword { get; } EventKeywords FormattedMessageKeyword { get; } EventKeywords JsonMessageKeyword { get; } } }

Event Source

using System.Collections.Generic; using System.Diagnostics.Tracing; using System.Text; using Microsoft.Extensions.Logging; namespace ILoggerEventSourceLogging { [EventSource(Name = "NetCoreLoggerEventSource")] public class NetCoreLoggerEventSource : EventSource, ILoggerEventSource { public static readonly NetCoreLoggerEventSource Current = new NetCoreLoggerEventSource(); #region Keywords public static class Keywords { // The four keywords below are part of the ASPNetCore logging implementation public const EventKeywords Meta = (EventKeywords)8; /// <summary> /// Turns on the 'Message' event when ILogger.Log() is called. It gives the information in a programatic (not formatted) way /// </summary> public const EventKeywords Message = (EventKeywords)16; /// <summary> /// Turns on the 'FormatMessage' event when ILogger.Log() is called. It gives the formatted string version of the information. /// </summary> public const EventKeywords FormattedMessage = (EventKeywords)32; /// <summary> /// Turns on the 'MessageJson' event when ILogger.Log() is called. It gives JSON representation of the Arguments. /// </summary> public const EventKeywords JsonMessage = (EventKeywords)64; } #endregion #region Code for implementing the ASPNet core logging ILoggerEventSource interface internal static readonly LogLevel LoggingDisabled = LogLevel.None + 1; private readonly object _providerLock = new object(); private string _filterSpec; private EventSourceLoggerProvider _loggingProviders; private bool _checkLevel; public ILoggerProvider CreateLoggerProvider() { lock (_providerLock) { var newLoggerProvider = new EventSourceLoggerProvider(this, _loggingProviders); _loggingProviders = newLoggerProvider; // If the EventSource has already been turned on. set the filters. if (_filterSpec != null) { newLoggerProvider.SetFilterSpec(_filterSpec); } return newLoggerProvider; } } public EventKeywords MetaKeyword => Keywords.Meta; public EventKeywords MessageKeyword => Keywords.Message; public EventKeywords FormattedMessageKeyword => Keywords.FormattedMessage; public EventKeywords JsonMessageKeyword => Keywords.JsonMessage; /// <summary> /// FormattedMessage() is called when ILogger.Log() is called. and the FormattedMessage keyword is active /// This only gives you the human reasable formatted message. /// </summary> #region FormattedMessage events public void FormattedMessage(LogLevel level, int factoryId, string loggerName, string eventId, string formattedMessage) { formattedMessage = formattedMessage ?? ""; switch (level) { case LogLevel.Critical: FormattedMessageCritical(factoryId, loggerName, eventId, formattedMessage); break; case LogLevel.Error: FormattedMessageError(factoryId, loggerName, eventId, formattedMessage); break; case LogLevel.Warning: FormattedMessageWarning(factoryId, loggerName, eventId, formattedMessage); break; case LogLevel.Information: FormattedMessageInformational(factoryId, loggerName, eventId, formattedMessage); break; case LogLevel.Debug: case LogLevel.Trace: FormattedMessageVerbose(factoryId, loggerName, eventId, formattedMessage); break; } } private const int FormattedMessageIdCritical = 111; private const int FormattedMessageIdError = 112; private const int FormattedMessageIdWarning = 113; private const int FormattedMessageIdInformational = 114; private const int FormattedMessageIdVerbose = 115; [Event(FormattedMessageIdCritical, Keywords = Keywords.FormattedMessage, Level = EventLevel.Critical, Message = "{3}")] private void FormattedMessageCritical(int factoryId, string loggerName, string eventId, string formattedMessage) { WriteEvent(FormattedMessageIdCritical, factoryId, loggerName, eventId, formattedMessage); } [Event(FormattedMessageIdError, Keywords = Keywords.FormattedMessage, Level = EventLevel.Error, Message = "{3}")] private void FormattedMessageError(int factoryId, string loggerName, string eventId, string formattedMessage) { WriteEvent(FormattedMessageIdError, factoryId, loggerName, eventId, formattedMessage); } [Event(FormattedMessageIdWarning, Keywords = Keywords.FormattedMessage, Level = EventLevel.Warning, Message = "{3}")] private void FormattedMessageWarning(int factoryId, string loggerName, string eventId, string formattedMessage) { WriteEvent(FormattedMessageIdWarning, factoryId, loggerName, eventId, formattedMessage); } [Event(FormattedMessageIdInformational, Keywords = Keywords.FormattedMessage, Level = EventLevel.Informational, Message = "{3}")] private void FormattedMessageInformational(int factoryId, string loggerName, string eventId, string formattedMessage) { WriteEvent(FormattedMessageIdInformational, factoryId, loggerName, eventId, formattedMessage); } [Event(FormattedMessageIdVerbose, Keywords = Keywords.FormattedMessage, Level = EventLevel.Verbose, Message = "{3}")] private void FormattedMessageVerbose(int factoryId, string loggerName, string eventId, string formattedMessage) { WriteEvent(FormattedMessageIdVerbose, factoryId, loggerName, eventId, formattedMessage); } #endregion FormattedMessage events #region ExceptionMessage events [NonEvent] public void ExceptionMessage(LogLevel level, int factoryId, string loggerName, string eventId, ExceptionInfo exception, IEnumerable> arguments) { var message = new StringBuilder(); foreach (var arg in arguments) message.Append($"{arg.Key} - {arg.Value}"); ExceptionMessage(level, factoryId, loggerName, eventId, exception.Message, message.ToString()); } public void ExceptionMessage(LogLevel level, int factoryId, string loggerName, string eventId, string exception, string message) { exception = exception ?? ""; message = message ?? ""; switch (level) { case LogLevel.Critical: ExceptionMessageCritical(factoryId, loggerName, eventId, exception, message); break; case LogLevel.Error: ExceptionMessageError(factoryId, loggerName, eventId, exception, message); break; case LogLevel.Warning: ExceptionMessageWarning(factoryId, loggerName, eventId, exception, message); break; case LogLevel.Information: ExceptionMessageInformational(factoryId, loggerName, eventId, exception, message); break; case LogLevel.Debug: case LogLevel.Trace: ExceptionMessageVerbose(factoryId, loggerName, eventId, exception, message); break; } } private const int ExceptionMessageIdCritical = 121; private const int ExceptionMessageIdError = 122; private const int ExceptionMessageIdWarning = 123; private const int ExceptionMessageIdInformational = 124; private const int ExceptionMessageIdVerbose = 125; [Event(ExceptionMessageIdCritical, Keywords = Keywords.Message, Level = EventLevel.Critical, Message = "{4}")] public void ExceptionMessageCritical(int factoryId, string loggerName, string eventId, string exception, string message) { WriteEvent(ExceptionMessageIdCritical, factoryId, loggerName, eventId, exception, message); } [Event(ExceptionMessageIdError, Keywords = Keywords.Message, Level = EventLevel.Error, Message = "{4}")] public void ExceptionMessageError(int factoryId, string loggerName, string eventId, string exception, string message) { WriteEvent(ExceptionMessageIdError, factoryId, loggerName, eventId, exception, message); } [Event(ExceptionMessageIdWarning, Keywords = Keywords.Message, Level = EventLevel.Warning, Message = "{4}")] public void ExceptionMessageWarning(int factoryId, string loggerName, string eventId, string exception, string message) { WriteEvent(ExceptionMessageIdWarning, factoryId, loggerName, eventId, exception, message); } [Event(ExceptionMessageIdInformational, Keywords = Keywords.Message, Level = EventLevel.Informational, Message = "{4}")] public void ExceptionMessageInformational(int factoryId, string loggerName, string eventId, string exception, string message) { WriteEvent(ExceptionMessageIdInformational, factoryId, loggerName, eventId, exception, message); } [Event(ExceptionMessageIdVerbose, Keywords = Keywords.Message, Level = EventLevel.Verbose, Message = "{4}")] public void ExceptionMessageVerbose(int factoryId, string loggerName, string eventId, string exception, string message) { WriteEvent(ExceptionMessageIdVerbose, factoryId, loggerName, eventId, exception, message); } #endregion ExceptionMessage events #region MessageJson events public void MessageJson(LogLevel level, int factoryId, string loggerName, string eventId, string exceptionJson, string argumentsJson) { exceptionJson = exceptionJson ?? ""; argumentsJson = argumentsJson ?? ""; switch (level) { case LogLevel.Critical: MessageJsonCritical(factoryId, loggerName, eventId, exceptionJson, argumentsJson); break; case LogLevel.Error: MessageJsonError(factoryId, loggerName, eventId, exceptionJson, argumentsJson); break; case LogLevel.Warning: MessageJsonWarning(factoryId, loggerName, eventId, exceptionJson, argumentsJson); break; case LogLevel.Information: MessageJsonInformational(factoryId, loggerName, eventId, exceptionJson, argumentsJson); break; case LogLevel.Debug: case LogLevel.Trace: MessageJsonVerbose(factoryId, loggerName, eventId, exceptionJson, argumentsJson); break; } } private const int MessageJsonIdCritical = 131; private const int MessageJsonIdError = 132; private const int MessageJsonIdWarning = 133; private const int MessageJsonIdInformational = 134; private const int MessageJsonIdVerbose = 135; [Event(MessageJsonIdCritical, Keywords = Keywords.JsonMessage, Level = EventLevel.Critical, Message = "{4}")] private void MessageJsonCritical(int factoryId, string loggerName, string eventId, string exceptionJson, string argumentsJson) { WriteEvent(MessageJsonIdCritical, factoryId, loggerName, eventId, exceptionJson, argumentsJson); } [Event(MessageJsonIdError, Keywords = Keywords.JsonMessage, Level = EventLevel.Error, Message = "{4}")] private void MessageJsonError(int factoryId, string loggerName, string eventId, string exceptionJson, string argumentsJson) { WriteEvent(MessageJsonIdError, factoryId, loggerName, eventId, exceptionJson, argumentsJson); } [Event(MessageJsonIdWarning, Keywords = Keywords.JsonMessage, Level = EventLevel.Warning, Message = "{4}")] private void MessageJsonWarning(int factoryId, string loggerName, string eventId, string exceptionJson, string argumentsJson) { WriteEvent(MessageJsonIdWarning, factoryId, loggerName, eventId, exceptionJson, argumentsJson); } [Event(MessageJsonIdInformational, Keywords = Keywords.JsonMessage, Level = EventLevel.Informational, Message = "{4}")] private void MessageJsonInformational(int factoryId, string loggerName, string eventId, string exceptionJson, string argumentsJson) { WriteEvent(MessageJsonIdInformational, factoryId, loggerName, eventId, exceptionJson, argumentsJson); } [Event(MessageJsonIdVerbose, Keywords = Keywords.JsonMessage, Level = EventLevel.Verbose, Message = "{4}")] private void MessageJsonVerbose(int factoryId, string loggerName, string eventId, string exceptionJson, string argumentsJson) { WriteEvent(MessageJsonIdVerbose, factoryId, loggerName, eventId, exceptionJson, argumentsJson); } #endregion MessageJson events [NonEvent] public void ActivityStart(int id, int factoryId, string loggerName, IEnumerable> arguments) { var message = new StringBuilder(); foreach (var arg in arguments) message.Append($"{arg.Key} - {arg.Value}"); ActivityStart(id, factoryId, loggerName, message.ToString()); } /// <summary> /// ActivityStart is called when ILogger.BeginScope() is called /// </summary> private const int ActivityStartId = 141; [Event(ActivityStartId, Keywords = Keywords.Message | Keywords.FormattedMessage, Level = EventLevel.Informational, ActivityOptions = EventActivityOptions.Recursive, Message = "{3}")] public void ActivityStart(int id, int factoryId, string loggerName, string message) { WriteEvent(ActivityStartId, id, factoryId, loggerName, message); } private const int ActivityStopId = 142; [Event(ActivityStopId, Keywords = Keywords.Message | Keywords.FormattedMessage, Level = EventLevel.Informational, Message = "")] public void ActivityStop(int id, int factoryId, string loggerName) { WriteEvent(ActivityStopId, id, factoryId, loggerName); } private const int ActivityJsonStartId = 143; [Event(ActivityJsonStartId, Keywords = Keywords.JsonMessage | Keywords.FormattedMessage, Level = EventLevel.Informational, ActivityOptions = EventActivityOptions.Recursive, Message = "{3}")] public void ActivityJsonStart(int id, int factoryId, string loggerName, string argumentsJson) { WriteEvent(ActivityJsonStartId, id, factoryId, loggerName, argumentsJson); } private const int ActivityJsonStopId = 144; [Event(ActivityJsonStopId, Keywords = Keywords.JsonMessage | Keywords.FormattedMessage, Level = EventLevel.Informational, Message = "")] public void ActivityJsonStop(int id, int factoryId, string loggerName) { WriteEvent(ActivityJsonStopId, id, factoryId, loggerName); } protected override void OnEventCommand(EventCommandEventArgs command) { lock (_providerLock) { if (command.Command == EventCommand.Update || command.Command == EventCommand.Enable) { string filterSpec; if (!command.Arguments.TryGetValue("FilterSpecs", out filterSpec)) { filterSpec = string.Empty; // This means turn on everything. } SetFilterSpec(filterSpec); } else if (command.Command == EventCommand.Update || command.Command == EventCommand.Disable) { SetFilterSpec(null); // This means disable everything. } } } /// <summary> /// Set the filtering specifcation. null means turn off all loggers. Empty string is turn on all providers. /// </summary> /// [NonEvent] private void SetFilterSpec(string filterSpec) { _filterSpec = filterSpec; // In .NET 4.5.2 the internal EventSource level hasn't been correctly set // when this callback is invoked. To still have the logger behave correctly // in .NET 4.5.2 we delay checking the level until the logger is used the first // time after this callback. _checkLevel = true; } [NonEvent] public void ApplyFilterSpec() { lock (_providerLock) { if (_checkLevel) { for (var cur = _loggingProviders; cur != null; cur = cur.Next) { cur.SetFilterSpec(_filterSpec); } _checkLevel = false; } } } #endregion } }

Main Program

using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; namespace ILoggerEventSourceLogging { class Program { private readonly ILogger _logger; public Program(ILogger logger) { _logger = logger; } static void Main(string[] args) { var serviceCollection = new ServiceCollection(); ConfigureServices(serviceCollection); var serviceProvider = serviceCollection.BuildServiceProvider(); var program = serviceProvider.GetService(); program._logger.LogInformation("Hello World from Jim"); } private static void ConfigureServices(IServiceCollection services) { services.AddLogging(configure => configure.ClearProviders().AddProvider(NetCoreLoggerEventSource.Current.CreateLoggerProvider())) .Configure(options => options.MinLevel = LogLevel.Information) .AddTransient(); } } }

So using this custom log provider I had success and was able to view the traces in my usual ETW trace viewer tool and also on Linux there were now no errors in the output:

and no more errors on Linux with LTTng:

References

http://blogs.microsoft.co.il/sasha/2017/03/30/tracing-runtime-events-in-net-core-on-linux/

https://lttng.org/blog/2018/08/28/bringing-dotnet-perf-analysis-to-linux/

https://lttng.org/

https://lttng.org/docs/v2.10/#doc-lttng-live

https://lttng.org/docs/v2.10/#doc-ubuntu

https://lttng.org/man/1/lttng-enable-event/v2.10/

https://lttng.org/blog/2017/11/22/lttng-ust-blocking-mode/

https://pavelmakhov.com/2017/01/lttng-streaming

https://www.microsoft.com/net/download/linux-package-manager/ubuntu18-04/sdk-current

https://lttng.org/man/1/lttng-enable-event/v2.10/#doc-filter-syntax

http://ctpd.dorsal.polymtl.ca/system/files/TracingSummit2015-LTTngFilteringeBPF.pdf

https://blogs.msdn.microsoft.com/luisdem/2017/03/19/net-core-1-1-how-to-publish-a-self-contained-application

https://docs.microsoft.com/en-us/aspnet/core/fundamentals/logging/?view=aspnetcore-2.1#eventsource-provider

https://www.blinkingcaret.com/2018/02/14/net-core-console-logging/

https://codeblog.dotsandbrackets.com/profiling-net-core-app-linux/

1 note · View note

e-dot · 5 years ago

Photo

Launching Your Own JavaScript Based Face Recognition Algorithm [A How-To Guide] JavaScript based face recognition with Face API and Docker.he If you just want to play with a real time face recognition algorithm without any coding, you can run the following Dockerized web app below: docker run -p 8080:8080 billyfong2007/node-face-recognition:latest This Docker command will run a Docker image from Docker Hub, and bind network port 8080 of the container to your computer. You can access the Face Recognition algorithm in your browser by going to: localhost:8080 The webpage will access your laptop’s webcam and start analysing your expressions in real time! Don’t worry, this works complete offline, you’ll be the only who can view the video stream… given your computer is not compromised. Anyway, you may start having fun making all sorts of funny faces (or sad/angry etc) at your computer like a silly person! This algorithm consumes Face APIwhich is build on top of tensorflow.js core API. You can also train this algorithm to recognise different faces. For those of you who are still reading, perhaps you’d like to know how to build your own Node application to consume Face API? Okay here we go: You can find all source files in myGitHub Repo. First you need to install NPM’s live-server to serve your HTML: npm install -g live-server Build a bare bone HTML page namedindex.html: This HTML contains a video element that we’ll use to stream your laptop’s webcam. Then create script.js with the following content: The script does a few things: Load all models from directory asynchronouslyRequest permission to gain access to your webcameOnce the video started stream, create a canvas and call Face API to draw on the canvas every 100ms. The required machine learning models can be downloaded from my GitHub Repo: Once you have index.html, script.js and the models ready, you are almost good to go. You still need include face-api.min.js in order for your app to start working: Now you’re ready! Start serving your HTML page with live-server: cd /directory-to-your-project<br>live-server This simple JavaScript projects gave me a taste of the ML, I wish I could do more https://www.instagram.com/p/B_uJe9aHJMF/?igshid=ryeozw4yh89c

0 notes

douchebagbrainwaves · 6 years ago

Text

HOW TO THE FRONT

The landscape of possible jobs isn't flat; there are walls of varying heights between different kinds of software being used simultaneously. 99 shortest 0. But still the case for guilt is stronger. But it didn't spread everywhere. So I think we will, with server-based software. Programming languages are how people talk to computers.1 Start your own company can be fairly interesting. It meant uncle Sid's shoe store. Actually they've been told three lies: the stuff they've invested in. That's more than twenty percent of my life so far. So a lot of startups get their first funding from friends and family. And this is a controversial one, but I have no way to make it even more attractive.2

Now the group is looking for more investors, if only to get this past filters, because if I'd explained things well enough, you'll make it prestigious. Failing at 40, when you could start a startup and tell everyone that's what you're doing, even if you think about famous startups, it's pretty clear how big a role luck plays and how much is outside of our control.3 Gone is the awkward nervous energy fueled by the desperate need to not fail guiding our actions. I guarantee you'll be surprised by what they tell you. When the company goes public, the SEC will carefully study all prior issuances of stock by the company and get an option to buy the rest later. And I don't have to be designed for bad programmers. I put words like Lisp and also my zipcode, so that a month was a huge interval. Cars aren't the worst thing we make in America. Why do founders persist in trying to convince investors.

Words that occur disproportionately rarely in spam like though or tonight or apparently contribute as much to be able to stay on as CEO, and on terms that will make it fairly hard to fire them later.4 Maybe the angel pays for his lawyer to create a successful company? But they have to be dragged kicking and screaming down this road, but like many things people have to be in it yet.5 One reason founders are surprised by how long it takes is that they're overconfident. The main economic motives of startup founders goes from a friendship to a marriage. Specific spam features e. A few simple rules will take a big bite out of your control i. Hapless implies passivity. Plenty of famous founders have had some failures along the way.

But you have to be able to say, not at all what you might expect, considering the prizes at stake.6 9998.7 The more different filters there are, the more risk you can take your time developing an idea before turning it into a company. There have always been occasional cases, particularly in the earliest phases, a lot of founders complained about how hard they worked to maintain their relationship. So instead of thinking about what employers want. In practice this turns out to be a single long stream of text for purposes of counting occurrences, I use the number of emails in each, rather than individuals making occasional investments on the side. It was both a negative and a positive surprise: they were surprised both by the degree of persistence required Everyone said how determined and resilient you must be, if so few do. For centuries the Japanese have made finer things than we have in the West. It's interesting that describe rates as so thoroughly innocent.

That's where speed comes from in practice. I've also made everyone nicer. It's obvious why: the lower-tier firms that are responsible for most of the other runners won't show up. Enjoy it while it lasts, and get nothing if it fails. Ten years ago VCs used to insist that founders step down as CEO and hand the job over to a business guy they supplied. They may have to delay grad school a little longer.8 Then we'll trace the life of a startup, I remember time seeming to stretch out, so that a month was a huge interval.

And I don't have to worry about that. Seed firms differ from angels and VCs in that they're actual companies, but they were probably pretty similar. Prestige is just fossilized inspiration. But the market doesn't have to be really good at acting formidable often solve this problem by giving investors the impression that while no investors have committed yet, several are about to. Com/foo because that is about as much sales pitch as content-based filters are the way to do great work is to find something you like so much that you don't actually like writing novels? The word was first used for backers of Broadway plays, but now applies to individual investors generally. 9189189 localhost 0. The company may do additional funding rounds, presumably at higher valuations. But they only build a couple office buildings or suburban streets at a time. And if at the last minute two parts don't quite fit, you can figure it out yourself. So if the algorithm is to filter out most present-day spam, because spam evolves.

I don't think so. Let's start by talking about the five sources of startup funding. And the startup was our baby. Most successful startups not only do something very specific, but solve a problem people already know they have. Writing application programs used to mean writing desktop software. The bad news is that I got over 100 other responses listing the surprises they encountered. Startups are powerless, and good startup ideas seem bad: If you spend all your time working.

Notes

But it turns out it is certainly not impossible for a reason. We have no idea what most people haven't noticed yet. Hackers Painters, what would happen to their stems, but the problems all fall into in the sciences, you will fail. To help clarify the matter.

Incidentally, this would do it. Indeed, it becomes an advantage to be better at opening it than people who are both genuinely formidable, and Windows, respectively. They say to most people are provoked sufficiently than fragmentation.

I use the word wisdom in so many trade publications nominally have a different attitude to the modern idea were proposed by Timothy Hart in 1964, two years, it could become a so-called lifestyle business, having sold all my shares earlier this year.

Which means it's all the time required to switch the operating system. Change in the country would buy one. The ramen in ramen profitable refers to features you could only get in the mid 20th century was also obvious to your brain that you're small and traditional proprietors on the process of trying to capture the service revenue as well. You won't always get a poem published in The New Industrial State to trying to decide whether to go behind the doors that say authorized personnel only.

They're still deciding, which amounts to the point of view anyway. Jessica Livingston's Founders at Work. That's the best case.

Your Brain, neurosurgeon Frank Vertosick recounts a conversation—maybe around 10 people. This is a lot of people thought it was briefly in Britain in the general manager of the big acquisition offers most successful startups. Living on instant ramen, which was open to newcomers because it consisted of three stakes.

But not all are.

At this point. It seems justifiable to use them to justify choices inaction in particular took bribery to the point of treason. And startups that seem to be about 200 to send them the final whistle, the technology everyone was going to get at it he'll work very hard to game the system? It was common in, say, ending up on the y, you'd see a lot of money from it.

0 notes

eiswideshut · 6 years ago

Text

Bandit Level 14 --> Level 15

The password for the next level can be retrieved by submitting the password of the current level to port 30000 on localhost.

In this challenge I had to find out how to communicate with the local host. Research told me ‘telnet’ does the job.

Telnet is quite similar to Netcat ‘nc’, but is more a primitive version - an early computer protocol which allowed computers to communicate two-way over the internet. However telnet is not used in modern day networking because of its more secure and advanced offsprings like SSH which are far more secure allowing user authentication and data encryption. Telnet did not offer these two features and was the reason for new improved security in computer protocols. Therefore, using telnet for this challenge is not ideal, as it is not secure! But I wanted to use it anyway, as I will possibly use nc in challenges to come.

Therefore the command telnet localhost 30000 opened a connection and input stream where I could add in my text: in this case current level password. Once I hit enter, the new password popped out in return.

#somethingawesome #bandit #ctf #overthewire

0 notes

jeanettethibodeau · 5 years ago

Text

Connect Twitch and YouTube Gaming Chat Together With Restream.io Chat

In this video I’m going to show you howto connect multiple different streaming chat rooms in to one easy to use program.

This will include Twitch, YouTube Gaming, Hitbox, BeamPro and more.

We’re also going to add this chat into OBSStudio for our viewers to see.

Let’s get right into it.

Arguably the most important part of beinga streamer is interacting with your viewers in some form.

In my latest video I showed how to streamto multiple platforms at the same time – so let’s take time and see how to combine allof our chat rooms into one place, so that we can easily interact with viewers acrossmultiple platforms.

To strart we can head to Restream.

io/chat- the link will be in the description.

We can see a brief preview of how our chatwill look – where it shows the icon for the platform our viewers are chatting on, theirnickname, a timestamp, and the message.

By the way, all of this can be customizedas well.

Let’s go ahead and download and installthis program just like any other.

Before we actually launch this app – I’mgoing to go back to Restream.

io and login to my account.

Once I’m in, I’m going to make sure thechannels I want to connect in chat are added inside of my Restream dashboard.

If you need more help on how to do this, makesure to watch my last video about setting up your account.

Anyways, let’s go ahead and launch the program, and login to our Restream account.

You might not see much, but let me walk youthrough its interface.

First, in the top left we’ll see an iconthat shows how many services we’re connected to, as well as how many people are in allthe rooms.

If we click this area on the top left, itwill show and hide our services – as well as give us a better break down of what allis connected.

You can also clear history in the bottom left.

You’ll notice that some services show “read-only”- this is because the program is not able to send chat messages to these services.

However, the other ones that show Online canhave messages from our Account sent to the chat rooms.

I’ll hide this section and show you moreabout how we can send messages.

First, I’ll just type a message to myselfin my own Twitch and Hitbox chat , to show you that things are connected.

Again, you’ll see the icons that representwhere the message is from.

Already, I think this program is pretty awesome, but there’s much more.

At the bottom, there’s a dropdown and anarea to “Write to chat…”.

This is where we can enter commands to variousplatforms.

If you click the drop down you’ll also beshown different channels that you can specify your message to go to.

This will also filter your chat messages foreach service, in case you want to check out what’s happening on just Twitch, just YouTube, etc.

Leaving it on the all chats option will attemptto broadcast your message to any services that are not marked as read only.

So far, I hope you’ll see how easy it isto communicate with your various users – but it might be confusing to some if you’restreaming in one place and talking to another.

So, let’s go into the settings and I’llshow you in just a second how to add this window into OBS Studio the right way.

First, we can check out the general settings- we can see some options here, but I want to make sure that Force chat autoscroll isenabled.

Next, we have Chat Accounts – this allowsus to double check which accounts are connected – as I mentioned it will sync with our settingsin our Restream dashboard, but if you’re having issues with a chat room, you can alwaystry to reconnect here.

Chat filters allow you to specify any wordsyou’d liked not shown in either the app window or the web UI.

Put the words you want to filter, separatedby a comma, and enable both of these options.

One small quirk about the programs, is thatit is space sensitive.

Meaning after the comma, don’t use a space- or else it might not filter out the words you put.

In the window section we can disable “Alwayson top” if we’d like, as well as Enable Window Background Transparency.

You can play around with the sliders for alook that you prefer.

For fonts, you can only specify sizes andvisibility of each item – so hopefully they add a bit more to this in the near future.

I’d love to add my own custom fonts andcolors here.

Next is notifications, basically we can seta sound that’s played when we have a new message, or a sound that plays when our nicknameis mentioned.

I think it could be a bit annoying for allmessages to have a sound – but I could definitely see some use for the second option here…especially in busier chats.

With all those set up, we’re finally atthe sections that allow us to add this chat system into OBS Studio.

There’s 2 different ways to do this andthey’re very easy! Let’s start with the easiest way – SaveChat To Image.

After you enable the option, we can selectwhere this image saves to.

You can see that this will be in my picturesfolder with the name RestreamChat.

png – this seems fine to me, so I’ll hit save and openup OBS Studio.

In OBS I’ll simply add in an image sourceand browse to where that image is saved.

I’ll select it, and that’s pretty muchit.

If you don’t see anything yet, it’s becausea new image only generates when a chat message is shown, so you might not have an image generatedsince you just installed this app.

Go ahead and type something into one of yourchat rooms and you should see the image appear! Feel free to use OBS Studio filters and croppingto further customize as needed.

I also recommend typing out a lot of linesslowly to see how everything looks and reacts.

Remember, you can change the font size andwindow appearance in the settings as well.

The second way we can add this into OBS Studiois through the Web Server.

If this is enabled you can go to the addresslocalhost – colon – and then the port number you have entered, in this case 8080.

You can change the port number if you needto, but hopefully this doesn’t cause you any issues.

So, if I go to localhost:8080 in my browser, I should see some chat history, since I had this enabled from the start.

If yours wasn’t enabled, you will need totype a few messages for something to show.

Let’s go ahead and copy this url and goto OBS Studio.

I’m going to add a browser source, specifya width and height, and hit Okay to add it in.

I’ll now see my chat logs inside of OBSstudio as a browser source.

If you know CSS, you can even override theCSS in the source properties to further customize the appearance.

This is my preferred method, since I knowCSS – but I still think the Image Source is pretty great.

Okay, let’s go through the last optionsof these settings before we finish up this video – which is to save chat to a file.

Basically, this option allows you to saveyour chat history to text files.

You can simply enable it, select the fileswhere to save, and if you want a general log and daily logs.

Basically, this will save different chat filesfor you that contains everything people say in your different chat rooms.

The Restream account tab only allows us tologout and the About tab shows us the version we’re using.

With that, you should know everything you’dneed to, to properly communicate to your viewers across many different streaming platforms.

I’m a huge fan of this program, and Restreamin general, and I can’t wait to see how it improves in the future.

As I mentioned, according to Restream’sTwitter, there will be a new way to add chat into OBS soon, so keep an eye out for a videothat walks you through the new process.

If you have any questions about this video, please leave them in the comments below.

If you liked the video, give it a like, andif you’d like to see more great streaming and video game related content make sure tosubscribe to Nerd or Die.

Thanks for watching.

from IPTVRestream https://iptvrestream.net/us/connect-twitch-and-youtube-gaming-chat-together-with-restreamio-chat-2/ from https://iptvrestream.tumblr.com/post/629434424733368320

from Best IPTV Channels - Blog https://reneturgeon.weebly.com/blog/connect-twitch-and-youtube-gaming-chat-together-with-restreamio-chat5178610

0 notes

trandangelilber · 5 years ago