System.Diagnostics.DiagnosticSource

This is the basic API to 'hook' parts of the framework. It is like an EventSource (which can also write object), but is intended to log complex objects that can't be serialized. Please See the DiagnosticSource Users Guide https://github.com/dotnet/corefx/blob/master/src/System.Diagnostics.DiagnosticSource/src/DiagnosticSourceUsersGuide.md for instructions on its use.

Write is a generic way of logging complex payloads. Each notification is given a name, which identifies it as well as a object (typically an anonymous type) that gives the information to pass to the notification, which is arbitrary. The name should be short (so don't use fully qualified names unless you have to to avoid ambiguity), but you want the name to be globally unique. Typically your componentName.eventName where componentName and eventName are strings less than 10 characters are a good compromise. notification names should NOT have '.' in them because component names have dots and for them both to have dots would lead to ambiguity. The suggestion is to use _ instead. It is assumed that listeners will use string prefixing to filter groups, thus having hierarchy in component names is good.

The name of the event being written. An object that represent the value being passed as a payload for the event. This is often a anonymous type which contains several sub-values.

Optional: if there is expensive setup for the notification, you can call IsEnabled before doing this setup. Consumers should not be assuming that they only get notifications for which IsEnabled is true however, it is optional for producers to call this API. The name should be the same as what is passed to Write.

The name of the event being written.

A DiagnosticListener is something that forwards on events written with DiagnosticSource. It is an IObservable (has Subscribe method), and it also has a Subscribe overload that lets you specify a 'IsEnabled' predicate that users of DiagnosticSource will use for 'quick checks'. The item in the stream is a KeyValuePair[string, object] where the string is the name of the diagnostic item and the object is the payload (typically an anonymous type). There may be many DiagnosticListeners in the system, but we encourage the use of The DiagnosticSource.DefaultSource which goes to the DiagnosticListener.DefaultListener. If you need to see 'everything' you can subscribe to the 'AllListeners' event that will fire for every live DiagnosticListener in the appdomain (past or present). Please See the DiagnosticSource Users Guide https://github.com/dotnet/corefx/blob/master/src/System.Diagnostics.DiagnosticSource/src/DiagnosticSourceUsersGuide.md for instructions on its use.

When you subscribe to this you get callbacks for all NotificationListeners in the appdomain as well as those that occurred in the past, and all future Listeners created in the future.

Add a subscriber (Observer). If 'IsEnabled' == null (or not present), then the Source's IsEnabled will always return true.

Same as other Subscribe overload where the predicate is assumed to always return true.

Make a new DiagnosticListener, it is a NotificationSource, which means the returned result can be used to log notifications, but it also has a Subscribe method so notifications can be forwarded arbitrarily. Thus its job is to forward things from the producer to all the listeners (multi-casting). Generally you should not be making your own DiagnosticListener but use the DiagnosticListener.Default, so that notifications are as 'public' as possible.

Clean up the NotificationListeners. Notification listeners do NOT DIE ON THEIR OWN because they are in a global list (for discoverability). You must dispose them explicitly. Note that we do not do the Dispose(bool) pattern because we frankly don't want to support subclasses that have non-managed state.

When a DiagnosticListener is created it is given a name. Return this.

Return the name for the ToString() to aid in debugging.

Override abstract method

Override abstract method

Logically AllListenerObservable has a very simple task. It has a linked list of subscribers that want a callback when a new listener gets created. When a new DiagnosticListener gets created it should call OnNewDiagnosticListener so that AllListenerObservable can forward it on to all the subscribers.

Called when a new DiagnosticListener gets created to tell anyone who subscribed that this happened.

Remove 'subscription from the list of subscriptions that the observable has. Called when subscriptions are disposed. Returns true if the subscription was removed.

One node in the linked list of subscriptions that AllListenerObservable keeps. It is IDisposable, and when that is called it removes itself from the list.

DiagnosticSourceEventSource serves two purposes 1) It allows debuggers to inject code via Function evaluation. This is the purpose of the BreakPointWithDebuggerFuncEval function in the 'OnEventCommand' method. Basically even in release code, debuggers can place a breakpoint in this method and then trigger the DiagnosticSourceEventSource via ETW. Thus from outside the process you can get a hook that is guaranteed to happen BEFORE any DiangosticSource events (if the process is just starting) or as soon as possible afterward if it is on attach. 2) It provides a 'bridge' that allows DiagnosticSource messages to be forwarded to EventListers or ETW. You can do this by enabling the Microsoft-Diagnostics-DiagnosticSource with the 'Events' keyword (for diagnostics purposes, you should also turn on the 'Messages' keyword. This EventSource defines a EventSource argument called 'FilterAndPayloadSpecs' that defines what DiagnsoticSources to enable and what parts of the payload to serialize into the key-value list that will be forwarded to the EventSource. If it is empty, all serializable parts of every DiagnosticSource event will be forwarded (this is NOT recommended for monitoring but can be useful for discovery). The FilterAndPayloadSpecs is one long string with the following structures * It is a newline separated list of FILTER_AND_PAYLOAD_SPEC * a FILTER_AND_PAYLOAD_SPEC can be * EVENT_NAME : TRANSFORM_SPECS * EMPTY - turns on all sources with implicit payload elements. * an EVENTNAME can be * DIAGNOSTIC_SOURCE_NAME / DIAGNOSTC_EVENT_NAME @ EVENT_SOURCE_EVENTNAME - give the name as well as the EventSource event to log it under. * DIAGNOSTIC_SOURCE_NAME / DIAGNOSTC_EVENT_NAME * DIAGNOSTIC_SOURCE_NAME - which wildcards every event in the Diagnostic source or * EMPTY - which turns on all sources * TRANSFORM_SPEC is a semicolon separated list of TRANSFORM_SPEC, which can be * - TRANSFORM_SPEC - the '-' indicates that implicit payload elements should be suppressed * VARIABLE_NAME = PROPERTY_SPEC - indicates that a payload element 'VARIABLE_NAME' is created from PROPERTY_SPEC * PROPERTY_SPEC - This is a shortcut where VARIABLE_NAME is the LAST property name * a PROPERTY_SPEC is basically a list of names separated by '.' * PROPERTY_NAME - fetches a property from the DiagnosticSource payload object * PROPERTY_NAME . PROPERTY NAME - fetches a sub-property of the object. Example1: "BridgeTestSource1/TestEvent1:cls_Point_X=cls.Point.X;cls_Point_Y=cls.Point.Y\r\n" + "BridgeTestSource2/TestEvent2:-cls.Url" This indicates that two events should be turned on, The 'TestEvent1' event in BridgeTestSource1 and the 'TestEvent2' in BridgeTestSource2. In the first case, because the transform did not begin with a - any primitive type/string of 'TestEvent1's payload will be serialized into the output. In addition if there a property of the payload object called 'cls' which in turn has a property 'Point' which in turn has a property 'X' then that data is also put in the output with the name cls_Point_X. Similarly if cls.Point.Y exists, then that value will also be put in the output with the name cls_Point_Y. For the 'BridgeTestSource2/TestEvent2' event, because the - was specified NO implicit fields will be generated, but if there is a property call 'cls' which has a property 'Url' then that will be placed in the output with the name 'Url' (since that was the last property name used and no Variable= clause was specified. Example: "BridgeTestSource1\r\n" + "BridgeTestSource2" This will enable all events for the BridgeTestSource1 and BridgeTestSource2 sources. Any string/primitive properties of any of the events will be serialized into the output. Example: "" This turns on all DiagnosticSources Any string/primitive properties of any of the events will be serialized into the output. This is not likely to be a good idea as it will be very verbose, but is useful to quickly discover what is available. * How data is logged in the EventSource By default all data from Diagnostic sources is logged to the the DiagnosticEventSouce event called 'Event' which has three fields string SourceName, string EventName, IEnumerable[KeyValuePair[string, string]] Argument However to support start-stop activity tracking, there are six other events that can be used Activity1Start Activity1Stop Activity2Start Activity2Stop RecursiveActivity1Start RecursiveActivity1Stop By using the SourceName/EventName@EventSourceName syntax, you can force particular DiagnosticSource events to be logged with one of these EventSource events. This is useful because the events above have start-stop semantics which means that they create activity IDs that are attached to all logging messages between the start and the stop (see https://blogs.msdn.microsoft.com/vancem/2015/09/14/exploring-eventsource-activity-correlation-and-causation-features/) For example the specification "MyDiagnosticSource/RequestStart@Activity1Start\r\n" + "MyDiagnosticSource/RequestStop@Activity1Stop\r\n" + "MyDiagnosticSource/SecurityStart@Activity2Start\r\n" + "MyDiagnosticSource/SecurityStop@Activity2Stop\r\n" Defines that RequestStart will be logged with the EventSource Event Activity1Start (and the cooresponding stop) which means that all events caused between these two markers will have an activity ID assocatied with this start event. Simmilarly SecurityStart is mapped to Activity2Start. Note you can map many DiangosticSource events to the same EventSource Event (e.g. Activity1Start). As long as the activities don't nest, you can reuse the same event name (since the payloads have the DiagnosticSource name which can disambiguate). However if they nest you need to use another EventSource event because the rules of EventSource activities state that a start of the same event terminates any existing activity of the same name. As its name suggests RecursiveActivity1Start, is marked as recursive and thus can be used when the activity can nest with itself. This should not be a 'top most' activity because it is not 'self healing' (if you miss a stop, then the activity NEVER ends). See the DiagnosticSourceEventSourceBridgeTest.cs for more explicit examples of using this bridge.

Indicates diagnostics messages from DiagnosticSourceEventSource should be included.

Indicates that all events from all diagnostic sources should be forwarded to the EventSource using the 'Event' event.

Used to send ad-hoc diagnostics to humans.

Events from DiagnosticSource can be forwarded to EventSource using this event.

This is only used on V4.5 systems that don't have the ability to log KeyValuePairs directly. It will eventually go away, but we should always reserve the ID for this.

Used to mark the beginning of an activity

Used to mark the end of an activity

Used to mark the beginning of an activity

Used to mark the end of an activity that can be recursive.

Used to mark the beginning of an activity

Used to mark the end of an activity that can be recursive.

Fires when a new DiagnosticSource becomes available.

This constructor uses EventSourceSettings which is only available on V4.6 and above systems. We use the EventSourceSettings to turn on support for complex types.

Called when the EventSource gets a command from a EventListener or ETW.

A function which is fully interruptible even in release code so we can stop here and do function evaluation in the debugger. Thus this is just a place that is useful for the debugger to place a breakpoint where it can inject code with function evaluation

FilterAndTransform represents on transformation specification from a DiagnosticsSource to EventSource's 'Event' method. (e.g. MySource/MyEvent:out=prop1.prop2.prop3). Its main method is 'Morph' which takes a DiagnosticSource object and morphs it into a list of string,string key value pairs. This method also contains that static 'Create/Destroy FilterAndTransformList, which simply parse a series of transformation specifications.

Parses filterAndPayloadSpecs which is a list of lines each of which has the from DiagnosticSourceName/EventName:PAYLOAD_SPEC where PAYLOADSPEC is a semicolon separated list of specifications of the form OutputName=Prop1.Prop2.PropN Into linked list of FilterAndTransform that together forward events from the given DiagnosticSource's to 'eventSource'. Sets the 'specList' variable to this value (destroying anything that was there previously). By default any serializable properties of the payload object are also included in the output payload, however this feature and be tuned off by prefixing the PAYLOADSPEC with a '-'.

This destroys (turns off) the FilterAndTransform stopping the forwarding started with CreateFilterAndTransformList

Creates one FilterAndTransform specification from filterAndPayloadSpec starting at 'startIdx' and ending just before 'endIdx'. This FilterAndTransform will subscribe to DiagnosticSources specified by the specification and forward them to 'eventSource. For convenience, the 'Next' field is set to the 'next' parameter, so you can easily form linked lists.

Transform spec represents a string that describes how to extract a piece of data from the DiagnosticSource payload. An example string is OUTSTR=EVENT_VALUE.PROP1.PROP2.PROP3 It has a Next field so they can be chained together in a linked list.

parse the strings 'spec' from startIdx to endIdx (points just beyond the last considered char) The syntax is ID1=ID2.ID3.ID4 .... Where ID1= is optional.

Given the DiagnosticSourcePayload 'obj', compute a key-value pair from it. For example if the spec is OUTSTR=EVENT_VALUE.PROP1.PROP2.PROP3 and the ultimate value of PROP3 is 10 then the return key value pair is KeyValuePair("OUTSTR","10")

A public field that can be used to form a linked list.

A PropertySpec represents information needed to fetch a property from and efficiently. Thus it represents a '.PROP' in a TransformSpec (and a transformSpec has a list of these).

Make a new PropertySpec for a property named 'propertyName'. For convenience you can set he 'next' field to form a linked list of PropertySpecs.

Given an object fetch the property that this PropertySpec represents.

A public field that can be used to form a linked list.

PropertyFetch is a helper class. It takes a PropertyInfo and then knows how to efficiently fetch that property from a .NET object (See Fetch method). It hides some slightly complex generic code.

Create a property fetcher from a .NET Reflection PropertyInfo class that represents a property of a particular type.

Given an object, fetch the property that this propertyFech represents.

CallbackObserver is a adapter class that creates an observer (which you can pass to IObservable.Subscribe), and calls the given callback every time the 'next' operation on the IObserver happens.