Metrics
Relativity Testing Framework API collects metrics using ApplicationInsights to provide more context around usage and debugging. In addition to the default information collected from ApplicationInsights, we also collect information about the parameters that are sent with the API request.
What is collected
Default ApplicationInsights Metrics
| Measurement | Description | Example |
|---|---|---|
| Event Time | The local time of the event. | 2/12/2021, 9:23:05 AM (Local time) |
| Event Name | The name of the event. | RelativityApplicationService.IsInstalledInWorkspace |
| Telemetry Type | The type of measurement. | customEvent |
| Device Type | Browser(JavaScript) or PC. | PC |
| Client IP Address | The IP address of the client device. | 0.0.0.0 |
| City | Scraped using https://dev.maxmind.com/geoip/geoip2/geolite2/ | Des Moines |
| State or Province | Scraped using https://dev.maxmind.com/geoip/geoip2/geolite2/ | Iowa |
| Country or region | Scraped using https://dev.maxmind.com/geoip/geoip2/geolite2/ | United States |
| Cloud role instance | The name of the computer running the code. | a01dpjkdeaba014.kCura.corp |
| SDK version | Version of Application insights being used. | dotnet:2.15.0-4479 |
| Sample rate | Number of samples. | 1 |
Custom Event Metrics
| Measurement | Description | Example |
|---|---|---|
| RelativityTestingFrameworkVersion | The version of RTF that is running. | 1.1.1 |
| RelativityVersion | The version of Relativity that RTF is running against. | 12.1.2.3 |
| TestAssemblyName | The name of the assembly that NUnit is running. | Relativity.Testing.Framework.Api.FunctionalTests |
| Parameters | Contains all parameters you send into a strategy. Note that object parameters will not be expanded. | 1015024 && 1018047 |
| Method | The name of the method/strategy being called. | IsInstalledInWorkspace |
| Class | The name of the class/service running. | RelativityApplicationService |
| ProcessingTime | The time in the milliseconds that the method took to run. | 1123.897 |
| RingSetupVersion | The version of the Relativity.Testing.Framework.RingSetup assembly, if loaded | 0.13.0 |
| Hostname | The hostname of the Relativity environment that RTF is running against. | P-DV-VM-CUP7WET |
Metrics Not Tracked during UsageOnly Mode: Method, Class and Parameters
API Error Metrics
| Measurement | Description | Example |
|---|---|---|
| Message | Contains error message. Might potentially contain sensitive information (information about the code being run) | Validation failed: -- The entered E-Mail Address is already associated with another user in the system. ... |
| Exception Type | Contains the type of exception. | System.Net.Http.HttpRequestException |
| Failed method | Contains info about a method where an exception was thrown. | Relativity.Testing.Framework.Api.HttpService.CheckResponseStatus |
| Call stack | The call stack of an error. Might potentially contain sensitive information (information about the code being run). | System.Net.Http.HttpRequestException: at Relativity.Testing.Framework.Api.HttpService.CheckResponseStatus (Relativity.Testing.Framework.Api, ... |
Metrics Not Tracked during UsageOnly Mode: Message and Call Stack
Opting out of Metrics Collection
If you wish to opt out of automatic metrics collection, set the EnableApplicationInsights TestRunParameter to UsageOnly or None.
<RunSettings>
<TestRunParameters>
<Parameter name="ServerBindingType" value="https" />
<Parameter name="RelativityHostAddress" value="P-DV-VM-CUP7WET" />
<Parameter name="AdminUsername" value="relativity.admin@kcura.com" />
<Parameter name="AdminPassword" value="APassword1234!" />
<Parameter name="RestServicesHostAddress" value="P-DV-VM-CUP7WET" />
<Parameter name="WebApiHostAddress" value="P-DV-VM-CUP7WET" />
<Parameter name="EnableApplicationInsights" value="None" />
</TestRunParameters>
</RunSettings>