Metrics

FLAnalytics is an add-on library that serves as a framework for recording comprehensive data metrics related to application, video playback, video downloads, and user events. The library can also be used to collect custom events and metadata. The library's data model is based on the Data Dictionary which is the standard schema used to normalize every piece of data captured in order to generate many popular video streaming quality of experience metrics.

Setup

Datazoom

Quickplay's Player QOE metrics are powered by Datazoom, a video data platform which continually gathers data from endpoints, like a CDN or a video player, through an ecosystem of collectors. Once the data is gathered, it is normalized using standardized data definitions. This data is then sent through available connectors to analytics platforms like Google BigQuery, Google Analytics, and Splunk and can be visualized in tools such as Looker and Superset. Datazoom is your key to a more effective and efficient data pipeline.

Supported SDK version

• DZ_Collector 3.8.1

Application Session

ApplicationSession provides the entry point to the library analytics protocols. It also serves as factory containing APIs to create other analytic sessions (PlaybackSession, UserSession, DownloadSession). Besides this ApplicationSession also provides APIs for reporting app start and report fatal and non-fatal errors.

Create ApplicationSession using the relevant APIs (each Object configuration, analyticsDelegate, commonReportingData are discussed in detail in below sections).

Name	Required	Description
configEndPoint	false	A QP endpoint which enables customised reporting.

import FLAnalytics

// Required to initialize before reporting events.
FLAnalyticsFactory.applicationSession.initialize(configuration: <#T##Configuration#>, commonData: <#T##CommonReportingData#>, configEndPoint: <#T##String?#>, delegate: <#T##AnalyticsDelegate#>, completion: <#T##(Result<Bool, Error>) -> Void#>)

// API to create PlaybackSession
FLAnalyticsFactory.applicationSession.createPlaybackSession()

// API to create UserSession
FLAnalyticsFactory.applicationSession.createUserSession()

// API to create DownloadSession
FLAnalyticsFactory.applicationSession.createDownloadSession()

note

Passing the config url(configEndPoint) on app launch and on success of session initialization making a new API call and parsing the response, and reporting event we need to filter out the Block list events and report it. Since ApplicationSession is a singleton class, FLAnalyticsFactory.applicationSession.initialize should be called before creating sessions and reporing events. The library client can start reporting events only if successful connectivity has been established to analytics system. Also note that ApplicationSession acts as a factory to create the remaining analytic session instances(UserSession, PlaybackSession, DownloadSession).

The library supports connection to multiple analytics provider (supports Datazoom at present) by create configuration to specific analytic provider, below is the one for creating configuration for datazoom.

Datazoom Configuration

Create configuration to initialize datazoom.

Name	Required	Description
Id	true	The unique Datazoom configuration key.

let configuration = Configuration.datazoom(id: <#T##String#>)

AnalyticsDelegate

AnalyticsDelegate provides the callbacks when there is any failure in analytics reporting.

class <class name>: MetricsDelegate {
   /// Listen analytics errors
   func didFail(with error: FLError) {
   }
}

CommonReportingData

The Metadata that is expected to remain constant for the entire duration of a session should be provided using CommonReportingData. It encapsulates all relevant optional and mandatory metadata regarding different aspects of the AnalyticSession.

note

Use API to update any of this common reporting data.

Application

Create application reporting data payload.

Name	Type	Required	Description
name	String	true	The name of the application.
version	String	true	The version of the application.
buildVersion	String	true	The build version of the application.
product	String	false	The customer product.

let appInfo = Application(name: <#T##String#>, version: <#T##String#>, buildVersion: <#T##String#>, product: <#T##String?#>)

User

Create user's reporting data payload.

Name	Type	Required	Description
type	UserType	true	Indicates appropriate user type.
id	String	false	The unique user identifier (eg: Customer Id)
subscription	Subscription	false	User subscription details.
profileID	String	false	The unique identifier for the user's profile

let userInfo = User(id: <#T##String?#>, type: <#T##UserType#>, subscription: <#T##Subscription?#>, profileId: <#T##String?#>)

Subscription

Create Subscription reporting data payload which will be reported as part of above user's payload.

Name	Type	Required	Description
id	String	true	The unique subscription identifier for the user.
ststus	SubscriptionStatus	false	Indicates appropriate subscription status.
plan	String	false	The subscription plan type name.
planId	String	false	The subscription plan type id

let subscription = Subscription(id: <#T##String#>, status: <#T##SubscriptionStatus?#>, plan: <#T##String?#>, planId: <#T##String?#>)

Device

Create user device's reporting data payload.

Name	Type	Required	Description
platformType	PlatformType	true	Represents how user consuming service (eg: APP, WEB)
customDeviceManufacturer	String	false	The manufacturer of the user's device if available.
customDeviceName	String	false	The name of the device model if available.

let deviceInfo = Device(platformType: <#T##PlatformType#>, customDeviceManufacturer: <#T##String?#>, customDeviceName: <#T##String?#>)

Toggle Custom Events

Custom Events reporting to the Analytics server can be toggled on/off remotely. Passing the config end point URL, where the black-listed items are hosted, to the createInstance will restricts the reporting of black-listed custom events

Name	Type	Required	Description
configEndPoint	String	false	The end point where custom events configuration details are hosted

ABTesting

Create ABTesting reporting data payload.

Name	Type	Required	Description
experimentID	String	false	The identifier for the AB testing experiment.
variantID	String	false	The identifier for the AB testing experiment variant.

let abTesting = ABTesting(experimentID: <#T##String?#>, variantID: <#T##String?#>)

Create CommonReportingData that has to be reported across all the events for the entire application lifecycle.

Name	Type	Required	Description
app	Application	true	The Application instance
user	User	false	The User instance.
device	Device	false	The Device instance.
abTesting	ABTesting	false	The [ABTesting] (metrics#abtesting) instance.

let commonData = CommonReportingData(application: <#T##Application#>, user: <#T##User?#>, device: <#T##Device?#>, abTesting: <#T##ABTesting?#>)

Standard Playback Events & Metrics

Create PlaybackSession to report player related events.

let playbackSession = FLAnalyticsFactory.applicationSession.createPlaybackSession()

Attach Player

PlaybackSession provides and API called attachPlayer which takes Player instance as parameter. This API enables the underlying analytics collection tool to access platform native player instance, therefore auto-reporting most of the playback related events.

playbackSession.attachPlayer(player: Player)

Playback events that are auto-captured by library itself

Name	Description
playback_start	This event is reported when the video starts playing for the user.
heartbeat	This event is reported at periodic intervals (1 minute) during content playback.
media_request	This event is reported after the user invokes playback.
player_ready	Signifies when the player has been initialized and is ready for playback.
pause	This event is reported when the player is paused.
resume	This event is reported when the user begins playing after pausing the video.
buffer_start	Buffer Start identifies anytime the player has to wait for the video buffer to fill with video segments.
buffer_end	Event is reported when video starts playing again after a buffer is completed.
stall_start	Stall Start event is reported when playback of video stops because the buffer has been depleted causing an unexpected interruption for the user.
stall_end	Event reported when video starts playing again after a stall and the buffer has been replenished and playback resumes.
error	Thrown if an error occurs during content playback or retrieval of the video.
playback_complete	This event signifies that the video player has reached the end of the currently playing content.
stop	This event is reported when the player has entered a stopped state.
milestone	Reported when the playheadPosition passes a predetermined percentile milestone of the video's duration. Current milestones are fired at the 10, 25, 50, 75, 90 & 95 percentiles.
seek_start	This event records when the user interacts with time controls within the player to move forward or backward in the video timeline.
see_end	Event is reported when the player stops moving the playhead position to jump to a specific point on the timeline.
playing	The media is no longer blocked from playback, and has started playing. Reported when playback resumes from Stall, Buffering or Seek.

Report Custom Playback Events

Report Playback Start

PlaybackSession provides API to report playback_request event.

PlaybackRequest

Content Specific metadata

Create data payload for specific type of contents as below.

SportContent

Create SportContent reporting data payload that is specific to sport event.

Name	Type	Required	Description
id	String	true	The unique identifier for content item.
type	ContentType	true	The type of content to report.
name	String	true	The content name from CMS.
providerId	String	false	The content's provider identifier.
genre	String	false	The genre of the content.
licenseWindowStartDate	String	false	The license window start date. The clear content may not have this parameter.
sportName	String	true	The name of the sport.
leagueName	String	true	The name of the sport league.
team1Name	String	true	The name of the first sport team.
team2Name	String	true	The name of the second sport team.
leagueID	String	false	The unique identifier of the sport league.
leagueExternalID	String	false	The external unique identifier of the sport league.
leagueShortName	String	false	The short name of the sport league.
gameID	String	false	The unique identifier of the sport game.
gameExternalId	String	false	The external unique identifier of the sport game.
venueName	String	false	The name of the sport event venue.
venueCity	String	false	The city where the sport event takes place.
venueCountry	String	false	The country where the sport event takes place.
team1ID	String	false	The unique identifier of the first sport team.
team1ExternalID	String	false	The external unique identifier of the first sport team
team1ShortName	String	false	The short name of the first sport team.
team2ID	String	false	The unique identifier of the second sport team.
team2ExternalID	String	false	he external unique identifier of the second sport team.
team2ShortName	String	false	The short name of the second sport team.

 let contentInfo = Sport(id: <#T##String#>, type: <#T##ContentType#>, name: <#T##String#>, providerId: <#T##String?#>, genre: <#T##String?#>, licenseWindowStartDate: <#T##String?#>, sportName: <#T##String#>, leagueId: <#T##String?#>, leagueExternalId: <#T##String?#>, leagueName: <#T##String#>, leagueShortName: <#T##String?#>, gameId: <#T##String?#>, gameExternalId: <#T##String?#>, venueName: <#T##String?#>, venueCity: <#T##String?#>, venueCountry: <#T##String?#>, team1Id: <#T##String?#>, team1ExternalId: <#T##String?#>, team1Name: <#T##String#>, team1ShortName: <#T##String?#>, team2Id: <#T##String?#>, team2ExternalId: <#T##String?#>, team2Name: <#T##String#>, team2ShortName: <#T##String?#>)

TVShow

Create TVShow reporting data payload that is specific to TV show.

Name	Type	Required	Description
id	String	true	The unique identifier for content item.
type	ContentType	true	The type of content to report.
name	String	true	The content name from CMS.
providerId	String	false	The content's provider identifier.
genre	String	false	The genre of the content.
licenseWindowStartDate	String	false	The license window start date. The clear content may not have this parameter.
seriesID	String	true	The unique identifier of the TV show series.
seriesName	String	true	The name of the TV show series.
seasonNumber	String	true	The season number of the TV show series.
episodeNumber	String	true	The number of one of the episodes in the TV show series.

 let contentInfo = TVShow(id: <#T##String#>, type: <#T##ContentType#>, name: <#T##String#>, providerId: <#T##String?#>, genre: <#T##String?#>, licenseWindowStartDate: <#T##String?#>, seriesId: <#T##String#>, seriesName: <#T##String#>, seasonNumber: <#T##String#>, episodeNumber: <#T##String#>)

MiscellaneousContent

Create MiscellaneousContent video content reporting data payload that belongs to categories other than SportContent or TVShow.

Name	Type	Required	Description
id	String	true	The unique identifier for content item.
type	ContentType	true	The type of content to report.
name	String	true	The content name from CMS.
providerId	String	false	The content's provider identifier.
genre	String	false	The genre of the content.
licenseWindowStartDate	String	false	The license window start date. The clear content may not have this parameter.

 let contentInfo = MiscellaneousContent(id: <#T##String#>, type: <#T##ContentType#>, name: <#T##String#>, genre: <#T##String?#>, providerId: <#T##String?#>, licenseWindowStartDate: <#T##String?#>)

note

If the content type do not fall under any of the categories like SportContent and TVShow then create Content as MiscellaneousContent.

ApplicationContainer

Create ApplicationContainer reporting data payload related to application container.

Name	Type	Required	Description
id	String	true	The unique identifier of the carousel where the content or link to the collection is located.
name	String	true	The name of the carousel where the content or link to the collection is located.

 let container = ApplicationContainer(id: <#T##String#>, name: <#T##String#>)

Create PlaybackRequest reporting data payloads related to video playback request event.

Name	Type	Required	Description
content	Content	true	The Content details.
playbackSource	PlaybackSource	true	The PlaybackSource details.
container	ApplicationContainer	false	The ApplicationContainer details.

 let source = PlaybackSource(type: <#T##PlaybackSourceType#>)

let playbackRequest = PlaybackRequest(content: <#T##Content#>, playbackSource: <#T##PlaybackSource#>, container: <#T##ApplicationContainer?#>)
playbackSession..start(value: playbackRequest)

Handling Playback Request Interruptions

Any interruptions prior to starting the playback (before creating a player) can be reported using PlaybackSession API. Example: Checks related to content authorization (content authentication, parental controls etc)

// Report Interruption start
let interruptStartEvent = CustomEvent(event: <#T##String#>, attributes: <#T##[String : Any]?#>)
playbackSession.interruptStart(value: <#T##CustomEvent#>)

// Report Interruption end
let interruptEndEvent = CustomEvent(event: <#T##String#>, attributes: <#T##[String : Any]?#>)
playbackSession.interruptEnd(value: <#T##CustomEvent#>)

Configure Playback Metadata

Add Playback custom metadata

PlaybackSession provides an API that add / update common metadata to be reported across all playback related events.

playbackSession.updateMetadataRecord(metadata: <#T##[String : Any]#>)

Report Playback Error

To report any error caused like content-auth failure, concurrent streams max limit reached (failure from server but not from player layer as fl-analytics auto-reports player errors) can be reported using stop API in PlaybackSession. Send Error as a parameter(optional) to report playback stop on any error.

playbackSession.stop(error: <#T##FLError?#>)

Report Retry Playback Error Event

Application could retry playback for certain error codes and report the same to metrics to keep track of retries. We recommend creating FLError object with code 2097152 (0x200000) and internal error of the actual error.

// Create retry error object wrapping the internal player error
// `error` object is the error reported through player delegate callback playerDidFail(with error:)
let retryError = FLFoundationFactory.error(errorCode: 2097152, errorDescription: "Player retry attempted, check internal error for actual cause", internalError: error.internalError)

// Report retry event to anaytics
playbackSession.stop(error: retryError)

note

Application while reporting retry error event, should not report the actual error. Instead, they should only report retry error event, so that we avoid duplicate error events reported in metrics.

Value Added Events and Metrics

Application Events

Report Application start

ApplicationSession provides an API to report app_start event along with app start time.

ApplicationMetrics

Create ApplicationMetrics reporting data payload related to application performance metrics.

Name	Type	Required	Description
startUpTime	Long	true	The time, in milliseconds, it takes to start the application.

FLAnalyticsFactory.applicationSession.start(value: ApplicationMetrics(startUpTime: <#T##TimeInterval#>))

Report Application End

ApplicationSession provides an API to report app_end event.

 FLAnalyticsFactory.applicationSession.stop(error: <#T##FLError?#>)

User Events

Create UserSession to report user related events.

let userSession = FLAnalyticsFactory.applicationSession.createUserSession()

UserSession provides the following API to report user action based events.

// Called when [User] executes "signup" flow.
userSession.signup(user: <#T##User#>)

// Called when [User] executes "login" flow.
userSession.login(user: <#T##User#>)

// Called when [User] executes "logout" flow.
userSession.logout(user: <#T##User#>)

// Called when [User] executes "create profile" flow.
userSession.createProfile(user: <#T##User#>)

// Called when [User] executes "update profile" flow.
userSession.updateProfile(user: <#T##User#>)

// Called when [User] executes "start subscription" flow.
userSession.startSubscription(user: <#T##User#>)

// Called when [User] executes "purchase subscription" flow.
userSession.purchaseSubscription(user: <#T##User#>)

// Called when [User] executes "change subscription" flow.
userSession.changeSubscription(user: <#T##User#>)

// Called when [User] executes "complete payment" flow.
userSession.completePayment(user: <#T##User#>)

note

It is client application responsibility to ensure that each user action is triggered in the correct sequence.

Download Events

Create DownloadSession to report download related events.

let downloadSession = FLAnalyticsFactory.applicationSession.createDownloadSession()

Report Download Start

DownloadSession provides API to report download_request event.

DownloadRequest

1. Create Content, video content reporting data payload based on type of content that is being requested. Create SportContent if sport content was requested or TVShow if TV show content was requested, MiscellaneousContent otherwise.

2. Create ApplicationContainer reporting data payload related to application container.

3. Create DownloadRequest reporting data payloads related to video playback request event.

   Name	Type	Required	Description
   content	Content	true	The Content details.
   playbackSource	PlaybackSource	true	The PlaybackSource details.
   container	ApplicationContainer	false	The ApplicationContainer details.

   let source = PlaybackSource(type: <#T##PlaybackSourceType#>)
   
   let downloadRequest = FLAnalytics.DownloadRequest(content: <#T##Content#>, playbackSource: <#T##PlaybackSource#>, container: <#T##ApplicationContainer?#>)
   downloadSession.start(value: downloadRequest)

Download events that are auto-captured by library itself

Name	Description
download_prepared	This event is reported when a download is prepared
download_started	This event is reported when a download is started
download_paused	This event is reported when a download is paused
download_resumed	This event is reported when a download is resumed
download_stopped	This event is reported when a download is stopped
download_completed	This event is reported when a download is completed
download_deleted	This event is reported when a download is deleted

Report Custom Events

ApplicationSession provides an API to report custom events. The library provides MiscellaneousEvent names for the client application to reuse.

FLAnalyticsFactory.applicationSession.addEvent(event: <#T##String#>, attributes: <#T##[String : Any]?#>)

note

1. In order to conform to the existing reporting scheme the event name should be provided in the lower case format and words should be separated by the underscore character e.g."playback_request"
2. In datazoom, any custom event will be reported as event name with custom_ prefix. (eg: playback_start as custom_playback_start)

caution

When trying to report an custom event with a name that is already reserved (the list of reserved event names is captured by QP_ANALYTICS_EVENTS), library will throw an error.

Configure Application Metadata

Add Application custom metadata

Any additional metadata that is to be attached to all application / playback / user events can be done using the the below API. All subsequent events will report this additional data once it is defined. Also use this api to update the metadata.

FLAnalyticsFactory.applicationSession.updateMetadataRecord(metadata: <#T##[String : Any]#>)

Application Errors

ApplicationSession provides APIs to report both fatal and non-fatal errors.

Report Fatal errors

Any application fatal error can be reported using below API. Send Error as a parameter(optional) to report playback stop on any error.

FLAnalyticsFactory.applicationSession.stop(error: <#T##FLError?#>)

Report Non-fatal errors

Any application non-fatal error can be reported using below API. Send FLError to report playback stop on any error.

FLAnalyticsFactory.applicationSession.addErrorEvent(error: <#T##FLError#>)

Download Errors

Any Download error can be reported using below API. Send Error to report download stop on any error.

downloadSession.stop(error: <#T##FLError?#>)

User Action Errors

Any User error can be reported using below API. Create UserError reporting data payload in case where one of user's action fails and User attribute along with it.

Create User Error

Name	Type	Required	Description
code	String	true	The unique error code.
message	String	true	The message describing the error.
description	String	false	The contextual description of the error.

let userError = UserError(code: <#T##String#>, message: <#T##String#>, description: <#T##String?#>)
userSession.addErrorEvent(error: <#T##UserError#>, value: <#T##User#>)

Out-of-box Player Metrics

Key Player Metrics

• Playback Startup Time - Playback startup time is the time taken for player to start the playback measured from the moment playback was requested.

• AverageVideoBitrate - The average bitrate of video track if it is unmuxed. Measured in bits per second.

• AverageAudioBitrate - The average bitrate of audio track. Measured in bits per second.

• IndicatedBitrate - The throughput required to play the stream, as advertised by the server. Measured in bits per second.

• DroppedFrames - The total number of dropped video frames.

• ObservedBandwidth - Provides the Observed bandwidth.

• BufferedRange - The accumulated duration of the media downloaded in the buffer so far (ratio of currentPosition and bufferedDuration).

• Playback Interruption Rate - Playback is interrupted whenever the media chunks are not available at a rate equivalent to the rate at which the player consumes the chunks.

• Playback Interruption Wait Ratio - Time taken to recover from playback interruption depends on two factors primarily, network bandwidth and variant availability.

• Media Quality of Playback - The measure of streaming quality is a number which says, is the user presented with the best possible media quality.

• Playback Failure Rate - The measure of total failures rate

AVPlayer provides playback statistics and error statistics via AVPlayerItemAccessLogEvent and AVPlayerItemErrorLogEvent objects. The below code snippet demonstrates how to get these informations. The AVPlayer instance is made available through rawPlayer property on FLPlayer.

// Register to receive notifications whenever a new access log entry or entry log entry is available with AVPlayer
override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)

    NotificationCenter.default.addObserver(self,
    selector: #selector(self.handlePlayerItemNewErrorLogEntry(notification:)),
        name: .AVPlayerItemNewErrorLogEntry,
      object: nil)

    NotificationCenter.default.addObserver(self,
    selector: #selector(self.handlePlayerItemNewAccessLogEntry(notification:)),
        name: .AVPlayerItemNewAccessLogEntry,
      object: nil)
}

@objc func handlePlayerItemNewErrorLogEntry(notification _: Notification) {
    guard let avplayer = player?.rawPlayer as? AVPlayer,
        let currentItem = avplayer.currentItem,
        let errorLog = currentItem.accessLog() else { return }
    if let errorLogEvent = errorLog.events.last {
        // process error log event
    }
}
@objc func handlePlayerItemNewAccessLogEntry(notification _: Notification) {
    guard let avplayer = player?.rawPlayer as? AVPlayer,
        let currentItem = avplayer.currentItem,
        let accessLog = currentItem.accessLog() else { return }
    if let accessLogEvent = accessLog.events.last {
        // process access log event
    }
}

For further reading on access log and error log events, please check Measure Playback Performance section of apple document¹.

note

Please refer to PlayerStats class in the FLPlayerSampleApp that comes with the library package.