Delivering video content with the best visual quality-of-experience (QoE) to end users is the central goal of modern video distribution services. User’s QoE is determined by two factors – the quality of the video streamed to the user device, and the quality of the playback when displaying the video on user’s device. The former is typically measured as the presentation quality of the video, meaning the perceptual quality of the video streams when it is played on the user device at its intended speed without interruption. Typical processes and factors that could affect the presentation quality include encoding/transcoding at the hosting servers or networks, and variations of the user device parameters (size, brightness, resolution, etc.). The latter may be interpreted as the playback smoothness, for which the mostly encountered quality issue is video freezing. Freezing may occur either at the beginning before the video starts or in the middle of the playback. It may be caused by insufficient network bandwidth, network delay and instability, unhealthy device buffer and power conditions, and unmatched decoding/display speed, etc.
Traditionally, video presentation quality and playback smoothness are evaluated as independent factors. Standard presentation quality measures include PSNR, SSIM, MS-SSIM, VQM etc. An emerging metric that is attracting increasing attention recently is SSIMPLUS, which is a better option not only for its accuracy and speed but also for its cross-device and cross-resolution capabilities. In practice, one does not have to wait until the video reaches the user device to measure the presentation quality. Instead, it may be well predicted early at the video hosting server, right at the output of encoders/transcoders. By contrast, playback smoothness has to be evaluated down the video delivery chain, either on the user device or near the edge of the network, where instantaneous streaming status information of individual video streaming services is accessible.
Typical playback smoothness based QoE measures use the duration and frequency of freezing events as the key parameters. Apparently, such QoE measures are short-handed because these parameters have nothing to do with the presentation quality of the video. Better solutions may add video bitrate as another parameter with the assumption that bitrate may be used as a guess of the presentation quality. This is again problematic, because it is well noted that different video contents coded at the same bit rate could have dramatically different perceptual quality, not to say the variations in the performance of the encoders being used to generate the videos, and the differences in the physical size, brightness, and pixel resolutions of the user display devices. Even better solutions may incorporate an advanced presentation quality measure, for example, the average SSIMPLUS value of the video, and then combine the presentation quality score with the playback smoothness score to yield an overall Viewer Score measure. This seems to produce a comprehensive solution with all the key factors included.
Interestingly, state-of-the-art research at the University of Waterloo suggests that even this “comprehensive” solution is not enough. What is missing is the interplay between presentation quality and playback smoothness. For example, the impact of a particular freezing event on the overall video QoE depends not only on its own duration but also on the content and perceptual quality of the freezing video frame. More specifically, it has been found that the negative impact of freezing increases with the quality of the freezing frame. In other words, users hate freezing more when watching a higher presentation quality video. This may be no surprise considering that user expectations are higher when watching higher quality videos. More detailed results and discussions can be found in the following publication:
Z. Duanmu, K. Zeng, K. Ma, A. Rehman, and Z. Wang, “A quality-of-experience index for streaming video,” IEEE Journal of Selected Topics in Signal Processing, Nov. 11, No. 1, Feb. 2017.