An Evaluation of Video Quality Assessment Metrics for Passive Gaming Video Streaming

Nabajeet Barman, Steven Schmidt, Saman Zadtootaghaj, Maria G. Martini, Sebastian Möller

In: Proceedings of the 23rd Packet Video Workshop. Packet Video Workshop (PV-18) located at ACM Multimedia Systems Conference (MMSys 2018) June 12-15 Amsterdam Netherlands Pages 7-12 ISBN 978-1-4503-5773-9 ACM New York, NY, USA 2018.


Video Quality assessment is imperative to estimate and hence manage the Quality of Experience (QoE) in video streaming applications to the end-user. Recent years have seen a tremendous advancement in the field of objective video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. However, no work so far has attempted to study the performance of such quality assessment metrics on gaming videos, which are artificial and synthetic and have different streaming requirements than traditionally streamed videos. Towards this end, we present in this paper a study of the performance of objective quality assessment metrics for gaming videos considering passive streaming applications. Objective quality assessment considering eight widely used VQA metrics is performed on a dataset of 24 reference videos and 576 compressed sequences obtained by encoding them at 24 different resolution-bitrate pairs. We present an evaluation of the performance behavior of the VQA metrics. Our results indicate that VMAF predicts subjective video quality ratings the best, while NIQE turns out to be a promising alternative as a no-reference metric in some scenarios.

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz