Speech quality assessment in crowdsourcing: Influence of environmental noise
* Presenting author
Micro-task crowdsourcing offers new possibilities for QoE research including investigation of realistic environmental influence factors due to the real-life environment of the subjects. This paper reports the influence of environmental noise on speech quality assessment ratings. In a two-phase experiment, subjects assessed the quality of speech stimuli using either their smartphone (and ear-plug) or a provided one, in different environments. The SwissQual 501 speech database from the ITU-T Rec. P.863 competition (kindly provided by SwissQual AG) was used. Phase A was conducted in the laboratory, in either silent or background noise simulation environments. In phase B, the same participants completed the same task in one of crowdsourcing environments. MOS values were calculated for each condition and compared to the scores reported from the standard laboratory test. The silent-laboratory condition achieved the highest correlation (r=.95). The correlation was improved when subjects were wearing in-ear headphones in the simulated noisy environments and also when they were listening in the living-room or unspecified locations (r=.89) in the crowdsourcing conditions. It was found that the loudness of the stimuli has the highest influence on the difference between MOS values obtained in different conditions and the standard laboratory scores.