Quality Assurance Assessment for Schools

Good quality Audio & Video in a videoconference is paramount to the success of the conference and prevents users from becoming disillusioned with the technology. One way of maintaining a high quality is to ensure that all endpoints using the JANET Videoconferencing Service (JVCS) are assessed on registration to check the quality of the audio, video and network connection. These checks are carried out remotely in a one-to-one videoconference with the JVCS Management Centre using a Multipoint Control Unit (MCU). The operations staff who carry out the assessment are very experienced in interpreting an acceptable level of quality and will talk you through the assessment which will take approximately 15 minutes.

This document details what is involved in carrying out a Quality Assurance Assessment (QAA).

Background

QAAs are carried out of both ISDN & IP endpoints. Should an endpoint have the facility to use both IP & ISDN technologies then an assessment should be carried out on both, separately. On completion of a QA Assessment a report will be sent to the administrator responsible for the endpoint with a summary of the assessment results along with any suggestions for improving quality (where applicable). A further QA Assessment may be carried out where a complaint has been received in regard to the performance of an endpoint in a scheduled conference. This is to try to alleviate further disruption to videoconferences. It is recommended that QA Assessments are carried out on a six-monthly basis to ensure that a high standard of videoconferencing is maintained.

Users may also request an additional QA Assessment when a change to their audio and/or video equipment has been made.

Ancillary Information

The JANET Videoconferencing Service provides technical information on all aspects of videoconferencing and can be contacted for technical assistance. See www.video.ja.net for further details.

The JANET Videoconferencing web-site at www.jvcs.ja.net provides a full description of available videoconferencing services with useful links to other technical sites.

The QA Assessment

New venues must be added to the booking service by an RBC/LA administrator prior to taking part in a QA Assessment. New users must register their details with the JVCS booking service by completing the on-line registration .

The audio and video quality from the JANET registered venues are assessed by experienced operations staff at the JANET Videoconferencing Management Centre. Tests are carried out both subjectively and objectively by using measuring equipment and by examining statistics collated on the Multipoint Control Unit (MCU).

The MCU will dial out to venues with IP connectivity using an E.164 number. However venues with ISDN connectivity are requested to dial into the MCU using the ISDN number assigned to them when the QA session is booked. The call will be set up using the highest speed that the remote endpoint can support. The MCU automatically negotiates the incoming audio and video protocol from the remote endpoint.

Audio Assessment

The venue's microphone audio output level is measured by equipment located in the Management Centre. All microphones are tested individually to ensure that the levels fall within a recommended threshold. The following levels are monitored:

Microphone Level - you will be asked to speak in to each microphone, one after another.
Mute Level - you will be asked to mute all microphones
Ambient Noise Level . you will be asked to unmute all microphones and remain quiet.

Echo

An echo is a single reflection of sound that mimics the original after a noticeable time delay. It can cause severe disruption to videoconferences. The Management Centre will conduct subjective tests to check for the presence of echo. The results will be assessed using the following five point Linkert scale:

5 - Imperceptible
4 - Perceptible, but not annoying
3 - Slightly annoying
2 - Annoying
1 - Very annoying

Double Talk

Double talk measures the quality of audio when two or more parties talk simultaneously. The Management Centre will conduct subjective tests of the audio under double talk conditions. The results will be assessed using the following five point Linkert scale:

5 - No impairment
4 - Perceptible, not annoying
3 - Slightly annoying
2 - Annoying
1 - Very annoying

Overall Audio Quality Rating

The overall quality of the audio to and from the venue is measured by using the following five point Linkert scale:

5 - Excellent
4 - Good
3 - Fair
2 - Poor
1 - Bad

Video Assessment

The video protocol used by the endpoint is auto-negotiated by the MCU during the call setup.

The quality of video to and from the venue is assessed by conducting two tests :

Picture Quality

A subjective test of picture quality is conducted between the Management Center and the endpoint. The results will be assessed using the following five point Linkert scale:

5 - Excellent
4 - Good
3 - fair
2 - Poor
1 - Bad

Incidence of Impairment

The Management Centre carries out a subjective test of the incoming video to determine the existence and nature of any incidence of impairment. The results will be assessed using the following five point Linkert scale:

1 - A single short burst < 1 Second
2 - Several short bursts < 1 Second
3 - A single short period 1 Second & < 5 seconds
4 - Several short periods 1 Second & < 5 Seconds
5 - A single prolonged period 5 Seconds
6 - Throughout
7 - None

Network Statistics

Venues with an IP connection will traverse a number of networks to establish a videoconference. Therefore the performance of intervening networks is important to the quality of the videoconference.

The Management Centre will evaluate the overall network performance by examining the order of audio and video packets entering and leaving the MCU. Any irregularities observed with the packets order may signify a problem with intervening networks and can deteriorate the quality or even prevent a videoconference from being established.

Typical causes of network deterioration and faults are: packet loss, errors, delay, jitter and duplex mismatch. All will have to be avoided before a successful videoconference can take place.

Conclusion

The QAA aims to assess the suitability of venues to participate in a multipoint videoconferencing without causing disruption. The results of the assessment are recorded and are emailed to the endpoint's administrator.