VTAS - Product Evaluations

JANET Video Technology Advisory Service (VTAS)

VTAS Home | User Guides | Reports | Services | Resources

EVALUATION OF ISDN/IP VIDEOCONFERENCING EQUIPMENT

Reference:	GD/VTAS/012
Version:	1.24
Date:	08.05.2008
Author:	Jim Sheach, MALTS, University of Edinburgh; Urwin Wood, ISS-TV Services, Newcastle University

1. OVERVIEW
2. THE EQUIPMENT TEST PROCEDURE
3. SUMMARY TABLES
4. IMPORTANT FACTORS/CONSIDERATIONS
5. TRADEMARKS AND ACKNOWLEDGEMENTS
- 5.1 Trademarks
- 5.2 Further Acknowledgements
APPENDIX A -Videoconferencing Equipment Test Procedure

The following test results have been presented as print-optimised pdf files. To view these you will need Adobe Acrobat Reader, available free from http://www.adobe.com

The manufacturers have been invited to comment. Where applicable, manufacturer comments are listed alongside the evaluation report.

			Test Date
Appendix 08/01 - LifeSize® Express			April 2008
Appendix 07/04 - Tandberg Content Server			Feb 2008
Appendix 07/03 - Tandberg Profile 6000 MXP			Nov 2007
Appendix 07/02 - Sony PCS-HG90			Nov 2007
Appendix 07/01 - Polycom RSS 2000			June 2007
Appendix 06/06 - LifeSize Team™			May 2007
Appendix 06/05 - Polycom HDX 9004			Mar 2007
Appendix 06/04 - ZTE ZXT500			Feb 2007
Appendix 06/03 - Tandberg Edge 95 MXP			Dec 2006
Appendix 06/02 - Codian IP VCR 2210			Oct 2006
Appendix 06/01 - LifeSize Room™			Sept 2006
Appendix 05/06 - Tandberg 990 MXP			May 2006
Appendix 05/05 - Aethra Vega X3			Mar 2006
Appendix 05/04 - Aethra Vega X5			Dec 2005
Appendix 05/03 - Tandberg 6000 MXP			Sept 2005
Appendix 05/02 - Polycom VSX6000			June 2005
Appendix 05/01 - Sony PCS-G70P			April 2005

Previous product evaluations may be found in the ARCHIVE

1. OVERVIEW

The Video Technology Advisory Service (VTAS) undertakes a number of technical studies including the evaluation of videoconferencing equipment. The main goal of this evaluation is to provide objective advice for higher and further education and research organisations so that they may make informed choices when purchasing videoconferencing equipment for use over ISDN (Integrated Services Digital Network), IP (Internet Protocol), and other networks.

To enable conferencing between more than two sites, the JANET Videoconferencing Service (JVCS) provides multipoint facilities for both ISDN and IP networks enabling multi-site communications between suitably equipped conference rooms and desktop systems. To be compatible all equipment must conform either to the ITU-T (International Telecommunications Union - Telecommunications Standardization Section) H.320 series of recommendations implemented for transmission over the ISDN switched system or to the ITU-T H.323 series of recommendations for transmission over IP networks.

Videoconferencing systems comprise:

Input Devices - Microphones, television cameras, white boards, etc. that provide the sound, vision and data input signals.
Coding-Decoding Equipment - The CODEC (COder-DECoder) that provides the analogue/digital interface for the input devices, the digital/analogue interface for the output devices and all the compression and signal processing necessary.
Output Devices - Loudspeakers, television picture monitors or data screens, white boards, etc. that generate the sound, vision and data output information at each site.

Recording Devices may also be used. Some, such as the Codian IP VCR, allow a conference to be recorded for later transmission or viewing either as a point to point, multipoint or a streamed conference.

There is a wide range of equipment on the market, matched by a wide variation of needs across the JANET community. The two main categories are:

Room-based Systems - Full facilities, ISDN2 - ISDN30 and IP network capability. Prices from 5,000. Not designed to be moved.
Desktop (PC based) Systems - Plug-in cards for a PC can provide ISDN and IP interfaces. Prices 500 - 3,000 (plus the cost of the PC). USB connected devices can provide IP-only interfaces which may easily be moved between host PCs or laptops, these devices are Windows only. Prices 400 - 600 (plus the cost of the PC).

Some overlap occurs between categories, e.g. some sophisticated PC based systems may be mounted on a trolley with a large television picture monitor and marketed as a rollabout or room system. In this document, compact portable CODECs are included with the room based systems, but it is worth noting that these compact portables are slightly different - they generally offer limited facilities for a relatively low cost.

The growth of interest and demand for videoconferencing generally in the JANET community, combined with the multipoint facilities of the JVCS, has created a need for objective advice on products that are standards compliant and able to work effectively in the JVCS environment.

Equipment may be offered to operate over a variety of networks. ISDN and IP systems require their own compression algorithms to achieve transmission, e.g. ISDN requires equipment conforming to the ITU-T H.320 standard whereas IP requires equipment to the ITU-T H.323 recommendations. Some products use other compression algorithms e.g. MPEG-1 and MPEG-2. Although these products are not supported by JVCS, non-standard equipment will still operate point-to-point, and so if conferencing is only required between two sites, e.g. between sites on a split campus, then these could be a good choice.

The list of manufacturers represented here is not exhaustive. VTAS would be pleased to consider suggestions for future products and suppliers although the in-depth evaluation of equipment is a time consuming process and VTAS is currently only able to test a limited number of products each year. The presence of a product in this list indicates only that the product has been evaluated by the VTAS product evaluation team, and should not be taken as a recommendation. Similarly, the absence of a popular product from the list should not be seen to reflect negatively on that product.

The evaluation process produces a snapshot of the features and performance of a product at a specific moment in time. It is worth noting that a significant period of time may have elapsed between the date of testing and the time of reading this report, and during this period some products may have been updated by the manufacturer or even superseded. Full technical specifications of all the products evaluated (together with many others) are included in the manufacturers' web sites. It is strongly recommended that these sites be read in conjunction with the test reports and their associated comments, so that a more complete assessment of a product is obtained.

Conferencing equipment is only one of the components necessary for videoconferencing. The effective and efficient use of videoconferencing is also dependent on the room environment, the network, booking procedures, etc. For further information see other material available on this site.

top

2. THE EQUIPMENT TEST PROCEDURE

The evaluations are conducted between two University sites, Newcastle and Edinburgh. Tests are also conducted through the Multipoint Control Unit (MCU) facilities, located across the network and managed by the JVCS Management Centre, which is located at the University of Edinburgh. The procedure was designed to test most aspects of the CODEC. Tests are carried out at ISDN 128, 384 and 768 kbit/s together with IP 384, 768 and higher speeds such as 1Mbit/s, 2Mbit/s, 4Mbit/s and higher. When other coding standards are used e.g. MPEG-1, MPEG-2, the testing will include network data rates appropriate to the particular product. High Definition (HD) systems will also be tested in their appropriate wide screen formats i.e. w720p, w448p, w228p etc.

Technical considerations such as audio quality and vision quality are paramount, and the ability to transfer data transparently is a desirable feature. However, just as important for the user, especially non-technical staff, is the ease of setting up and operating the equipment, dialling the remote site and the reliability of connections during a conference.

The American National Standards Institute (ANSI) has carried out extensive objective and subjective assessments of image degradation during videoconferencing. For our test procedure we used similar images to those described in the ANSI document Video test scenes for subjective and objective performance assessment in videoconferencing (ANSI 801.1).

In order to minimise possible variations between different assessors, it was decided that only one site (The University of Newcastle) would carry out subjective impairment tests. All tests were conducted using a standard test tape (recorded on U-Matic SP tape format) at both sites. The tape included:

test signals to check system gain and a selection of scenes typical of videoconferencing, e.g. talking heads, nodding heads, slow and fast zooming, close ups and wide angle shots;
images which allow us to test the legibility of text (various sizes) and a large map;
a person teaching at a flip-chart;
a moving football (off air) sequence.

Jerkiness was assessed by a method devised specifically for this evaluation. A test pattern comprising blocks of colour joined by spokes of a wheel was rotated at four fixed synchronous speeds on a record turntable (driven from a tone oscillator through a power amplifier). Any jerkiness in the images was immediately apparent.

The test procedure can be summarised as follows. The complete test schedule is set out in Appendix A.

2.1 Vision Signal Tests
Objective signal levels are measured using analogue test signals to verify that vision gain settings are accurate and that the colour information is not degraded unduly. For some PC based products external connections are not provided, in these products cameras feed directly into the PC USB (Universal Serial Bus) port and the received signals are fed directly to the PC Visual Display Unit (VDU). For this equipment it is therefore not possible to conduct all of the tests. In these cases the test tape can not be viewed so the only method of judging the vision quality is by subjectively assessing the normal conference video through the PC screen. It is possible that additional signal degradation could be introduced by the PC VDU. This potential extra source of degradation is outside of the scope of the testing procedure.

A series of subjective vision assessments are then carried out by referring to an agreed scale of degradation:

Imperceptible	1
Perceptible	2
Slightly annoying	3
Annoying	4
Very annoying	5

The following specific impairments were assessed where appropriate:

lip synchronisation (LS);
block distortion (tiling) (BLK);
blurring (BLR);
colour errors (CLR);
jerkiness (distortion of smooth motion) (JRK);
object persistence (image retention from a previous frame) (OP);
scene cut response (time to build up a new image) (SCR).

2.2 Audio Signal Tests
Where appropriate, frequency response and headroom (i.e. the capacity for signal overload or peaks) are measured for the following audio standards:

Standard	Data rate
G.711 (nominal bandwidth 3.5kHz)	48, 56 or 64kbit/s
G.722 (nominal bandwidth 7kHz)	48, 56 or 64kbit/s
G.728 (nominal bandwidth 3.5kHz)	16kbit/s
MPEG-4 AAC-LD	64 kbit/s and 128kbit/s

The G.728 standard is important for low data rate conferences (i.e. ISDN 2, 128kbit/s) to allow adequate bandwidth for the vision signal.

For MPEG-1 and MPEG-2 products, the appropriate audio coding standard is tested e.g. ISO/IEC 11172-3 Layer II 16bit 48KHz Stereo.

MPEG-4 AAC-LD coding achieves high quality with a low transit delay (latency) during a conference and has both 64kbit/s and 128kbit/s coding standards.

Some manufacturers provide proprietary audio coding, to achieve extended frequency response, but this does require each site to have the appropriate coding within the CODECs.

Siren 14.24 (nominal bandwidth 14 kHz)	H.320 data rate 128 kbit/s
Siren 14.32 (nominal bandwidth 14 kHz)	H.320 data rate 384 kbit/s
Siren 14.48 (nominal bandwidth 14 kHz)	H.323 up to 4Mbit/s

To ensure products will work effectively through the JVCS MCU service an additional audio level check is carried out. A connection is made to the JVCS management centre and the received audio level on normal speech is measured. A received level peaking between 0dBm to +4dBm is acceptable. This test is identical to that carried out for a pre-registration QA test.

The operation of echo cancellers is tested in some detail, in both a conference room and in the more demanding environment of a small lecture theatre seating 100.

As for the video tests some Desktop products are not provided with external audio output connections so objective measurements cannot be taken.

2.3 Data Transfer
Products can offer a variety of methods of transferring data between sites. The ITU-T recommendation for data interchange over H.320 and H.323 is T.120. For products that offer this, files are transferred between sites and the effect on the video and audio signals is monitored. If other methods are offered, e.g. a separate signal (eg Dual Video), channel or perhaps the Internet, then these will also be tested.

2.4 Ease of Use
This is considered a prime requirement so several aspects are assessed:

quality of documentation;
type and method of connections to peripheral equipment;
set-up procedure;
methods of operation, i.e. remote controls, menus, etc;
portability of equipment.

2.5 Standards Conformance - Inter-working
If the unit is in any way incompatible in either point-to-point or multipoint situations, then a conference will not be successful. Both point-to-point compatibility and compatibility through the JVCS MCUs are examined.

2.6 Network Connections
In this section, the ease of dialing a remote site on ISDN2 and ISDN6, or the ease of storing and recalling IP/DNS addresses is examined. The effect of losing one circuit on an ISDN6 call is also examined. The IP connection is deliberately disconnected and reconnected to simulate a connection failure. The time the CODEC requires to re-establish the conference is then measured.

2.6.1 ISDN Networks
ISDN switched networks provide Guaranteed Quality of Service (GQoS). For all intents and purposes the network can be viewed as transparent (up to the bandwidth provided). If an ISDN6 (384kbit/s) circuit is dialled then 384kbit/s can be relied on to be available continuously for the duration of the conference. Creating a reliable connection between ISDN sites is not always straightforward. Different manufacturers approach the method of connection in various ways. If the conference system is 'intelligent' it will iron out many of these difficulties and make connection easier and more reliable.

2.6.2 IP Networks
IP connections over LANs (Local Area Networks), MANs (Metropolitan Area Networks) and WANs (Wide Area Networks) using H.323 compliant equipment may not provide a Guaranteed Quality of Service. This means that during a conference the operational bandwidth may or may not be provided for some or all of the time. IP networks normally carry a significant amount of other data traffic so the quality of the circuit provided will depend on the density and frequency of this competing traffic. The effect of bandwidth being robbed from the IP videoconferencing connection can be quite dramatic. At its worst the intelligibility of the sound can be lost and the picture can break up completely. Some networks can prioritise IP for specific types of traffic including videoconferencing. Another approach is to provide so much bandwidth over the network that saturation cannot occur, however this is an expensive solution. During VTAS testing of IP products the network is optimised by eliminating local bottlenecks i.e. CODECs are connected directly to network routers to avoid congested local networks. This does not represent a typical IP network but does enable the limiting audio and video performance of the CODEC alone to be realistically assessed without other factors interfering.

top

3. SUMMARY TABLES

The results of the products tested are summarised in the tables below. Full results of the tests are detailed in the individual product test reports. To make comparison more meaningful for prospective purchasers, only recent products tested will be shown together with those tested during the preceding two years. Older products tested are included for reference under ARCHIVE.

3.1 Video Performance
Sixteen separate tests were carried out, involving test signals and scenes typically found during a videoconference. Where fitting, each test was used to assess blocking, blurring, jerkiness, colour response, persistence and scene cut response. Testing was carried out at ISDN data rates of 128kbit/s - 384kbit/s and for data compression algorithms H.261, H.263, H.264, and up to 60 frames per second (fps) options, as appropriate. For those products with H.323, i.e. IP, CODECs tests were repeated at IP 384 - 8Mb/s, as appropriate. For MPEG-1 products, data rates appropriate to the equipment were tested. A quick comparison of video performance can be made from the summary tables below.

The summary table scores were obtained by first calculating the sum of all scores for each particular test, (e.g. for test 1, adding figures for blocking, blurring, jerkiness and colour response), and then combining the totals for tests 1 - 15. The lower the score, the better, so a perfect product, scoring 1 for each measurement, would have produced a total score of 67 in the table. While not giving the whole picture (i.e. indicating the performance of individual products in a particular test such as fast moving images), the table does provide a very useful indicator to overall video performance and is in line with overall subjective impressions. The tables should be examined closely as rather contrary to expectation not all the latest products perform better than products they replace. The 'Category' classification indicates whether the product is a room-based (R), or PC-based (PC) solution.

Table 1 H.320 (ISDN) Video Performance
	128kbit/s			384kbit/s			Category
	H.261	H.263	H.264	H.261	H.263	H.264
Aethra Vega X3	205	199	188				R
Aethra Vega X5	205	197	187	147	134	118	R
Polycom V500	See report for details						R
Sony PCS-G70P	196	182	172	141	134	110	R
Tandberg 990 MXP	185	182	153	126	117	104	R
Tandberg Edge 95 MXP						123	R

Table 2 H.323 (IP) Video Performance

H.261	384kbit/s	768kbit/s	1Mbit/s	2Mbit/s	4Mbit/s
Aethra Vega X3	150	122		88
Aethra Vega X5	144	113		80
Sony PCS-G70P	141	113		91
Tandberg 6000 MXP	127	100		80(1)
Tandberg 990 MXP	126	104		83
Tandberg Profile 6000 MXP	132	104	94	87
ZTE ZXT500	131	108		89

High Definition CODECs identified by shading.
1. Tandberg 6000 CODEC scores at 3Mbit/s not 4Mbit/s
2. LifeSizeRoom and Team CODECs score at 2.5Mbit/s not 2 Mbit/s

Table 3 H.323 (IP) Video Performance

H.263	384kbit/s	768kbit/s	1Mbit/s	2Mbit/s	4Mbit/s
Aethra Vega X3	136	103		81
Aethra Vega X5	133	94		78
Sony PCS-G70P	136	99			85
Tandberg 6000 MXP	120	92
Tandberg 990 MXP	116	97		78
Tandberg Profile 6000 MXP	124	95	82		75
ZTE ZXT500	118	86		78

High Definition CODECs identified by shading.
1. Tandberg 6000 CODEC scores at 3Mbit/s not 4Mbit/s
2. LifeSizeRoom and Team CODECs score at 2.5Mbit/s not 2 Mbit/s

Table 4 H.323 (IP) Video Performance

H.264	384kbit/s	768kbit/s	1Mbit/s	2Mbit/s	4Mbit/s	8Mbit/s
Aethra Vega X3	121	96
Aethra Vega X5	116	90
HD LifeSize RoomT	123	104	95	75(2)
HD LifeSize TeamT	120	100	91	82(2)
HD LifeSize Express	124	100	97	85
Polycom VSX 6000	108	86
Polycom HDX 9004	132	114	103	95	89
Sony PCS-G70P	113	90			82
Sony PCS-HG90				91	73	69
Tandberg 6000 MXP	109	90
Tandberg 990 MXP	111	83		86
Tandberg Edge 95 MXP	123	93		86
Tandberg Profile 6000MXP	122	96	88	74
ZTE ZXT500	108	91		74

High Definition CODECs identified by shading.
1. Tandberg 6000 CODEC scores at 3Mbit/s not 4Mbit/s
2. LifeSizeRoom and Team CODECs score at 2.5Mbit/s not 2 Mbit/s

3.2 Recording Devices
To assess the transparency of recording devices, comparative tests were carried out. Subjective scores were taken for the direct point-to-point link and then a playback from the recording device, as outlined below:

A point-to-point connection between a Tandberg 6000 and a Polycom VSX8000.
A recording from the Tandberg 6000 to the RSS2000 and then replayed from the RSS 2000 to a Polycom VSX 8000.

All devices were set to auto-negotiate audio and video protocols.

Table 5 H.323 (IP) Video Performance of Recording Devices
Recording Device	384kbit/s		768kbit/s		2Mbit/s
	Point-to-point	Recording	Point-to-point	Recording	Point-to-point	Recording
Codian IP VCR 2210	124	131	103	109	81	84
Polycom RSS 2000	128	138	99	102	78	81

3.3 Streaming Devices

For streaming devices such as the Tandberg Content Server that records H.323 conferences and then live-streams, archives and subsequently replays the conferences as a media stream it is inappropriate to conduct the normal CODEC video, audio and data tests. Refer to the relevant test report for more detailed information.

3.4 Audio Performance
Echo cancellation is essential for hands free, natural videoconferencing, i.e. without the use of telephone handsets or headphones. Large lecture theatres can be very reverberant so efficient echo cancellation is necessary for good results. In the past a separate, external echo canceller was recommended for good results, but as CODECs now generally incorporate excellent echo cancellation within the unit, a separate echo canceller may not be necessary. The ability for both sites to converse simultaneously is referred to as double talk. This is the natural way to conference, but efficient echo cancellation is needed in order to enable double talk. Most of the products achieving a good or better score under echo cancellation in Table 6 enabled effective double talk in a small conference room and some performed well in a small lecture theatre (100 seats). This issue is examined in greater detail in section 4.7, Audio.

Table 6 Audio Performance
	Audio Quality	Enhanced Audio coding	Echo Cancellation	Audio Level	Software patch available? ⁽¹⁾
Aethra Vega X3	Satisfactory	MPEG-4 AAC-LD	Excellent	Satisfactory
Aethra Vega X5	Satisfactory	MPEG-4 AAC-LD	Excellent	Satisfactory
Codian IP VCR 2210	Some problems	Siren 14, AAC-LC, AAC-LD	N/A	Satisfactory
LifeSize Room	Acceptable	MPEG-4 AAC-LD	Excellent	Satisfactory
LifeSize Team	Very good	MPEG-4 AAC-LD, Siren 14	Excellent	Satisfactory
LifeSize Express	Very good	MPEG-4 AAC-LD, Siren 14	Excellent	Satisfactory
Polycom VSX 6000	Very good	Siren 14	Very good	Satisfactory
Polycom HDX 9004	Very good	Siren	Excellent	Satisfactory	Yes
Polycom RSS 2000	Acceptable		N/A	Satisfactory
Sony PCS-G70P	Acceptable	MPEG-4	Good	Satisfactory
Sony® PCS-HG90	Good	MPEG4 AAC-48K MPEG4 AAC-96K	Good	Satisfactory
Sony Contact 6000	Acceptable		Fair	Low
Tandberg 6000 MXP	Good	MPEG-4 AAC-LD	Excellent	Satisfactory
Tandberg 990 MXP	Good	MPEG-4 AAC-LD	Excellent	Satisfactory
Tandberg Edge 95 MXP	Good	MPEG-4 AAC-LD	Some problems ⁽²⁾	Satisfactory
Tandberg Profile 6000 MXP	Some Problems	MPEG-4 AAC-LD	Good	Satisfactory
ZTE ZXT500	Good	Siren ⁽³⁾	Good	Satisfactory

^{About this table

1. A patch is available from Polycom. Please contact your local supplier.

2. A high frequency whistle was experienced with the equipment on test but subsequent checking by Tandberg UK could not reproduce the problem.

3. Although Siren Audio was specified only G.722 could be achieved during testing.}

3.5 Data Transfer

Table 7 Data transfer
	Comments
Aethra Vega X3	Using the Aethra software, AePPTManager, PowerPoint presentations could be transmitted to the remote site as a series of still images only. T.120 also supported.
Aethra Vega X5	Using the Aethra software, AePPTManager, PowerPoint presentations could be transmitted to the remote site as a series of still images only. T.120 also supported.
Codian IP VCR 2210	N/A
LifeSize Room	Data may be introduced via the VGA interface.
LifeSize Team	Data may be introduced via the VGA interface.
LifeSize Express	Data may be introduced via the VGA interface.
Polycom VSX 6000	Two methods available for introducing PC images into a conference: a hardware option, Visual Concert VSX for direct connections; and a software option, People + Content IP for web access.
Polycom HDX 9004	DVI interface for PC connection.
Polycom RSS 2000	N/A
Sony PCS-G70P	Scan converter option, proprietary whiteboard port (Sony PCS units only). Memory stick port for image transfer.
Sony Contact 6000	Data sharing achieved over both H.320 and H.323 networks.
Sony PCS-HG90	Data may be introduced via the VGA interface.
Tandberg 6000 MXP	Data sharing via Soft Presenter/Virtual Computer Network.
Tandberg 990 MXP	Data sharing via Soft Presenter/Virtual Computer Network or direct via CODEC DVI-I interface.
Tandberg Edge 95 MXP	Data sharing via Soft Presenter/Virtual Computer Network and T120 or direct via CODEC DVI-I interface.
Tandberg Profile 6000 MXP	Data sharing via Soft Presenter/Virtual Computer Network and T120 or direct via CODEC DVI-I interface.
ZTE ZXT500	T.120 supported

3.6 Inter-working
All models tested conferenced satisfactorily with another identical unit. This test specifically checks the performance of the CODEC with units from other manufacturers and also through the JVCS MCUs. The VBrick 3000/6000 could not be tested for inter-working as no other products operating to the MPEG-1/MPEG2 standards were available at the testing sites.

Table 8 H.320 (ISDN) Inter-working Performance
	With Another CODEC	Multipoint Conference through JVCS	MCU Control	Far End Camera Control
Aethra Vega X3	Satisfactory	Satisfactory	No	Yes ⁽²⁾
Aethra Vega X5	Satisfactory	Some problems	No	Yes ⁽²⁾
Codian IP VCR 2210	Some problems	Satisfactory	N/A	N/A
Sony PCS-G70P	Satisfactory	Satisfactory	No	Yes⁽²⁾
Sony Contact 6000	Some problems	Satisfactory ⁽¹⁾	Yes	Yes
Tandberg 6000 MXP	Some problems	Satisfactory	Yes	Yes
Tandberg 990 MXP	Some problems	Satisfactory	No	Yes
Tandberg Edge 95 MXP	Some problems	Some problems	No	Yes ⁽²⁾

^{About this table

Where "some problems" is indicated in the table please refer to the detailed test report for the relevant CODEC.

1. No problem with connecting, but audio level low (see Audio Test).

2. Far-end camera control and remote source selection.}

Table 9 H.323 (IP) Inter-working Performance**
	With Another CODEC	Multipoint Conference through JVCS	MCU Control	Far End Camera Control
Aethra Vega X3	Some problems	Satisfactory	No	Yes ⁽¹⁾
Aethra Vega X5	Some problems	Satisfactory	No	Yes ⁽¹⁾
LifeSize Room	Some problems	Some problems	No	Yes
LifeSize Team	Satisfactory	Satisfactory	No	Yes
LifeSize Express	Some problems	Satisfactory	No	Yes ⁽¹⁾
Polycom VSX 6000	Some problems	Satisfactory	No	Yes ⁽¹⁾
Polycom HDX 9004	Some problems	Satisfactory	No	Yes ⁽¹⁾
Polycom RSS 2000	Some problems	Satisfactory	N/A	N/A
Sony PCS-G70P	Some problems	Satisfactory	No	Yes⁽¹⁾
Sony PCS-HG90	Satisfactory	JVCS not HD capable	No	Yes⁽¹⁾
Tandberg 6000 MXP	Some problems	Satisfactory	Yes	Yes
Tandberg 990 MXP	Satisfactory	Satisfactory	No	Yes
Tandberg Profile 6000MXP	Minor problems	Satisfactory	No	Yes ⁽¹⁾
Tandberg Edge 95 MXP	Some problems	Some problems	No	Yes ⁽¹⁾
ZTE ZXT500	Some problems	Satisfactory	No	Yes

About this table

** For further details of the specific problems experienced refer to the individual product test reports ^{1. Far-end camera control and remote source selection.}

top

4. IMPORTANT FACTORS/CONSIDERATIONS

The first product evaluation took place in 1997, the latest in November 2007. The technology has improved significantly during this period, so it is unrealistic to compare products directly unless they were tested at a similar date. Most products tested before 2001 have now been improved considerably. As some products have also now been superseded it is strongly recommended that the manufacturers' web sites be consulted to obtain current information.

4.1 Video Coding
H.261 was the original video coding specified by the ITU to ensure compatibility between different manufacturers' products. It still remains the base coding across ISDN and IP networks to ensure communication between products should other coding algorithms prove incompatible. To improve results especially at low bandwidths H.263 coding was introduced and is now established as the main coding method. Latterly, enhancements to H.263, namely H.263+ and H.263++, have brought further improvements in quality. The latest development, H.264 coding, is designed to give good results with a saving in bandwidth, and has now been adopted by most manufacturers.

4.2 Dual Video Streams
Dual video streams are now a feature of several CODECs. This enables simultaneous transmission of both participants and content (e.g. the output from a document camera). There has to be a compromise to fit both signals in the same pass-band. Systems normally allow user definition of the bandwidth allocation between streams to optimise the chosen information. Although dual video working is described in ITU-T standards, it is not currently supported by the JVCS MCUs. As a result, this mode is only possible in point to point links.
Where manufacturers provide an MCU as part of the CODEC package, dual video streams (people and content) may be supported between compatible units, such as the Polycom VSX series CODECs. The ITU-T standard H.239 was created to achieve dual video stream compatibility between manufacturers and is now being implemented by several manufacturers.

4.3 Picture Optimisation
Several manufacturers provide intelligent picture optimisation. When a static picture from, for example, a document camera is being transmitted the CODEC will ramp up the resolution to 4CIF at the expense of movement rendition. When however movement is important, e.g. a video cassette replay, then resolution is reduced to the normal CIF, but coding is optimised to reproduce good movement. Tandberg also offer selectable still frame enhancement for data exchange through its Presentation Package software option Digital Clarity.

4.4 On Board Multipoint Control Units (MCUs)
Several manufacturers now offer on board MCU facilities, and some offer internal MCUs with mixed H.320 and H.323 working, which some sites may choose to investigate. Testing is outside the remit of these evaluations. The VTAS evaluations do not include a full investigation of internal MCUs but only a brief appraisal of their operation. Cascaded MCUs are not part of the H.323 standard.

4.5 Remote Camera Control/Source Selection
Far end (or remote) camera control (i.e. control of the camera at the remote site from the local site) is a common feature, but was initially only available for H.320 systems. Functionality over H.323 systems is now commonplace. Remote source selection i.e. selection of the remote sites camera, Visualiser, PC, etc. from the local site is also a feature on several CODECs. These facilties may not operate through an internal or standalone MCU.

4.6 IP Video Tests
IP videoconferencing has developed into an established form of communication. Both the Welsh Video Network (WVN) and the Scottish universities network, known as SMVCN, rely on this method of connection for their day to day conferencing activities. Where adequate bandwidth is available it is better than ISDN as a means of connection. Local bottlenecks can however still compromise quality when competing data can disrupt the sound and vision signals. The growth in use of IP-based conferencing systems has influenced the products now offered. In the past ISDN H.320 CODECs were the norm, with IP (H.323) functionality available as an option. The situation is now reversed, with most products now offering IP/H323 as the basic configuration, with only a few products offering ISDN functionality as an option.

4.7 Audio
The JVCS Management Centre, located at Edinburgh, offers MCU facilities to registered users for establishing multi-site conferences. To ensure sites can conference effectively they are all required to pass a regular Quality Assurance (QA) test before being registered as a user of the JVCS MCUs. This QA test covers many aspects of performance but a critical parameter is the audio level received at the Management Centre from an individual site. Unfortunately several site CODECs have in the past been unable to meet this requirement and the low audio level has caused some failures. For point-to-point conferencing between identical units this should not cause a problem but, when multi-conferencing through an MCU, large level differences between sites can seriously degrade the audio quality and could affect echo canceller performance. JVCS MCUs can accommodate and equalise these level differences, however this does not guarantee that the site will pass the QA test.

Echo cancellers (ECs) are now an integral part of most good quality videoconferencing systems and are almost entirely automatic in operation, requiring no setup by the installer. The majority function efficiently in a conference room environment. When conferencing from a large lecture theatre is envisaged, the control of echo is much more difficult and a separate EC with a wider window of correction may be required. In this case it is essential that the conferencing system can provide the necessary input and output connections to allow an external EC to be introduced. Not all CODECs permit external ECs. Some CODECs are provided with a separate EC for each microphone and manage difficult environments very efficiently, so it is certainly worth checking their operation in the intended room before purchasing a separate echo canceller. For small groups (two to three people) a single microphone (or microphone unit) will be adequate. For larger groups additional microphones will be needed to ensure good sound pick up. The conferencing system should be able to accommodate additional microphones easily and/or provide for an external audio mixer to be connected to enable this. See the VTAS guide, Videoconferencing Audio and Video Equipment, for further details.

4.8 Advanced Audio Coding (AAC-LC, AAC-LD)

An appreciable delay on the audio is experienced during most H.320 and H.323 conferences. To alleviate this, Advanced Audio Coding has been introduced by several manufacturers. This improves sound quality for low bit rate transmission.

The basic interpretation, AAC-LC, extends the upper audio frequency response to around 16kHz in comparison with traditional audio CODECs which deliver 3.5 or 7kHz audio frequency response. The more advanced interpretation, AAC-LD, also reduces transit delay during the coding process. For example, AAC-LC has a typical delay for a 64kbit/s data stream of around 130ms. AAC-LD reduces this to nearer 20ms and, as such, enables high quality audio for music and speech with a low transit time delay.

4.9 Data Transfer/Application Sharing
Early attempts at data conferencing over ISDN, usually involved interleaving the data within the overall videoconferencing passband. These systems were reliant on the ITU-T T.120 recommendations for transmission. While T.120 did guarantee data exchange independent of platform or system, it did have some limitations when used within low data rate conferencing systems (e.g. 128kbit/s, ISDN 2), as the data would degrade the conference video images. To minimise this effect the T.120 data exchange rate was limited to 19.2 or 38.4kbit/s, but this limitation proved far too slow for large software programmes such as Microsoft PowerPoint and so other methods are now preferred.

A separate channel is now used for the data; either the Internet or the second channel provided by CODECs with dual video, leaving the main conferencing channel intact. Simultaneous audio/video conferencing together with the data are thus transmitted and received and is much more successful for teaching purposes.

Some CODECs provide an SVGA/XGA IP connection to allow the screen output from a PC or laptop to be transmitted to a remote site. This image may be transmitted in native SVGA/XGA resolution or converted to CIF or 4CIF. The transmission of the PC image either replaces the video image from the system camera or occupies a second video channel (Dual Video systems). Some manufacturers such as Tandberg and Polycom enable an internet connected PC to transfer images into a conference without a direct physical connection to the CODEC. Polycom's People + ContentT IP for Windows and Tandberg's Soft Presenter option using Virtual Computing Network (VCN) server software achieve this transfer. These methods enable data exchange to occur but do not enable data sharing between sites.

4.10 Network/Dialling
All H.320 or H.323 compliant products should be capable of communicating effectively. For identical products this is certainly the case but some problems still remain when conferencing between dissimilar equipment. Generally these problems are fairly minor (as detailed in the test reports) and can involve little more than resetting some menu options. This does demonstrate, however, the importance of always having a trial run before a conference with a new site. As no difficulties in dialling overseas with ISDN CODECs have been encountered in testing since 1999 this test is no longer conducted. During an ISDN6 (384kbit/s) connection it is not uncommon to lose one of the lines through congestion or fault conditions on the network. When this happens most systems now carry on conferencing, albeit at a lower data rate. IP systems generally reconnect quickly if the connection is broken momentarily.

4.11 Ease of Use
Most of the products now being tested are very easy to operate. Manufacturers have devoted considerable effort to improving this aspect of performance. Unfortunately some products still demand considerable effort to install, requiring assistance from technical staff. The test reports highlight difficulties encountered.

A web-based interface for diagnostics and set-up is now provided by several products. This can be a real asset as software updates can easily be downloaded and suppliers can interrogate the system to fault find. With this facility, units in remote parts of the campus can also be monitored from a central area manned by technical staff.

4.12 Auto Tracking Cameras
Early attempts at auto tracking cameras were not effective, but the technology has improved so that the better systems are now a real asset during a conference. They enable accurate and effective camera framing of the participants speaking, by locating the source of their voice. This avoids manual camera adjustment and is a definite bonus for new conference users.

4.13 Value for Money
The packaging of products can sometimes be misleading. For this reason the less expensive modest looking set top CODEC should not be rejected on appearance alone. Some of these 'portable' units incorporate all of the features of much more costly room based systems, and in many cases will meet most needs admirably.

For personal use, videophones such as that marketed by Motion Media may be an option as they are much less cumbersome than a PC based system and can allow a PC to be used independently perhaps for data exchange during a conference. Such products are outside the scope of this document, but may be worthy of independent investigation.

4.14 Multi-vendor Solutions
It is unusual for a single manufacturer to have a broad enough capacity to provide all of the components of a videoconference system. As a consequence it is common practice for CODEC manufacturers to incorporate third party components e.g. cameras, microphones, echo cancellers and picture monitors within their products to achieve a state of the art package. The Tandberg Director has taken this a stage further by combining its 6000 CODEC with SMART Technologies Incorporated Smartboard rear projection large touch screen display system. This integrated unit enables the technology of both systems to be exploited in a user-friendly package.

4.15 Encryption
To provide a measure of security to a conference the transmitted data may be encrypted. Two encryption algorithms are currently in use:

Data Encryption Standard (DES) with a 56 bit session key.
Advanced Encryption Standard (AES) with a 128 bit session key.

4.16 Dynamic Host Configuration Protocol (DHCP)
For devices such as PCs and videoconferencing CODECs to communicate over an IP network they normally require a dedicated address - these addresses need to be known in advance by each endpoint for communication to be established.

As an Internet Service Provider (ISP) may have thousands of individual customers, rather than allocating a dedicated address to each customer a dynamic system of addressing is frequently used. Each time they log on, a customer is assigned one of a batch of addresses held by the ISP, so they may be using a different address every time.

To enable the network to track and route dynamic addresses the appropriate software has to be in place at each terminal and throughout the network.

DHCP allows dynamic addressing by automatically sending a new IP address when a computer is plugged into a different place in the network.

Videoconferencing products are now being marketed for use over DSL home connections via an ISP, and it is essential that these CODECs support DHCP.

4.17 Digital Video Interface (DVI)
The transfer of picture information from a PC to its visual display unit (VDU) or picture monitor has in the past always been achieved through the VGA interface. The VGA interface used analogue signals, so the digital picture information from the PC had to be converted into its analogue equivalent to enable the cathode ray tube (CRT) based VDU to display the images. This is because the CRT is an analogue device that requires analogue Red Green and Blue component signals to operate. With the advent of flat panel LCD and similar display devices that are basically digital in operation the digital to analogue conversion to VGA and the subsequent analogue to digital conversion for the LCD device introduced unnecessary signal processing and degradation. The DVI interface avoids this by transferring picture information directly in the digital domain.

4.18 The Sony Philips Digital Interface (SPDIF)
To overcome the signal processing necessary to digitise analogue audio signals in the transmission chain Sony and Philips have introduced the SPDIF interface. This digital interface based on an RCA plug/socket enables the audio paths of video equipment to be connected together in the digital domain.

4.19 High Definition Capability
High definition videoconferencing products using wide screen formats, e.g. 1280 x 720 pixel w720p, are now being marketed and will become more prominent with both the mainstream players and some smaller specialists offering products.

4.20 Session Initiation Protocol (SIP)
In basic IP transmission a digital signal may be coded into several data streams that can each take different routes before arriving at their destination. This means parts of the data can arrive at different times. For most IP traffic this does not cause a problem as the original signal may still be decoded accurately. For digitised audio and video over IP, all the data has to arrive at a similar time i.e. it has to take the same route, otherwise the original signals cannot be reproduced accurately. SIP is an international standard that aims to achieve this.

4.21 Widescreen Formats
High Definition (HD) television systems display images in a 16:9 widescreen format as opposed to the 4:3 (width:height) aspect ratio images of normal definition television. One digital widescreen format that uses progressive (p) scan techniques to construct the image and has a vertical resolution of 720 television lines is known as w720p. This format has a resolution of 1280x720 pixels. Another format, known as w448p, resolves 768x448 pixels. The w228p widescreen format only has a resolution of 512x288 which is less than VGA (640x480) so, although wide screen, it does not qualify as high definition.

Interlaced CIF (iCIF)
The Common Interface Format (CIF) is a part of the H.261 standard that defines the vision signal. It enables videoconferencing communication between American NTSC television systems and European PAL systems without standards conversion. CIF, sometimes termed Full CIF to differentiate it from Quarter CIF (QCIF) achieves compatibility by combining elements of both TV systems. It requires 352 (horizontal) by 288 (vertical pixels) at a repetition rate of 30 TV frames/second. The refresh or repetition rate can be doubled to 60 frames/second by interleaving a second frame in between the horizontal pixels of the first frame. This is termed interlacing. Interlaced CIF or iCIF utilises 352 horizontal by 576 vertical pixels at 60 frames/second and greatly improves the quality of moving images.

top

5. TRADEMARKS AND ACKNOWLEDGEMENTS

5.1 Trademarks

Aethra Vega Star Gold, Aethra Vega X5, Aethra Vega X3 and AePPTManager are trademarks of Aethra.

Gentner and V-There 2200 are trademarks or registered trademarks of ClearOne Communications Inc.

LifeSize Room and LifeSize Team are registered trademarks of LifeSize Communications.

Microsoft NetMeeting and Microsoft PowerPoint are registered trademarks of the Microsoft Corporation.

PictureTel, Polycom iPower, Vortex, Polyspan, Viewstation and V500 are trademarks or registered trademarks of Polycom Inc.

Sony Contact are registered trademarks of the Sony Corporation, Japan.

VBrick is a registered trademark of VBrick Systems Inc.

VCON, MediaConnect and ViGO are registered trademarks of VCON, Ltd.

Zydacron and OnWAN are registered trademarks of SCOTTY Tele-Transport Corporation Inc.

5.2 Further Acknowledgements
The following manufacturers/suppliers generously made their products available for the evaluation and were most helpful during the testing.

Aethra Review Video Gibson Videoconferencing Ltd	- Aethra Vega Star Gold
AuDeo Systems Ltd.	- Polyspan ViewStation
Call2View	- Tandberg 6000
Codian Ltd	- Codian IP VCR 2210
First Connections	- Polyspan FX
Gentner Communications	- V-There 2200 CODEC
LifeSize Communications NuVideo	- LifeSize Room - LifeSize Team - LifeSize Express
McMillan UK Ltd	- Tandberg 800
MVC UK	- Tandberg 500
Pinacl	- PictureTel 680 4P - PictureTel 970
Polycom UK	- Polycom iPower 9800 - Polycom Vortex EF2241 - Polycom VSX 7000 - Polycom V500 - Polycom VSX 8000 - Polycom VSX 6000 - Polycom HDX 9004 - Polycom RSS 2000
Questmark	- Aethra Vega X5 - Aethra Vega X3
Review Video	- ZTE ZXT500
Satelcom UK Ltd	- VCON Quickconnect
Sony	- Sony Contact 6000 - Sony PCS-1P - Sony PCS-G70P - Sony PCS-HG90
Tandberg UK Ltd	- Vision 800 and 5000 - Director System - Tandber Content Server - Tandberg 6000 with H.264 - Tandberg 990 - Tandberg 3000 - Tandberg 6000 MXP - Tandberg 990 MXP - Tandberg Edge 990 MXP - Tandberg Profile 6000 MXP
The UK Office	- VBrick 6000
VBrick Systems	- VBrick 3000
VCON UK	- VCON ViGO - VCON MediaConnect 9000
VideoCentric Ltd	- VCON HD5000
VTEL Europe Ltd	- VTEL TC1000/2000
First Connections	- Zydacron OnWAN Z340

The manufacturers have been invited to comment.
Responses can be found listed alongside individual reports.

top