H.263 and H.264 - The Advanced Videoconference Codec

H.263 General Information

H.263 is a standard video-conferencing codec. As such, it is optimized for low data rates and relatively low motion.

H.263 is an advancement of the ISO H.261 standard, mainly It was used as a starting point for the development of MPEG (which is optimized for higher data-rates.)

Architectures Supported

• Apple QuickTime


• Microsoft NetShow


• Microsoft VFW Video for Windows


• Microsoft DirectShow - Part of DirectX

Pros, Cons & Tips

Pros

• H.263 is optimized for low data rates


• Generally better quality than H.261

Cons

• H.263 is CPU intensive, so data rates higher than 50 KBps may bog down most machines


• It may not play well on lower-end machines

Tips

• H.263 has a strong temporal compression component, and works best on movies in which there is little change between frames

IBM Toolkit for MPEG-4

The IBM Toolkit for MPEG-4 consists of a set of JavaTM classes and APIs with five sample applications: three cross-platform playback applications and two tools for generating MPEG-4 content for use with MPEG-4-compliant devices. These applications are as follows:

• AVgen: a simple, easy-to-use GUI tool for creating audio/video-only content for ISMA- or 3GPP-compliant devices


• XMTBatch: a tool for creating rich MPEG-4 content beyond simple audio and video


• M4Play: an MPEG-4 client playback application


• M4Applet for ISMA: a Java player applet for ISMA-compliant content


• M4Applet for HTTP: a Java applet for MPEG-4 content played back over HTTP

Video Codec Reference Source Code

I have had so many requests for reference code , Well here is a link, you on you own after that.These are NOT OPEN SOURCE or FREE. There are all sorts of Liscences and Patents, so I recomend you do you homework before making a product from this. All of these links were found through Google once you know what your looking for.

They are on other peoples server, I am not in any way responsibe for this code being available publicly nor for it's integrity or even explaining how it works.That being said, it's a great source to learn how these codec's work, the code all in C (Not C++) and run well on Unix/Linux. The code is NOT Optimized in any way, shape or form, so it's slow! If any of these links are broken, shoot me an E-mail, I recomend google searching the file names, your bound to find someone with this code lying around.

H.263 Source Code

• h263.tar.gz


• tmn-2.0.tar.gz


• tmndec-2.0.tar.gz

Draft, ITU H.263 Standards Document

ITU H.263, H.263L, H.26L, H.263E, ISO/IEC 14496 These video codecs are the Basis for MPEG4 Simple Profile. MPEG-4 adds advanced error detection and correction services on top of H.263.

3GPP and ISMA are versions of H.263 and MPEG-4 for streaming and mobile applications. These are really a variation of Transport stream.

There is now MPEG-4 AVC/H.264

Also called JVT ITU H.264 ITU-T Rec. H.264 MPEG-4 Part 10 ISO 14496-10 AVC Advanced Video Coding AVC ...whatever you call it, this is revolutionary. H.264 is being widely recognized as the future platform of video compression for applications such as new HDTV services, portable game console, mobile broadcast video services, video on solid-state camcorders, instant video messaging on cell phone. H.264 is the most advanced video coding standard available today. It uses many new coding techniques not available in MPEG2, MPEG4 and H.263. This chart shows the evolution of video coding standards.

This revolutionary technology promises to facilitate whole new applications, and to unleash dramatic cost savings in the broad spectrum of video applications. Examples include:

• Transmission of HDTV programs twice as efficiently than with MPEG2


• Storing 2 Hrs of HD Movies on an ordinary red laser DVD


• Facilitating the emergence of HD content PVR for consumer use


• Doubling the stored program time of PVRs by transcoding the recorded content to H264


• Affordable high picture quality solid state/HDD based camcorder


• CIF quality Video programming for mobile devices

Another Excellent paper on H.264

MPEG-4 Video: Opportunities and Challenges Ahead

For you who are already familiar with video compression standards and want to know more about the H.264 standard, the following publication (19 pages) is an excellent source:

A draft of the whole H.264 Standard (264 pages) from the 'ITU-T 7th meeting: Pattaya, Thailand, 7-14 March 2003', is also downloadable below:

The whole H.264 Standard

H.264 Performance

The new H.264 video compression standard is the first to be ratified by both the ISO/IEC and ITU-T. It raises the bar considerably compared to MPEG4-ASP and yields better picture quality while significantly lowering the bit rate. This allows lower overall system cost, reduced infrastructure requirements and enables many new video applications. H.264 is considered the greatest achievement in video compression in the past 10 years and is already finding its way into many mainstream video applications.

Below follow some practical comparisons between different coding technologies for 90 min of DVD quality video:

H.264 Patent Licensing Fees

One more great aspect of H.264 is the licensing fee structure by the patent holders of technology that went into the standard. MPEG4 was significantly hampered by complicated and excessive licensing fees demanded by its patent holder. For H.264 the industry has learned from its mistakes with MPEG-4 and created a very simple and low cost licensing structure. There are two H.264 patent licensing organizations: MPEG-LA and Via Licensing. Both have similar licensing terms. Here are some key points from the MPEG-LA patent licensing organization:

• Until Jan 2005 there are no licensing fees at all


• After that, there are still no licensing fees for the first 100,000 units shipped


• Above 100,000 units, licensing fees vary between $0.10 and $0.20 per unit

H.264 History

Originaly proposed by Toshiba

Main Profile, Level 4.I:

• I + P picture types, In-loop deblocking,1/4 sample motion compensation, VLC-based entropy coding


• Tree-structured motion segmentation down to 4x4 block size, 4:2:0


• Bi-predictive slices, CABAC, Weighted prediction, Adaptive block-size transforms


• Interlace pictures, frame/field adaptive at picture and macroblock level


• Max bitrate: 29.4Mbps


• Prediction (GOVU) structure

ITU-T SG15 H.263

ITU-T SG15 H.263 came out in February 1995. Designed for low bit-rate communications, it never really worked well over pots (plain old telephone service) lines. It has replaced H.261 for video conferencing in most applications and also dominates Internet video streaming today.

The coding algorithm of H.263 is similar to that used by H.261, however with some improvements and changes to improve performance and error recovery. The differences between the H.261 and H.263 coding algorithms are listed below. Half pixel precision is used for motion compensation whereas H.261 used full pixel precision and a loop filter. Some parts of the hierarchical structure of the data-stream are now optional, so the codec can be configured for a lower data-rate or better error recovery. There are now four optional negotiable options included to improve performance: Unrestricted Motion Vectors, Syntax-based arithmetic coding, Advance prediction, and forward and backward frame prediction similar to MPEG called P-B frames.

H.263 supports five resolutions. In addition to QCIF and CIF that were supported by H.261 there is SQCIF, 4CIF, and 16CIF. SQCIF is approximately half the resolution of QCIF. 4CIF and 16CIF are 4 and 16 times the resolution of CIF respectively. With the support of 4CIF and 16CIF the codec has surpassed other higher bit-rate video coding standards such as the MPEG 1 and 2 standards and a variant is now officially adopted as MPEG 4.

MPEG-4

Work on MPEG-4 started at the MPEG meeting in Seoul Korea November 1993 with completion that was scheduled for 1998. MPEG4 adds a layer of error resilience. This is achieved through resynchronizations, byte alignment, data partitioning the reversible variable length code (RVLC), adaptive intra refresh (AIR), and error detection and concealment.

1. Resyncrhonization: Under the MPEG-4 spec, a resynchronization marker can reduce the error propagation caused by the nature of variable length code (VLC) into single frame. In MPEG-4, the resynchronization marker is inserted at the top of a new group of blocks GOB with the header information (multiplexed block number [MBN], quantization parameters) and optional HEC, so that decoding can be done independently. It is a good idea to place the resynchronization marker prior to important objects like people to improve error resilience with minimum increase of overhead.

2. Byte alignment: Bit-stuffing for the byte alignment gives additional error detection capability through its violation check.

3. Data partitioning: A new synchronization code named motion marker separates the motion vector (MV) and discrete cosine transform (DCT) field to prevent from inter-field error propagation, thus allowing effective error concealment to be performed. When errors are detected solely in the DCT field, that multiplexed block (MB) will be reconstructed using correct MV. This results in natural motion better than simple MB replacement of the previous frame.

4. RVLC: The RVLC enables forward and backward decoding without significant impact on coding efficiency. This feature localizes error propagation ideally into single MB.

5. AIR: Different from the conventional cyclic intra refresh, AIR employs motion-weighted intra refresh, which results in better perceptual quality with quick recovery in corrupted objects.

6. Error detection and concealment: Errors can be detected through exception or violation in the decoding process, and then concealment will be applied. The functionality is included for mobile application. The endpoint of H.324 can support for MPEG-4 audio, so that MPEG-4 audio could be used for H.324 mobile phone terminal.

MPEG2 info

ISO/IEC 13818-1 Systems
ISO/IEC 13818-6 DSM-CC
MPEG-1 and MPEG-2 Digital Video Coding Standards
WMV9 (Windows Media Version 9) Extended STREAM_ID: 0xFD as defined in ISO/IEC 13818-1:2000/FPDAM
MPEG-4 AVC encapsulation into MPEG2 PS, Encapsulation as MPEG video stream
STREAM_ID: 0xE1 (DVD MPEG2 Video Stream ID: 0xE0)

MPEG4 info: Coding of audio-visual objects

• ISO/IEC 14496-1 Systems


• ISO/IEC 14496-2 Audio


• ISO/IEC 14496-2/FDAmd 1 Error resilient simple scalable profile


• ISO/IEC 14496-3 Visual


• ISO/IEC 14496-5 Reference Software (Nokia, Samsung)


• ISO/IEC 14496-6 DMIF (Delivery Multimedia Integration Framework)


• ISO/IEC 14496-10 Advanced Video Coding H.264


• MPEG4 Systems Patents


• MPEG 4 doc's link


• Good Circuit Cellar Article, Streaming Media the MPEG4 Approach


• JPEG 2000 ISO/IEC 15444-1 There is a Motion JPEG2000 spec

H.264 received Final Draft status on March 28, 2003 by the Joint Video Team (JVT), a committee of Moving Picture Experts Group (MPEG) and International Standards Organization/International Telecommunications Union (ISO/ITU). Final ratification of H.264 by the ITU happened on May 30th. H.264 is also known as MPEG 4 part 10.

The bottom line is that H.264 enables videoconferences to connect at half the bandwidth and still retain the same quality or it will deliver twice the quality at the same bandwidth normally used.

What is H.264?

A new video encoding and decoding scheme has been added as a standard under the International Telecommunications Union's (ITU) H.320 and H.323 umbrella for low bandwidth videoconferencing over ISDN and IP. This new codec provides near broadcast quality video, in part, by supporting 60 fields per second (30 full frames) and an advanced method of reducing, if not eliminating, the pixelization we often see when using H.261 or H.263 video codecs when there is a lot of motion or scene changes.

The ITU, along with Internet streaming and other standards organizations, formed a "Joint Video Team" (JVT) to come up with a common format codec for both MPEG users and videoconferencing users. The resulting new standard codec is know as JVT/AVC (Advanced Video Coder). It is known by the MPEG industry at MPEG 2 Part 10 and by the ITU and the videoconferencing industry as H.264.

H.263, the codec in use the past several years, supports 30 frames per second video. In television video, each frame has two interlaced fields. Prior to compressing, one field is tossed aside (every other line) by H.263 to produce 30 frames per second video at the narrow bandwidths used in videoconferencing. Thus, each compressed video frame corresponds to only one of two input video fields. The result is a somewhat lower video resolution quality of 352x288 pixels or "video pixelization."

The H.264 protocol improves the video resolution quality in the H.323 protocol suite by encoding and transmitting two interlaced fields for each frame. That is 30 frames per second and 60 fields per second (instead of only 30 fields per second of H.263). This process allows us to present decoded video that is immensely more fluid and lifelike. The result of this enhancement is a substantially higher resolution quality that approaches or matches MPEG-2 quality at a 64% lower bandwidth cost.

H.264 also handles the encoding of the pixel blocks more efficiantly than H.263, practically eliminating the tiling or pixellation seen on videoconferences today when there is a "scene" change or a lot of motion.

Wisconsin VCS H.264 Research and Demonstrations:

Since May 5th, they have been demonstrating H.264 video to many of their Wisconsin videoconference users and potential users. During the demonstrations, they compared H.264 to H.263 in side-by-side projected images. They also compared H.264 with MPEG 2 in side by side images.

At 384K and 768K, there is a remarkable difference in the video quality as perceived by the human eye. They have also compared H.264 at 384K against H.263 at 768K and even then found, though more subtle, a difference in the video quality.

They have also compared side by side, H.264 at 768K with MPEG 2 at 4 Meg. The H.264 video is amazingly similar in quality to the MPEG 2 signal. Some of the fine lines in the video were just a bit more jagged in the H.264, which was not noticed by many of those attending the demos.

These demonstrations were conducted over a lab simulated network configured similar to the WiscNet backbone with two member LANs connected to the edge. The codecs used were Polycom FX's for the H.263 video and two Polycom iPower 9000 codecs for the H.264 video. They also used a Minerva MPEG encoder and a decoder to provide the MPEG 2 video.

Test video is from a DVD movie with a lot of motion. The video is fed into the three codecs at Site B. The three codecs are each connected to the same model codec at Site A over the lab network (FX to FX, 9000 to 9000, Minerva to Minerva). At Site A, they projected the decoded H.263, H.264 and the MPEG 2 side by side.

Will my legacy Polycom unit support H.264?

We consider endpoints with H.264 compatibility essential to our endpoint recommendations. The protocol will alleviate current H.323 resolution quality perceptions, provide more bandwidth for multiple, concurrent videoconferences and permit interconnection with the substantial deployment of non-H.264 compatible H.323 systems.

We understand that Polycom is developing a software upgrade for the FX, EX and 4000 models of their codecs to support H.264 to be available in 2004. Older models, such as the 512 and 128 will not be upgradable. We have no word on the ViaVideo at this time.

Where can I learn more about H.264?

ITU-T Study Group 16 for the latest approval status. A 264 page Draft Documentis available (March 2003). H.264/AVC The Emerging Standard, by Ralf Schäfer, Thomas Wiegand and Heiko Schwarz Heinrich Hertz Institute, Berlin, Germany as it appeared in the January 2003 EBU Technical Review. (This is a technical paper on the new standard. This paper is found on many web sites when searched on Google, so here it is, too.) Concepts and Performance of Next-Generation Video Compression Standardization by Till Halbach, Norwegian University of Science and Technology; and Mathias Wien, Aachen University. Vcodex: H.264 / MPEG-4 Part 10 Tutorials - resources and research The Internet Engineering Task Force (IETF): RTP Payload format for the ITU-T Recommendation H.264 video codec. Advanced Google Search for "H.264" in relation to H.323. Apple QuickTime and MPEG 4 for 3G (Third Generation Cell Phones)