Video Basics

There are many video formats to consider. Some understanding of them may help one with decisions to be made in user space: how are digital video signals characterized; what type of new TV to buy; is an AVR necessary? what to look for in a new AVR; where in the signal path should certain video signal conversions take place; what kind of component interconnect cables should be used in the video signal chain;  how does where one sits and what one watches determine the type of TV to buy; how much equipment future-proofing is reasonable/cost effective, considering how broadcast video standards lag far behind digital video display technology; how to feed the youtube machine (what codecs and containers should one use, and what parameters should be specified to provide best quality)?

Video signals represent a sequence of discrete frames, each consisting of a set of horizontal lines of pixels that comprise one complete displayed image. Thus frame means image. The product of the number of lines in a frame with the pixel length of each line determines the image’s static pixel resolution (image pixel size). The ratio of number of pixels per horizontal line divided by the number of horizontal lines in a frame determines the image’s form factor, either 4:3 (640/480 as in ‘normal TV’ of the last 6 decades); or 16:9 (1280/720 for typical high definition (HD) or wide screen TV; or 21:9 (for cinema ultra-wide screen TV).

Each video signal has a refresh rate, the number of frames per second (fps) shown on the display. A signal’s temporal resolution, the number of pixels the display presents to the viewer each second, is equal to the static resolution times the refresh rate. Let’s consider a standard HD video signal with 720 horizontal lines refreshed 60 times a second, designated 720/60. In a 16:9 HD form factor, each line will have 1280 pixels. The HD temporal resolution is therefore 720 x 1280 x 60, or ~55 Mpixels/sec.

A scan format such as 720/60 above, refreshing all the lines in a frame at each refresh instant, is called a progressive scan, denoted by a ‘p’ suffix (e.g. 720/60p). There is another older scan format, i for interlaced, where each image frame is divided into two fields, the odd/even horizontal lines, which are displayed in alternation. Interlaced scan formats typically are refreshed at 60 fields (1/2 frames) per second, equivalent to 30 fps. Thus, a signal called 480/60i offers the same temporal resolution as 480/30p. One will normally drop the fps designator, understanding that in most cases, interlaced format scans at 30 fps and progressive scans at 60 fps. The temporal resolution for 1080i is ~62 Mpixels/sec (1920 x 1080 x 30), only 12% better than 720p.

Not too long ago, there was only one broadcast video signal type, NTSC (480i). This is called standard definition (SD), and was for decades the broadcast TV standard in the US, as well as the signal natively reproduced by CRT TV sets for US use. It is still the signal type natively recorded on DVD-Video.

After the introduction of wide-screen TVs with progressive scan technologies, there were created other non-broadcast signal types. The first step beyond broadcast NTSC is 480p, called extended definition (ED) and supported by the earliest flat panel displays. Then came high definition HD, which is either 720p or 1080i. Finally there is true or full HD, 1080/30p. No one broadcasts in this format yet, but Blu-Ray disc players and game consoles output a version of this format. At these high rates, compression is required to tame the signal to fit the prescribed broadcast bandwidth, so the extra resolution may be compromised by lossy compression.

Now we arrive at the date this article was first written. A new digital broadcast standard, ATSC, is replacing NTSC in the USA in 2009. It rolls up all the prior SD, ED, and HD standards into one broadcast standard. ATSC HD broadcasts are usually either 720p or 1080i. Various broadcasters choose one or the other format to best represent their typical programming content. ESPN and ABC, who broadcast a lot of fast moving sports content in HD, broadcast in 720p, because de-interlacing 1080i will cause blur and double image artifacts in action frames. Fox also chooses 720p, as does Disney, because they do not broadcast much film-based content. Movies and a lot of television drama are actually shot at 24 fps, the standard for film. Stations providing primarily film-type HD content choose to broadcast at 1080i because they can use the telecine process (see below) for converting film scan rates to video scan format (30 fps). About 75% of broadcasters, including PBS, NBC and CBS, choose 1080i.

ATSC standard includes 1080/24p, 720/24p, and 1080/30p scan formats. So why do broadcasters only use 720p and 1080i? The highest rate in the ATSC standard, 1080/30p, is not processable by the majority of TV sets in the field, so would need to be de-scaled in the reception equipment. Also, bandwidth considerations would again force some type of compression on a true HD signal. Both these situations serve to defeat the benefit of true HD broadcast.

Aside: Europe, which has been slower to adopt standards and hence gets to chose more modern broadcast standards, has adopted the newer MPEG-4 standard instead of ATSC’s original MPEG-2. The improved compression available, combined with the lower PAL scan rate of 50 fps, may make true HD broadcast closer to a reality there. ATSC subsequently extended its encoding standards to include MPEG-4 Part 10 (aka AVC aka H.264), to utilize the best video compression algorithms available.

ATSC’s /24p scan rates apply only to film-like sources, so are not generally suitable for all broadcast content. Only if the broadcasters could perform hot switching between broadcast scan formats based on content would these film formats be directly broadcast. But any hot switching would cause video glitches that the consumer would find irritating. So broadcasters use a technique called 2:3 pulldown (telecine) to convert /24p signals to 1080i. Receiving components can either simply play the video-converted film material at 1080i, or reverse the pulldown in the broadcast signal, restoring to /24p for a better viewing experience.

HD flat panel TV displays have a fixed internal (native) resolution; our 50″ plasma display has 1366×768 resolution (16:9). Such displays usually accept all broadcast signal types that are in use when they are first designed. As a consequence, the display itself will convert each type input signal to its internal progressive format and native resolution. If the input signal is progressive, then a one step conversion, called a scaler, is required to resize the signal’s image resolution to the display resolution. If an interlaced signal is input, then a 2-step conversion is needed, a de-interlacer followed by a scaler. Although the HDTV must de-interlace the 1080i signals for display, a lossy process, this interlaced scan format has much more static resolution than 720p. Also, 720p input has slightly fewer horizontal lines per frame than the typical maximum native vertical resolution of HD displays, so the extra lines must be synthesized. A 1080i de-interlaced input has more information than necessary, so some can be merged to scale down to the internal display frame size.

None of the above discussion can support any more than a ‘hand-waving’ argument that one HD format is superior to another. When the final result is judged on the screen, actual resolution from a 720p or 1080i signal will typically fall between 550-750 perceived lines of vertical resolution, depending on the algorithms used for the scaler and de-interlacer, further influenced by the hardware quality (ability to control digital jitter, etc). Also, the higher resolutions often come at the expense of more potential visible artifacts. Most people, me included, think the actual result from a quality HDTV is more than adequate. As always, brilliance on the test bench won’t generate profits if it does not translate to a noticeable difference in perceived quality to the average consumer.

Since video conversion (scaling and de-interlacing) is unavoidable and is never lossless, the video signal path should only convert once, if possible. The HDTV usually must do scaling conversion, because it is the exceptional display whose internal resolution matches exactly one of the standard video input resolutions. So scaling is arguably best left to the TV, and ideally, the source should be set to output its native resolution, avoiding scaling at the source (but see the exception regarding audio bandwidth in the HDMI discussion in a subsequent blog page). In any case, the AVR may be best set to video pass-through, to avoid intermediate conversions.

Early HD displays came in two resolutions: HD, vertical resolution ~720 lines); ‘full’ HD, vertical resolution ~1080 lines. In the abstract, the higher the native resolution the better. In practice, the actual benefit of high resolution varies, increasing with larger screen size, closer viewing distance, and availability of 1080p content. For a 50″ display, a viewer seated more than 8 feet away would find it impossible to tell the difference in picture quality between a 720 line and 1080 line display.

Combine this factoid with the observation that there will likely be no 1080p broadcast video in the lifetime of displays bought today, and further that all DVD-video content and all non-HD broadcast content is scaled SD quality, it becomes easy to save $$ and pick the lower resolution. The only exceptions might be those planning to watch a lot of Blu-Ray video content, or hard-core gamers who sit 5 feet away from 65″ displays.

Since our viewing distance is 14′ from our 50″ display, and we play no Blu-Ray content yet, our lesser HD is all we need. [Aside: Of course we could use a 55-60″ display already, because a rule of thumb is HD screen size should be greater than viewing distance / 3.]

The two main HD display technologies have been LCD and plasma. Most today choose LCD technology, because it looks better in the average large showroom, and it is more energy efficient. I find plasma more natural looking, particularly with a non-reflective screen surface. Plasma is quicker in changing pixel values, so can capture motion with less artifacts. Plasma can be viewed from a wider off-axis angle without loss of image contrast. Plasma offers better contrast ratios (deeper blacks), although new LCDs with dynamic LED backlighting will lessen this distinction. But these are all minor nits; it’s a purely personal preference decision. All technologies provide quality images.

Time to bring this discussion to a personal level. The user herself can be a video producer, whipping out her iPhone 1S to get a video of Spot pulling clothes off the clothesline. This is a brave new world because now, it is not the equipment manufacturers who are making decisions for you, but it is you, Betty Jane, who must decide how to make your video look its best when it plays at a YouTube theater near you.

It’s good news though. YouTube always re-encodes its uploads and ensures a good result from a wide variety of commercial video formats. Their advice is to create your video with the maximum quality settings available on your camera, 1080p if available. (They are trying to pry us from our accustomed 720p world that has been as good as needed for over a decade.) Further, one uploads at the same frame rate as the video source i.e. p24, p30, or P60.

YouTube further classifies uploads as either ‘standard’ or ‘high quality’. Assuming we are all now high in quality, YouTube would like 1080P video bit rates of 50mbps, accompanied by 384kbps stereo audio with 48khz sampling. Since my video devices output H.264 encoded video within MOV containers, I upload the same, although YouTube seems to favor basic MP4 containers. They also recommend de-interlacing any interlaced clips in the upload.

There are detailed recommendations from YouTube experts regarding H.264 encoding parameters, involving key frame frequency, use of B-frames, etc. These encoding details are beyond this discussion; such esoterica are the domain of the professionals.

(2015 Update) Not much has changed to the above discussion that would affect the average HDTV consumer. But the world is slowly moving on, to mainstream (affordable) higher resolution and higher contrast technology.

The new standard resolution will be 4K or Ultra HD (UHD), 3840×2160 pixels with the same 16:9 form factor as the previous generation HD resolutions. 4K screens of 55-85″ are becoming the affordable standard for TVs, and display panels in general. While one will find little justification for the added resolution by itself, the other improvements will surely justify replacing old HD technologies with newer, more vibrant and accurate color displays. In this replacement cycle, the bump to 4k resolution is ubiquitous and hence comes, relatively speaking, for free (except for a huge bump in bandwidth requirement, to accommodate the raw 4K UltraHD temporal resolution of ~495 Mpixels/sec, more than 8 times that of the original HD specification).

By 2017, maturing OLED (Organic LED) display technology and High Dynamic Range (HDR) implementations will become generally affordable, enabled by HDMI 2.0a as the standard AV hardware interconnect. The organic designation refers to OLED’s hydrocarbon emissive layer, substituting for the LCD’s semiconductor (heavy metal) layer, for generating the light pixels. Several advantages are inherent in OLED over LCD:

  • very high refresh rate
  • wide color gamut
  • high contrast, due to the blackest blacks
  • wide viewing angle
  • thin
  • light weight
  • environmentally safe
  • low power usage
  • more durable
  • simpler, thus, in the long run, potentially less costly than LCD to manufacture

HDR technology is applicable to both LCD and OLED technologies, but will likely always produce superior results with OLED screens. HDR provides both higher contrast, and greater color depth. It requires participation by content providers and video transmission bandwidth providers to achieve greatest benefit. HDR will initially come in two flavors, proprietary Dolby Vision, and the open standard HDR10 base layer. It will also come with two spec levels, one for LCD technology with its higher luminance potential, another for OLED technology with its lower black level potential. Neither technology can compete on the other’s high ground, thus the double standard. Viewers whose taste tends to brighty-bright blingy-bling will certainly prefer LCD HDR; those viewers preferring the greatest accuracy and detail, and the moods invoked by the deepest, velvety  levels of black, will surely opt for OLED.

When done right, as in the LG 2016 flagship models, OLED screen technology already provides the finest display experience ever. Yet only a couple of vendors still chase this technology, because early manufacturing yields were poor on the larger panels, pushing early costs very high, and because early-on, there was lower life expectancy of the blue-producing emissive material, raising questions of overall technology viability. The Panasonic/Sony OLED consortium died in 2013 and Samsung bailed on OLED in 2015.

Only LG continues to bet the farm on big OLED, so it will be the producer of all large OLED displays in the near future. Other vendors would have to have very understanding investors to jump into such a scenario, with only one viable supplier who is its own biggest customer. Small OLED screens for desktop monitors and mobile devices will likely be the more viable market for other dabblers in OLED technology, until manufacturing and technology breakthroughs arrive for large panel production.

In the near term, OLED is facing some competition as the new Quantum Dot (nano-crystal) technology binds vendors to LCD technology for a while longer. Quantum dots are implemented as a drop-in substitute panel within the standard LCD display manufacturing process, so there is no expense for a new production line.

When illuminated by the blue LED backlight, the nano-crystals emit colored light, the color dependent on the crystal size. The advantages of quantum dot technology are greater brightness, better color saturation and accuracy, and less energy usage. The downside is color bleeding, and there is a scramble to fix this problem.

OLED will still be king, because of its incredible thinness, native contrast, color punch, and off-axis viewing quality. While absolute brightness, together with early motion blur, color accuracy, and longevity  issues, have been  weak spots, OLED, when implemented well, largely overcomes these difficulties, offering the best picture quality available. Nano-crystals will not likely close the gap, and LCD technology has its own weaknesses, particularly in its typical ‘zoned’ HDR implementations.

Curved panels are also a common choice by 2016. Curved and flat OLED screens will be similarly priced. For panels of 65″ diagonal or larger, a curved display can enhance the viewing experience for an entire family. For smaller displays, there will likely be viewing compromises that make a curved screen only satisfying for a couple seated at a sweet spot. Immersion within a scene, with high edge-to-edge contrast and color punch are the strengths of a curved screen. But if bright objects are directly opposite the screen, reflections may be a larger than normal distraction. And from an aesthetic standpoint, a wall-mounted curved display takes some getting used to.

Those of us hanging onto our Panasonic plasma screens now have somewhere to turn if those plasma devices ever decide to stop working. OLED is noticeably superior to plasma in image quality, even at 1080p HD without HDR. And if 4K OLED and HDR get perfected one more notch, together with a price decline of ~50%, 2017 could be LG’s great year, finding many of us relegating our still-working plasma screens to secondary viewing areas, to make room for the new king.

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s