Television in the USA is often a murky mix of political interference and cow-towing to the interests of business at the expense of consumers. I am not only talking about what’s shown but how the picture arrives on the screen. Yesterday in his diary on the orange one, Blue Meme pointed out how the internet companies are trying to get legislation through to allow differential priority to higher paying users. One reason for this is to allow high speed delivery of internet protocol television, especially in high definition. While Blue Meme picked up on this ahead of the measure passing, everyone here seems to have missed another back at the end of January that will have a profound effect on almost everyone who watches TV in the USA.
Congress voted to shut off analog television transmission by February 17, 2009. Americans have not quite realised how profound an effect the bill as passed will have, particularly on the poor.
The problems have been further exacerbated by the standard chosen to replace NTSC. This will drive up the cost of converting to digital and mean it’s almost certain that you will not longer be able to use a portable TV. All those people at sports matches who watch replays of the action on their hand held portables for example will now have to subscribe to TV-on-cellphone services as those little LCD TV sets will not work. I’d like to take you through some of the reasons and consequences for the problems. Some of this will refer to the technical standards but I hope I can simplify these by separating the “techie” bits to blockquoted footnotes while retaining a narrative if you skip reading them.
The USA has always tried to be at the forefront of TV developments but TV manufacturers have always had a big influence on what’s offered. In the very early days, they owned the stations and there was the very real prospect that you would have to buy a TV using a completely different system if you wanted to watch a rival’s station, only finally resolved by a decision by the FCC to approve a color system based one recommended by the National Television Standard Committee, an industry based body.
Footnote 1
RCA were not the only player and CBS entered the field with a proposed electro-mechanical color system. Unfortunately this was incompatible with RCA’s mono broadcasts. RCA and another developer proposed two different electronic systems. The CBS system was approved for broadcasting in June 1951
Footnote 2
CBS started broadcasting color but only for an hour each day between 4 and 5 pm. As viewers with monochrome sets could not view, they switch over to watch NBC and did not bother to change back. Ratings disaster! The situation was relieved when Korean War restrictions forbad the manufacture of color sets. During the hiatus, the National Television Standards Committee (NTSC) net again and agreed on an all-electronic system based on the RCA monochrome 525 line system with color information broadcast alongside. That was approved as a standard by the FCC in 1953. As a result the USA became the first country to have a color tv broadcasting service. The system became known as NTSC but as well as having some advantages, it also had drawbacks. It gave somewhat exaggerated colors and suffered drifts in color balance – to the extent it was sometimes unkindly said the acronym stood for “Never The Same Color”. NTSC sets usually had a “tint” control that other systems did not require (although the early Sony sets imported to the UK which had been designed for their home market did have them). A few countries adopted NTSC and France and the USSR used similar systems to send the color information alongside the “luminance” signal as it has the advantage of being able to be transmitted over fairly long distances. Germany meanwhile had come up with another system called PAL (Phase Alternate Line) which solved the problem of color drift and had generally more natural color.
Footnote 3
Moving to Digital
The NTSC system based on 525 “lines” and 30 “frames per second” is simple to construct if you have an electricity system based on the US standard 60Hz supply. PAL’s 625 line/25 fps works is optimal for the 50Hz supply used in Europe. Here is where the US TV industry (and industry in general) has a problem. Very few other countries had adopted the US standard. This was either after thorough technical assessment, as with the BBC, or simply because of trade ties with former colonial powers. All well and good until programming in electronic form starts to be sold between countries that use different standards (no problem when you only need to send film). Also in terms of the size screens the systems were designed for, their definition was perfectly adequate. Things moved on though and larger screens became more and more popular and cheaper. Although nominally 525 and 625 “lines”, the number actually displayed is rather fewer and they become apparent on large size sets. It was obvious that there was a need to move up to a greater definition standard and from the 5:4 format to widescreen 16:9 to more adequately display movies. For many years these had been made in a variety of wide formats. The old standard meant they had to be “panned and scanned” to follow the main action within the frame or broadcast as a “letterbox” with black bands at the top and bottom, reducing even further the number of lines used for the actual picture. Various ways were proposed to produce widescreen and improve picture quality. The real breakthrough was the development of digital technology and the introduction of the DVD format. This uses clever compression techniques (MPEG2) to reduce the amount of storage space needed for a full length film to represent the moving image as a series of digits.
Footnote 4
Transmitting Digital
If MPEG2 requires so little data to be recorded, the logic is to use the same system to broadcast DVD quality TV. That’s OK for widescreen sets up to about 30inches from corner to corner but it is not the “high definition” needed for the very large displays now available for “home cinema” nor does it full use the possibilities of the sets. HDTV uses MPEG2 compression but has far more “pixels” on display. This means that for the highest quality picture around 5 times the picture information is needed compared to DVD quality. Having said that, even a DVD quality picture is considerably better than most consumers are used to, providing the conditions are right. The other great thing is that digitally produced pictures are not longer dependant on fixed standards, just like your computer monitor can change resolution, the screen can be adapted to display any standard up to its maximum resolution.
Now TV transmissions work on part of the radio frequency. It would be theoretically possible to transmit the data from a DVD to a receiver and have the decoding part of a DVD player convert the MPEG2 information for use by the TV. That would be ideal in a perfect world. Even in DVD players the environment is not perfect so the disc is spun much faster than necessary so the same data can be re-read several times (called over-sampling) so that if a piece of dust deflects the laser on one pass, multiple reading can tell the player what is the most accurate data. The broadcast environment is even worse. Reflections from tall buildings, aircraft or even passing cars produces “ghosting” on analog television. In digital systems, the “multipathing” signals arriving at very slightly different times can cancel the data. Other things can interfer with the signal. Appliances like freeezers or microwaves switching on and off can send pulses of radio waves that block reception of a digital TV system. So in sending the MPEG2 information there has to be some form of error correction. Here things get necessarily a bit technical but there are two error correction methods available. One, 8VSB, repeats the data several times so if one set is lost, it can be reconstruced fron the other but depends on a signal being received at precisely fixed intervals to lock onto the right data. The other, COFDM, relies on the fact that different frequencies are reflected in slightly different ways. It treats and FM signal as if it were thousands of different frequencies. Some will be badly affected by the reflections and so on but others will have a much better “signal to noise ratio”. CODFM selects the frequencies that are least affect and uses the data they carry. A more detailed decription is available here which also has a comparison of the two systems and why the NTSC chose 8VSM:
There is no doubt that 8VSB has advantages over COFDM in some respects. 8VSB uses the spectrum more efficiently which basically means that for a given transmitter power you get a greater area of coverage. Transmitter power costs broadcasters considerable amounts of money, both for installation and for operation. More powerful transmissions also inevitably lead to more interference with other services. In laboratory conditions, and in field trials, 8VSB’s advantages and relative simplicity won out, and that is why 8VSB is in use today. However there is a world of difference between laboratory conditions linked to practical field trials, and real world use. In practice, 8VSB has some serious shortcomings.
Firstly 8VSB, despite its adaptive equalizer, is not sufficiently resilient to multipath. If you have an outdoor antenna, expertly aligned, then you will probably enjoy very good pictures from your chosen transmitter, at least as good as NTSC. Indoor antennas fare considerably less well and have to be carefully orientated, and even placed near a window, to work at all. One of the key propositions in introducing DTV to the US was that it should work in every situation where NTSC works, and plainly it doesn’t. An associated problem is that the antenna is likely to require re-orientation when the channel is changed. Of course this can happen with NTSC too, but users are likely to accept a slight degree of degradation and not consider re-orienting the antenna. COFDM however is the multipath slayer and is very tolerant of antenna orientation. In a New York Times article one commentator said that with 8VSB it was difficult to get a signal indoors; with COFDM it was difficult not to get a signal
One point noted later is that the very precise timing of the locking signal is thrown out by the doppler effect. This is the same effect you get when a police car passes you and the tone of the warning signal changes as it passes.
In other words, 8VSB provides less “glitchy” reception providing you have a good signal and you are not moving. To get that in practice you almost certainly will need an outdoor antenna aligned fairly accurately. Set-top “bunny ears” antennas or those simple radio style ones on hand portable sets will simply not work. CODFM on the other hand requires more transmitter power but is very much more tolerant of reflected signals. (On a personal note, the other side of my apartment block has a direct view of a CODFM transmitter but we have incredibly thick walls as our building is over 200 years old. The architecture cuts out the direct signal so until we had a new system installed I was dependant on a “fishbone” set-top antenna. This received its signals reflected off office blocks on the other side of the river. An analog picture was unwatchable but the digital reception was spot on, apart from strangely at very low.)
By now you will have guessed what has happened. Once again as an early implementer of a television system, the NTSC has chosen a digital system that is advantageous to the industry at the expense of ordinary viewers. NTSC’s chosen system for land-based transmission cannot be received unless you at a fixed location and you probably need and outside antenna that has been professionally aligned. In contrast the rival systems most other countries have adopted, DVB. utilises CODFM. At the expense of the occasional “burp” as a system re-aquires the signal, reception is far more robust. Although there are not any portable sets at the moment, you can get a little USB receiver dongle for your laptop that will received DVB-T with a short antenna for about £30 or $50 with the sales tax stripped out.
In case that last abbreviation was a surprise, I should explain that with the numbers of developers working on DVB there are different “flavours” available. DVB-T as a standard for terrestial, land transmitter based broadcasts; DVB-C for cable, DVB-S for satellite and even a version for cell phones. Some of the last three have been adopted by companies providing satellite, cable or cellphone services in the USA.
So if you do not intend to get a new high definition TV why should you be worried about digital TV? Simply because Congress has decided that the transmitters will no longer be licensed at the latest by February 2009. Now there must be a reason for this right? The number of digital sets has reached the point, as in the UK, that so many people have it that the analog service can be ceased. No, the choice is simply because Feb 17 2009 is “after the Super Bowl, one of the year’s major TV viewing events, and before the “March Madness” basketball tournaments.”
What about the poor who are not going to be able to afford to buy a new digital HDTV? Well if you are poor enough, Congress has decided you can get maybe two vouchers you can be used towards the cost of a set-top converter- that’s if you are poor enough. Valued at $40 each, you can buy one for your TV and one for your VCR or for a set in the kid’s room, although you will have to pay the balance of maybe $20 each. That is if the cost comes down as quickly as Congress expects tho with a limited market in the few ex-NTSC analog countries, this may be optimistic. Remember the longer the chip runs, the cheaper they are and those for DVB have already reduced to the point where you can get one for $45 ex sales tax …..except at that price they do not include an RF output , only SCART (see footnote 4) so you can add perhaps $5 or more for an RF circuit. Oh and they will also need to get a decent outside antenna if they do not have one. A problem if they are in an apartment block or in a bad reception area will mean they will be cut off.
Still in a Marie Antoinette moment Congressmen might suggest “let them buy cable”. Well the framers of the new legislation have an answer for that:
Dropped from the final bill was a provision that would have allowed the cable television industry to downconvert all high-definition digital signals to standard definition for viewing by analog cable customers.
This could have a huge impact since 39 million cable subscribers, the vast majority, are still on analog cable systems. Unless some interim action is taken, many of those subscribers may have to upgrade to more expensive digital cable service in order to receive broadcast programming from local stations.
So how did this situation come about? Here’s my take. By the early 1990s it was apparent to US television manufactures that there was going to be a demand for bigger home cinema type setups to play DVDs . It was obvious that these sets would show up the short comings of the 40 year old NTSC standard. High definition would be the way forward and that mandated a changeover to a digital transmission system. The added advantage would be the prospect of being able to sell large numbers of these sets as consumers replaced their old sets with the latest gadgets. But to do that they needed content in high definition to be available. This is where the first major concession to the status quo happened. Every station with a UHF analog transmitter was automatically entitled to have another frequency to use for HDTV. Picture quality was, in the terminology of marketers, to be digital television’s “unique selling point” along with the prosepect of getting a widescreen set you could watch DVDs on.
Now the glitches in their plan. We have the obvious that there was actually not a lot of content to show off your HDTV set until fairly recently. Program makers saw the need for a “critical mass” to make shows in HDTV. The traditional driver of new mass communications is pornography. Arguably that goes back as far as printing but in terms of moving images you have the flurry of “stag” movies for theatre showing, “loops” for viewing in adult stores and sales on amateur formats like 16mm and 8mm film. Finally in home electronics, there was a huge boost to VCRs once pornography became widely available on VHS. The writing was on the wall when DVDs became available. Although they were cheaper to produce than VHS, the extra quality initially meant they had a premium price. This inertia continued with the next move, to widescreen where even today very few US productions and only a few from Europe are in that format. Getting a widescreen set in the US meant getting a digital receiver and there was little content. Little content meant few buyers which meant little production and so the circle continued. Now we have an even worse position with HD where there is no agreed format, few players (in Xbox 360s) capable of reading HD versions of DVDs – in other words, what we saw at the start of the VHS/Betamax fiasco. Ordinary over-the-air content is increasing as the major networks gear up to HDTV production for sale as a premium product.
For Americans there are worrying parallels in the cellphone market. The USA chose an almost unique system while the rest of the world signed up to GSM. The result is that in Europe for example, the penetration in some countries exceed 100% – there are more cellphones than the entire population as people get additional phones for specific purpose – one for work and one for home as an example. The sub-use for SMS messages has been incredibly slow in the USA where the older pager systems still predominate. In contrast the use among the young has been so rapid in especially the UK that a version of English, TXT is recognised.
In Europe, the introduction of digital TV has also taken a different course. Radio frequencies are regarded as a national resource. The bandwidth for cellphones was subject to auctions and produced a lot of income for their governments. The digital changeover is seen as a means of using a scare resource more efficiently. Rather than one HDTV station for each TV channel, the model has been to allow multiple standard definition stations on one “multiplex” which is easily implemented on DVB. In the UK six muliplexes allow the reception of about 30 TV stations free to air, roughly 10 encrypted as pay per view, additional bandwidth is used for alternative services like coverage of alternative sports in the Olympics coverage, all national BBC radio stations and many commercial radio stations are broadcast using the spare capacity and digital versions of teletext – information services using a technique similar to closed coupling are also available. Extra stations are fitted in by time sharing bandwidth so two BBC children’s stations close at 7 pm to free up the capacity for BBC3 and BBC4. All these are in DVD quality. The set-top boxes start around $45 with sales tax excluded and the use of SCART connections rather than RCA plugs ensures that there is little degradation when you “daisychain” a converter box and VCR/DVDR to the TV set. Penetration of digital using terrestrial, satellite and (far less popular than in the USA) cable has reached at least 50% and is likely to be over 75% when analog switch off starts from 2009, progressing by region until completion in 2012. The important thing to note her is that there are carrots as well as sticks. For no subsciption a lot more stations than the 5ish analog stations become available. Widescreen sets using standard definition are cheap and virtually the only option if you want a large screen and almost all programs are made in 16:9 – almost the sole exception being some local news.
This also helps illustrate a difference between the marketing of services in Europe compared to the USA. Taking the UK as an example, HDTV is only now starting. This will be linked to the holding of the World Cup soccer tournament. Rather than trying to sell sets on the basis of the technology, the sales line is “watch the World Cup in HD”, ie it’s content rather than electronics driven.
Here again is an example of US manufacturing insdustry doing its own thing which is perhaps typified by its sticking with imperial measures like inches, feet and pounds rather than going metric as everyone else has done. NTSC analog meant it had an almost exclusive TV system, the cellphones used a different standard from that adopted in the rest of the world and now the NTSC has introduced a system of digital TV transmission virtually exclusive to North America and Mexico. Limiting your manufacturing experience to a system virtually exclusive to your home market surely endangers your export capability. Humvees may be aspirational vehicles for some groups but are hardly the basis of bulk exports. Perhaps if you need to know why US manufacturing is in decline you need look no further than the one eyed beast in the corner of the room.
………………………………………………….
Footnotes
Footnote 1
History of Analog Color TV
The early history of black and white was dominated by RCA, a major shareholder in NBC.
Tests began in 1935 using a 343-line system running at 30 frames per second. In 1937, the number of lines was increased to 441. By the end of that year, with FCC encouragement, seventeen broadcasters and manufacturers were operating experimental electronic TV stations. Of course, the BBC had already been making public television broadcasts in 405 lines since 1936. The United States had to wait until 1939 for the first public demonstration of television at the New York World’s Fair. RCA went to great lengths to promote its system as the standard for US television, but it had some competition. CBS, among other companies, publicly doubted whether the RCA standard was sufficiently advanced to withstand the test of time. Of course behind the scenes they were not happy with RCA’s domination of the industry. The FCC were caught on the fence between encouraging technical development and not allowing RCA to become established as a near monopoly. So what would any self respecting governing body do in such a situation? Set up a committee of course, and that is what the FCC did. They took a number of industry luminaries together with a non-RCA chair and created the National Television Standards Committee or NTSC. The NTSC was inaugurated in 1940 and by March 1941 had completed its deliberations and submitted its report. Much of the RCA technology was incorporated, but at a rather better resolution of 525 lines.
Footnote 2
CBS had developed a 343-line 20 frames per second field sequential colour television system using mechanical components. It worked, up to a point, but was flickery and of inferior definition. The system, which transmitted red, green and blue images in sequence was also prone to coloured banding on moving edges. By the end of World War II, CBS were urging the FCC to adopt their system as the colour standard, alongside RCA’s monochrome standard. By this time the number of lines had been increased to 525 and the frame rate to 24 fps. Unfortunately the system required twice as much bandwidth per channel as a monochrome broadcast. Also, the RCA and CBS systems were totally mutually incompatible. Not until 1949 when the standards were compromised to fit the CBS-standard signal within a 6 MHz channel did the FCC begin to relent. They did not relent quickly however and a struggle ensued between CBS and RCA’s newly proposed dot sequential colour system, and one other competitor with a system which displayed red, green and blue as a sequence of lines, which lasted until 28th May 1951. From 25th June 1951, the CBS system was approved for commercial broadcasting in the United States.
Footnote 3
You can divide a TV up into pixels just like a still picture you get from your digital still camera. A TV picture is roughly 500 by 650 pixels. If you recorded every one of those, the storage space on a DVD would be eaten up in about 10 minutes. MPEG2, the system used in compression, (very crudely) takes one frame and breaks it up into blocks. It then identifies similar blocks and only saves the first set of information, using that data to re-create the otherwise unrecorded areas. Think of a black and white chequerboard. To represent it digitally, all you have to do is record one black and one white pixel. To get one tile in a chequerboard, you just have to say (for example) “use a white pixel ten times horizontally and ten times vertically” the next chequerboard tile can be reconstructed by saying “do the same with a black pixel”. Now you have defined a white tile and a black tile you can build a corner by saying “one white tile, one black tile above one black tile and one white tile” Now you have a corner, to describe the whole chequerboard you just have to say “repeat corner four times vertically and four horizontally” Obviously a full color detailed picture requires a lot more information but the math is very complex in identifying areas of similarity.
That’s a very crude idea of how MPEG digitises one still picure but what about the 30 or 25 needed to get a second of moving image? You could repeat the process with every frame but that is wasteful. MPEG2 uses a system where it only does a full digitising every few frames or when a very obvious change is detected. Think of one of those white chequerboard tiles moving against a still but very detailed background. To describe it fully, all you would have to do is digitise the background and then say where the tile is from frame to frame but you cannot quite do that. What you can do is just identify the elements in the picture that have changed. So as the square moves from left to right, you just have to say “this part of the picture has changed to expose the background” and “this part has changed to all white”.
Although that is a very much simplified description of the process involved, you can see how much information you do not need to record but still reconstruct the pictures and thus save space.
Footnote 4
It is obviously pointless if a high definition or even DVD quality signal has to pass through anything that converts the picture to a “radio frequency” signal so that a lead can go into a TV’s aerial (antenna) socket and be used like an ordinary broadcast via the set’s tuner. For that reason sets have other inputs. In the USA these are familiar as “RCA” plugs and sockets. These are usually yellow for the video signal and white and red for the stereo audio. A better quality picture is seen if you use an “S Video” output/lead/socket which is available on some analog sets. Color TVs use a system of red, green and blue dots. In a conventional tube consist essentially of three tubes in one, the three “guns” at the back of the tube send a black and white image to the front of the tube. As they are aligned to only hit one of the three sets of color dots, your eye/brain interprets this as a full color picture.
RCA plugs send a signal that combines the information needed by all the three guns. S Video has two signals that can be decyphered into the three colors with greater clarity. There is a third possibility that is implemented in the 21 pin European “SCART” system. That is to send one signal for each of the three colors. This completely bypasses most of the set’s circuitry so no degredation is introduced. This was never adopted in the USA as manufacturers failed to agree. HDTV sets use a digital connection to interface called HDMI although this is sometimes called “digital SCART”
It should be noted that the very worse connection to use is the “RF” output to the antenna socket.
