{"id":272,"date":"2012-10-02T12:08:53","date_gmt":"2012-10-02T16:08:53","guid":{"rendered":"http:\/\/www.drfilm.net\/blog\/?p=272"},"modified":"2020-12-06T21:31:03","modified_gmt":"2020-12-07T02:31:03","slug":"digital-is-over-there-its-only-a-matter-of-sampling","status":"publish","type":"post","link":"https:\/\/www.drfilm.net\/blog\/?p=272","title":{"rendered":"Digital is Over There!  It\u2019s Only a Matter of Sampling!"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-273 aligncenter\" style=\"color: #444444; font-family: Georgia, 'Bitstream Charter', serif; line-height: 1.5; border-style: initial; border-color: initial; cursor: default; margin-top: 4px; margin-right: 24px; margin-bottom: 12px; margin-left: 0px; float: left; display: inline; max-width: 100%; height: auto; border-width: 0px;\" title=\"Screen shot 2012-10-01 at 8.40.30 PM\" src=\"http:\/\/www.drfilm.net\/blog\/wp-content\/uploads\/2012\/10\/Screen-shot-2012-10-01-at-8.40.30-PM.png\" alt=\"\" width=\"714\" height=\"327\" srcset=\"https:\/\/www.drfilm.net\/blog\/wp-content\/uploads\/2012\/10\/Screen-shot-2012-10-01-at-8.40.30-PM.png 714w, https:\/\/www.drfilm.net\/blog\/wp-content\/uploads\/2012\/10\/Screen-shot-2012-10-01-at-8.40.30-PM-400x183.png 400w, https:\/\/www.drfilm.net\/blog\/wp-content\/uploads\/2012\/10\/Screen-shot-2012-10-01-at-8.40.30-PM-300x137.png 300w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px\" \/><\/p>\n<p>Bruce Lawton made me aware of <a href=\"http:\/\/www.nytimes.com\/2012\/09\/30\/movies\/lawrence-of-arabia-mended-returns-to-screen-and-blu-ray.html?pagewanted=all&amp;_r=0\" target=\"_blank\" rel=\"noopener noreferrer\">an article in the <em>New York Times<\/em><\/a> that I found highly annoying.\u00a0 It was highly annoying because it was inaccurate.\u00a0 It reflects the complete misunderstanding of what \u201cdigital\u201d means in the media and public.\u00a0 In short, the public and media seem to believe this:<\/p>\n<p>\u201cDigital imaging processes are a modern miracle and are a complete replacement and upgrade from older technologies.\u00a0 All digital images are perfect by their nature and will never degrade or become outdated.\u201d<\/p>\n<p>This is simply not true.\u00a0 I hate to burst your bubble.\u00a0 A closer summation would be this:<\/p>\n<p>\u201cDigital imaging is a miraculous tool that allows us to do things that were previously impossible to accomplish.\u00a0 They can produce very high\u00a0quality, not perfect, reproductions of their source images.\u00a0 Their biggest drawback is that they become outdated quickly and most digital storage devices have short shelf lives.\u201d<\/p>\n<p>Now, once again, I\u2019ll draw criticism from the masses: \u201cYou hate anything digital!\u00a0 You\u2019re a luddite!\u00a0 You\u2019re clinging to an outdated technology like film!\u00a0 Get with the modern program!\u201d<\/p>\n<p>Once again, this is not true.\u00a0 I use digital imaging all the time.\u00a0 I think it\u2019s great.\u00a0 I did digital restorations for the <a title=\"The Lost Weekend With Buster\" href=\"http:\/\/www.drfilm.net\/blog\/?p=157\" target=\"_blank\" rel=\"noopener noreferrer\">Buster Keaton picture <em>Seven Chances<\/em><\/a>.\u00a0 I am doing a <a href=\"http:\/\/www.kickstarter.com\/projects\/1622418422\/king-of-the-kongo-film-and-sound-restoration\/posts\/300303\" target=\"_blank\" rel=\"noopener noreferrer\">digital restoration on <em>King of the Kongo<\/em><\/a>.\u00a0 But I still believe in film.\u00a0 Film doesn\u2019t get computer viruses, hard drive crashes, or incompatible software upgrades.<\/p>\n<p>I have film, actual film stock, manufactured in 1926 that is still projectable in modern projectors and plays fine.\u00a0 I have digital images from 1991, carefully saved and copied,\u00a0 that are incompatible with any modern program.<\/p>\n<p>What would you think of a library that had a book from 1991 that you couldn\u2019t read anymore?\u00a0 Not because it was damaged in some way, but rather because they couldn\u2019t figure out how to open it. You\u2019d say they were crazy.\u00a0 You\u2019d be right.<\/p>\n<p>I\u2019m going to refute the <em>New York Times<\/em> article point by point, but first I have to lay out some ground work.\u00a0 Fear not, technophobes. I\u2019ll try to make it as clear as possible and minimize all the math.\u00a0 It really is pretty simple, but for some reason, people want to believe in the miracle part of it instead of the truth.<\/p>\n<p>In the early 1980s, Disney made the first real computer feature.\u00a0 It took years to complete, but it was called <em>Tron<\/em>, released in 1982.\u00a0 <em>Tron<\/em> was made with a bank of computers each with less computing power than your iPhone.\u00a0 Your old iPhone.\u00a0 Yeah, that slow one.<\/p>\n<p><em>Tron<\/em> is not notable for many dramatic triumphs (after all, it\u2019s basically <em>The Wizard of Oz <\/em>set inside a computer), but for cinema, it was a real breakthrough.\u00a0 Disney experimented with various resolutions.\u00a0 Now, before you get all paranoid about a scary word like <em>resolutions<\/em>, let me explain.\u00a0 It simply means how many pixels (little squares, like the ones you see in the image above) are used in the image.<\/p>\n<p>Higher resolution = more pixels = smaller squares = sharper image.\u00a0 In television, this is also measured in lines, which is the number of horizontal lines in the TV picture.\u00a0 You know how people keep trying to sell you 1080p HDTV?\u00a0 Well, standard definition was 525 lines, and HDTV is 1080.\u00a0 Again, more lines = more pixels = sharper image.\u00a0 See?\u00a0 Simple!<\/p>\n<p>Disney knew that they would have to output their computer graphics to 35mm film in some way.\u00a0 There was no digital projection at the time.\u00a0 They were very concerned about \u201cstair-stepping.\u201d\u00a0 This is an effect also called aliasing.\u00a0 Don\u2019t be scared.\u00a0 Look at the picture above.\u00a0 You notice that it\u2019s made of little squares?\u00a0 Omar Sharif\u2019s collar isn\u2019t a collar, but it\u2019s a jagged set of white lines.\u00a0 You went to plot something that was supposed to be a line and you ended up with a jagged representation instead.\u00a0 It\u2019s <em>aliased <\/em>because the thing you tried to plot isn\u2019t what you got!<\/p>\n<p>Disney\u2019s people discovered that they could see aliasing on most images until they put the resolution at 4000 lines.\u00a0 This has been the \u201cgold standard\u201d of digital imaging for years.\u00a0 Well, almost.\u00a0 <em>Tron <\/em>had a limited color palette because of the software and hardware of the time.\u00a0 This made jagged lines easier to spot.\u00a0 As we were able to represent more colors and shades, we discovered that we could drop the resolution to 2000 lines, and it still looked pretty good&#8230; just a little blurry to some people. Remember, this is for material generated by the computer, not something scanned from an outside source.<\/p>\n<p>In engineering parlance, 4000 lines = 4K, 2000 lines = 2K, and HDTV at 1080 lines makes almost exactly 1K.<\/p>\n<p>I have to introduce one last concept.\u00a0 It\u2019s called the <a href=\"http:\/\/en.wikipedia.org\/wiki\/Nyquist\u2013Shannon_sampling_theorem\" target=\"_blank\" rel=\"noopener noreferrer\">Nyquist Sampling Theorem<\/a>.\u00a0 I know, it\u2019s an engineer thing.\u00a0 Nyquist is a law of digital sampling.\u00a0 It says that if you are scanning an analog signal (like a piece of film), the minimum rate you can use, so that you get no significant loss of data, is twice the number of the highest frequency in the source.<\/p>\n<p>Oh, no.\u00a0 The mathophobes are dying now.\u00a0 Please don\u2019t.\u00a0 That simply means if you\u2019re scanning a 4K image, you need to scan it at 8K or else\u00a0you\u2019re get a picture blurrier than it should be.\u00a0 For a 2K image, you scan at 4K.<\/p>\n<p>Now, we can tackle this article.\u00a0 Take a deep breath.<\/p>\n<p>Error 1:<\/p>\n<p>\u201c(<em>Lawrence of Arabia<\/em> was shot in 65 millimeter \u2014 nearly twice the width of a 35-millimeter frame \u2014 so its negative had to be scanned in 8K, creating 8,192 pixels across each line. But it is still referred to as a 4K scan because it has the same density of pixels, the same resolution across 65 millimeters that 4K has across 35 millimeters.)\u201d<\/p>\n<p>This is a very poor way of explaining the concept.\u00a0 They\u2019re saying that this means they\u2019re scanning more lines because the negative is bigger, not because they\u2019re scanning more lines per inch of film.<\/p>\n<p>And, guess what?\u00a0 What we\u2019re seeing here, by Nyquist, through Disney\u2019s research, shows that they\u2019re undersampling (blurring) the negative.\u00a0 Now, I don\u2019t blame them, and it\u2019s probably \u201cgood\u00a0enough,\u201d and very expensive to do more, but let\u2019s start on the right playing field.<\/p>\n<p>Errors 2-3:<\/p>\n<p>\u201cWhen <em>Lawrence<\/em> was last restored, in 1988, some of these flaws could be disguised by \u2018wetgate printing,\u2019 a process of dousing the print in a special solution. But the new restoration has no prints. The film\u2019s digital data are stored on a hard drive, about the size of an old videocassette, which is inserted into a 4K digital projector. In short, the problems would now have to be fixed.\u201d<\/p>\n<p>Wetgate printing is still used.\u00a0 It\u2019s simple enough.\u00a0 You take the <em>negative <\/em>(not the print), and soak it gently in a fluid (some archives use dry cleaning fluid), that fills in the scratches on the clear film base.\u00a0 That fluid evaporates by the time the film hits the takeup reel.\u00a0 Similar processes can be used in scanning.\u00a0 If it wasn\u2019t done that way in this case, then it means more work for the people retouching the images.<\/p>\n<p>The new restoration has no prints.\u00a0 SO WHAT?\u00a0 That has nothing\u00a0to do with what you\u2019re talking about and is a diversion from the point.\u00a0 Wetgate has to do with the scanning or printing the negative, not projection. Note to the sticklers out there: yes, we can use wetgate transfers on prints, if that&#8217;s all we have, but that is not what is happening here.<\/p>\n<p>Error 4:<\/p>\n<p>\u201cLuckily,\u00a0there have been dramatic advances in digital-restoration technology in just the last few years. New software can erase scratches, clean dirt and modify contrast and colors not just frame by frame but pixel by pixel. In the old days (circa 2006), if you wanted to brighten the desert sand in one scene because it was too dark, you\u2019d have to brighten the sky too. Now you can brighten the sand \u2014 or even a few grains of the sand \u2014 while leaving everything else alone. And in those days there was a limited palette for restoring faded colors. Today\u2019s digital palettes are much vaster.<\/p>\n<p>&#8220;In one sense, this restored <em>Lawrence<\/em> might look\u00a0better than the original. Because of the film stock\u2019s exposure to the desert\u2019s heat, some of its photochemical emulsion dried and cracked, resulting in vertical fissures. \u2018Some were just a few pixels wide,&#8217; Mr. Crisp said, &#8216;but some scenes had hundreds of them, filling as much as one-eighth of the frame.\u2019\u201d<\/p>\n<p>The way this is written implies that there were shooting errors that caused exposure problems with things being too dark or too bright.\u00a0 It further implied that Grover Crisp and his co-workers are going in and haplessly changing things to suit their own artistic eye, not that of director of photography Freddie Young or director David Lean.<\/p>\n<p>I have a lot of respect for Grover Crisp, and I know he\u2019s not doing that.<\/p>\n<p><em>Lawrence of Arabia <\/em>was shot on Eastman color stock that was very unstable (it was especially bad from 1958-63.)\u00a0 The colors fade unevenly, and brightness fades unevenly.\u00a0 What they are actually doing, despite the way the article is written, is to match the colors with the way some of the old Technicolor reference prints look (Technicolor prints don\u2019t fade, but they are 35mm and 2-3 generations down from the negative).\u00a0 This is restoration, not willy-nilly artistry.\u00a0 There are certain colors that will be almost entirely gone (especially blues and greens).<\/p>\n<p>Error 5:<\/p>\n<p>\u201cSony went to so much trouble to create not just this release but also a new archive for the ages. Film degrades; digital files of 0\u2019s and 1\u2019s do not. In the coming years, new software might allow still better restorations. But the technicians making them can work from the 4K scan. They won\u2019t have to go back to the negative.\u201d<\/p>\n<p>This is just crazy on a lot of levels:<\/p>\n<ol>\n<li>Robert Harris made a nice duplicate negative in 65mm, on color-stable stock, for the 1980s restoration. \u00a0 At the time he made it, there were <em>already <\/em>a number of unrecoverable scenes and missing bits.\u00a0 This article makes it seem that Harris\u2019 work is now outdated and rather trivial.\u00a0 Nothing could be further from the truth.\u00a0 Harris and director David Lean worked together to save <em>Lawrence of Arabia<\/em>, and without them, <em>Lawrence <\/em>would be less than it is today.<\/li>\n<li>Ones and zeroes don\u2019t degrade.\u00a0 Hard drives do.\u00a0 These are spinning media that are subject to magnetic fields, ball bearing problems, heat, cold, and probably the most fatal problem, sticktion.\u00a0 A hard drive with sticktion has had the spinning magnetic rotor stick to the read head (much like a sticky record album sticking to the needle).\u00a0 If it sticks too hard, then the drive can\u2019t spin, and the disk is ruined.<\/li>\n<li>Ones and zeroes don\u2019t degrade, but file formats aren\u2019t forever.\u00a0 Neither are disk drive formats.\u00a0 Had <em>Lawrence of Arabia <\/em>been restored digitally in 1989, the results could have been saved on 5.25\u201d floppy disks, and no one could read them today.<\/li>\n<li>Scanners are wonderful and they get better every day.\u00a0 I\u2019d bet that if the film is stored well, it will hang together well enough to survive until better scanners come along so that it can be scanned and improved again.<\/li>\n<\/ol>\n<p>This same thing happens often with other \u201crestorations.\u201d\u00a0 <em>Gone With the Wind <\/em>and <em>The Wizard of Oz <\/em>were shot in 3-strip Technicolor, which produces three extremely stable black-and-white negatives.\u00a0 These are a pain to reproduce, so they got \u201crestored\u201d in the 60s to \u201cmodern\u201d Eastman color stock.<\/p>\n<p>Whoops, the restoration faded in a few years.\u00a0 No trouble.\u00a0 They reprinted it again, with better technology, in the 1970s.\u00a0 They went back to the black-and-white negatives, which were still around.<\/p>\n<p>Whoops, that restoration faded too.\u00a0 No trouble.\u00a0 Another restoration was done in the 1980s.\u00a0 Guess how?\u00a0 From the black-and-white negatives.<\/p>\n<p>Oh, wait, they got a better way to reproduce the film and make the alignment sharper?\u00a0 Back to the negatives.<\/p>\n<p>And they needed to re-scan to make a Blu-ray (well, this time, they did an 8K transfer, which is what the Nyquist sampling theorem says we should do for such a film).\u00a0 Gee, they went back to the negatives.<\/p>\n<p>The moral of the story: save the negatives for as long as you can because they seem to get used a lot for restorations.<\/p>\n<p>Error 6:<\/p>\n<p>\u201cBetween the detective work and lots of video improvement (before the days of digital), it took Mr. Harris 26 months to restore the movie \u2014 10 months longer than it took David Lean to make it.\u201d<\/p>\n<p>The preservation work Harris did on <em>Lawrence of Arabia <\/em>was on film.\u00a0 He didn\u2019t use video improvement.\u00a0 There was no video that would do the work.<\/p>\n<p>Error 7-8:<\/p>\n<p>\u201cIts life in home video has been spotty as well. The first DVD, in 2001, was made from a badly done HD transfer: colors were way off, contrasts too bright or dim. A redo, two years later, was much better, but the dirt and scratches were cleaned up by a ham-fisted process called \u2018digital noise resolution\u2019 \u2014 the easiest and, for some problems, the only technique available at the time, but it softened the focus and dulled detail.\u201d<\/p>\n<p>I am not sure, and it\u2019s not really worth looking up, but I doubt that the DVD was made from an HD (High Definition) transfer in 2001.\u00a0 It\u2019s technically possible, but it\u2019s unlikely.\u00a0 It was probably done from a standard definition transfer, which would also account for the color drift, since the color gamut on standard definition television is pretty limited.<\/p>\n<p>I have no idea what \u201cdigital noise resolution\u201d is.\u00a0 I suspect that what he means is \u201cdigital video noise reduction\u201d (also DVNR), which is an automated process to remove scratches and other imperfections from films.\u00a0 Cartoon aficionados have been bemoaning this for years.\u00a0 DVNR is still used, fairly often in fact, but it can be done gently or in a ham-fisted way that the author describes.<\/p>\n<p>\u201cA forthcoming Blu-ray Disc of the film, out Nov. 13, fixes all those problems, in part because it\u2019s Blu-ray but more because it\u2019s mastered from the same 4K restoration as the theatrical release.\u201d<\/p>\n<p>Is the mere fact that something is Blu-ray some way of saying it\u2019s anointed with a perfection not yet seen?\u00a0 Blu-rays, DVDs, films, and videos can all look great or terrible depending on how they are handled technically.<\/p>\n<p>The overarching thing that the author misses (and that others are not missing) is that this digital restoration is not archival no matter how much we would like it to be.\u00a0 I\u2019m on mailing list after mailing list from archives in a panic about how to store things so that they will last.<\/p>\n<p>I was at the Library of Congress recently seeing the process of the entire run of <em>Laugh-In<\/em> being copied from 2\u201d tape, a format now long obsolete, to something now (we hope) more permanent.<\/p>\n<p>At the same visit, I saw a roll of film made in 1893 by the Edison people.<\/p>\n<p>Which of these is archival?<\/p>\n<p>The Library of Congress still uses, and intends to use, 35mm film for archival storage.\u00a0 They haven\u2019t found anything to beat it yet.\u00a0 They are keeping Kodak and Fuji from shutting down the manufacturing lines.\u00a0 Other archives demand film, too.\u00a0 It just holds up better.<\/p>\n<p>That doesn\u2019t mean digital doesn\u2019t have its place.\u00a0 It\u2019s just that digital isn\u2019t the magic panacea that cured the world\u2019s ills.<\/p>\n<p>It\u2019s a tool, just like anything else.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Bruce Lawton made me aware of an article in the New York Times that I found highly annoying.\u00a0 It was highly annoying because it was inaccurate.\u00a0 It reflects the complete misunderstanding of what \u201cdigital\u201d means in the media and public.\u00a0 In short, the public and media seem to believe this: \u201cDigital imaging processes are a &hellip; <a href=\"https:\/\/www.drfilm.net\/blog\/?p=272\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Digital is Over There!  It\u2019s Only a Matter of Sampling!&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"powered_cache_disable_cache":false,"footnotes":""},"categories":[6,207],"tags":[70,103,102,106,104,105],"class_list":["post-272","post","type-post","status-publish","format-standard","hentry","category-dr-films-pocket-rants","category-film","tag-blu-ray","tag-digital-restoration","tag-lawrence-of-arabia","tag-new-york-times","tag-problems","tag-sampling"],"_links":{"self":[{"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=\/wp\/v2\/posts\/272","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=272"}],"version-history":[{"count":6,"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=\/wp\/v2\/posts\/272\/revisions"}],"predecessor-version":[{"id":275,"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=\/wp\/v2\/posts\/272\/revisions\/275"}],"wp:attachment":[{"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=272"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=272"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.drfilm.net\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=272"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}