RAW (NEF) vs. JPEG
Written by Wednesday, 17 November 2004 02:00
There seems to be a lot of confusion among many digital camera owners about exactly what the difference is between RAW, JPEG and TIFF files. In this article we will try to do two things. First, deal with the basics and try to give you some (elementary) information about each file format and second, tell you why, when and how we use each one of them. This is to be used as a recommendation and not as a rule. After all, as you are going to find out yourself, even the two of us have different opinions as far as the preferred image file format is concerned. Since some of the information in this article comes from other sources, all of them are listed at the bottom of the page and the reader is encouraged to spend some time there, too. There is a lot more there for those who want to know more.
The digital sensor in the majority of digital cameras is what is known as a BAYER PATTERN sensor. This relates to the arrangement of red, green and blue sensitive areas within the body of the sensor. Each pixel in the sensor responds to either red, green or blue light and there are 2 green sensitive pixels for each red and blue pixel. There are more green pixels because the eye is more sensitive to green, so the green channel is the most important. The sensor measures the intensity of light falling on it. The green pixels measure the green light, the red the red and the blue the blue. The readout form the sensor is of the form color:intensity for each individual pixel, where color can be red, green or blue and intensity runs from 0 to 4095 (for a 12-bit sensor).
A conventional digital image has pixels which can be red, green, blue of any one of millions of other colors, so to generate such an image from the data output by the sensor, a significant amount of signal processing is required. This processing is called Bayer interpolation because it must interpolate (i.e. calculate) what the color of each pixel should be. The color and intensity of each pixel is calculated based on the relative strengths of the red, green and blue channel data from all the neighboring pixels. Each pixel in the converted image now has three parameters: red:intensity, blue:intensity and green:intensity. It goes without saying that every camera uses its own algorithm to make those calculations and - in general - the more computing power the faster the camera can do it, which results in higher frame rates when shooting but also in a (significantly) higher price.
Raw data
RAW data (which Nikon call NEF data, an acronym for Nikon Electronic File) is the output from each of the original red, green and blue sensitive pixels of the image sensor, after being read out of the array by the array electronics and passing through an analog to digital converter. The readout electronics collect and amplify the sensor data and it's at this point that "ISO" (relative sensor speed) is set. If readout is done with little amplification, that corresponds to a low ISO (say ISO 100), while if the data is read out with a lot of amplification, that corresponds to a high ISO setting (say ISO 3200). As far as I know, RAW isn't an acronym, it doesn't stand for anything, it just means raw, unprocessed, data.
Now one of two things can be done with the RAW data. It can be stored on the memory card, or it can be further processed to yield a JPEG image.
If the data is stored as a JPEG file, it goes through the Bayer interpolation, is modified by in camera set parameters such as white balance, saturation, sharpness, contrast etc, is subject to JPEG compression and then stored. The advantage of saving JPEG data is that the file size is smaller and the file can be directly read by many programs or even sent directly to a printer. The disadvantage is that there is a quality loss, the amount of loss depending on how much compression is used. The more compression, the smaller the file but the lower the image quality. Lightly compressed JPEG files can save a significant amount of space and lose very little quality.
RAW to JPEG or TIFF conversion
If you save the RAW data, you can then convert it to a viewable JPEG or TIFF file at a later time on a PC. Since this process takes place on a PC you can now pick whatever white balance, contrast, saturation, sharpness etc. you want. So here's the first advantage of saving RAW data. You can change many of the shooting parameters after exposure. You can't change the exposure (obviously) and you can't change the ISO, but you can change many other parameters.
When to shoot RAW, when to shoot JPEG?
The main reason to shoot JPEG is that you get more shots on a memory card and it's faster, both in camera and afterwards. If you shoot RAW files you have to then convert them to TIFF or JPEG on a PC before you can view or print them. If you have hundreds of images, this can take some time. If you know you have the correct exposure and white balance as well as the optimum camera set parameters, then a high quality JPEG will give you a print just as good as one from a converted RAW file, so you may as well shoot JPEG.
For more information, you can read RAW, JPEG and TIFF by Bob Atkins
Let's come to us now. It is evident from the information above that you can send your RAW (NEF) files to anybody as an attachment to an e-mail message or store them but you can't do much else with it. Even sending such a file via e-mail may not be possible, especially if your ISP provider has set some size limits per e-mail message. NEF files can get really huge. A 6 Mpixel camera (like the D70) will produce a NEF file sized about 5 Mpixels (which will take more than 15 minutes to download on a 56Kbps modem connection. A 12.4 Mpixel camera (like the D2X) will create a 20 MB NEF file which nearly exceeds the limits set by most ISP providers so you can't even share it with your friends (who would have to wait about an hour to download it with a 56Kbps connection - if they love your pictures that much).
Moreover, the quality of the NEF file is not directly "usable", even on your own computer. You can't print a NEF file or use it on the internet for instance. The advantage of the NEF file is that all the information of the shot is in that file so you can use it to create another (usable) file using either a lossless format (TIFF) or a "compressed" (in both size and quality) one (JPEG). In short, the NEF file keeps all the data taken from each and every photodiode, all the settings of the camera and lens used, the flash used etc. The resulting TIFF or JPEG file is the outcome of the interpretation of those settings by the photo editing software you used, according to your preferences (e.g. more contrast, sharpness, shadow detail, different white balance etc.). The initial NEF file is not changed during the process, even when you "edit" it, which means it is like keeping an electronic negative. All that changes is the darkroom process and of course the results you see on screen or your printouts. You can revert any kind of "changes" (= interpretation) you do to that file at any time - the initial file will always stay the same (in contrast to JPEG files which lose quality every time you open/edit/save them). On the other hand, they come with some inherited disadvantages. They are much bulkier to work with (D70=5 MB; D2X=20 MB typical sizes), take a lot of space on your card / hard drive, require a much more powerful computer system to process them and almost always a special plug-in to allow them to be opened by photo editing software (like PS). Working with 20 MB NEF files will bring all but the most powerful computers to their knees while keeping 2 or 3 of them open at the same time (for comparison) will increase your RAM requirements significantly. I personally use a 3.2 GHz, Pentium 4 Extreme Edition system with 2 GB of RAM installed and 4 Hard disk drives (2 x 120 GB, 1 X 160 GB and 1 x 300 GB) to cope with this kind of files and still I can "feel" the tremendous efforts of my system to keep up with them. In contrast, when I use the 5 MB JPEG files of the D2X, the system reacts instantly.
Another issue, which was recently resolved, is the fact that, in contrast to JPEG or TIFF files, you couldn't see the contents of a RAW (NEF) file in Windows Explorer. This called for a camera setting of "NEF + JPEG basic" which effectively stored the NEF file and a small JPEG file at the same time, the latter being used as a thumbnail by the operating system. Seeing the contents of a NEF file is essential since you can evaluate it and delete it on the spot saving a lot of space on your hard drives. A recent addition in Windows XP has solved this issue so you can now see the content of such a file without the need of an accompanying JPEG one.
I am not an "editing" aficionado myself. My belief, originating in those good old film / slide days is that a photo is (or should be) like a woman. In this metaphor, make-up is the editing and "natural beauty" is the picture, as shot. I like beauty, but I appreciate natural beauty more than anything else. Almost any woman can look beautiful with the right kind and amount of make up but the real issue for the male population is which are the ones who will do so even without it. On the same note, some beautiful women can benefit by the use of make up and look gorgeous. Within some limits, I can make an average picture look good (or even very good some times) using the tools of Photoshop. However, every time I do it, I know that this is not the "real" thing. The only real thing that exists is the mistake I did ! In short, I prefer to spend more time while actually taking the photo and edit is as less as possible afterwards. This is not always feasible but usually I can get away with some minor adjustments. I have also found that if you are careful when you take the picture and you follow the basic rules for the exposure (go for the highlights) and composition, you don't really need the NEF file. You can work with the highest quality JPEG file your camera can produce and edit it afterwards, leaving the original JPEG file intact, by saving your edited images under a different name. I usually add the "_mch" extension to all the files which may appear on MCH some time later and the "_mch_t" extension to the thumbnail version of that same file. This way I have the original JPEG file, a file which is ready to be uploaded and a file for the webpage. In short, a total of 5.5 MB instead of the 20 MB needed for the NEF file. Now, this makes a difference of almost 15 MB. If you multiply this by the 5.000 pictures I store every year (out of the 15.000 pictures I take) this saves me 75 GB every year. Actually, the real figure is double that since I save a second copy of every picture (better safe than sorry) on the second external hard drive. The way I work (which I recommend to anybody willing to listen) is to copy the files from the compact flash card to my computer (lately I copy them to my mobile hard disk while still on the field). Once I see them on my computer screen, I then format the CF in the camera (always do this step). All the files are now stored on drive C:\. Next step is to create a folder in the two external hard drives and copy everything there. Thus, no matter what happens, I can always take one of them, plug it on another computer and my pictures are there. I then do all the editing with the copies stored on drive C:\. Once finished, I copy the edited files in the two external drives, too. Every time one of the photos is uploaded in this website, the picture is deleted from drive C:\ only. Keeping multiple copies of your files is something I used to laugh at when I was young and ignorant... Until one nice evening when, due to a system crash, I had to format drive C:\ losing about 6.000 photos. At that point I said to myself, never again. Copying my pictures on CD and DVD is easy but not very practical for two reasons. First, I quickly ended up with piles of CD and then DVD while finding a specific picture became almost impossible no matter what kind of archiving I tried. Second, I have a nice habit of losing things and this covers everything from mobile phones to CD and DVD. Thus, it was necessary to be hard drives, preferably external ones (I use the Maxtor ones).
As you probably understand, file size is an issue with this type of (highly recommended) triple storage. Storing them as JPEG files means 1/4th the space I would need if I was shooting NEF. This results in saving a 150 GB storage device every year. Thus, I have concluded that, unless I foresee that a particular shot will need a modification of the settings which were used to take it, I shoot JPEG, fine, large. With the D70 I use NEF or RAW on a 50-50 basis. With the D2X I use JPEG all the time, especially when I am on a trip. Getting just 50 shots / GB of card is too little, let alone that in each trip I come back with 500-1000 photos which means 25-50 GB and I need weeks to edit all of them. Recently I got myself an external mass storage device with a capability of storing 80 GB which may change this.. to some extend.
To be fair, I must confess that I rarely print my pictures and most of them are viewed on my PC monitor or uploaded in this website. In this respect, I always have to reduce the size of my JPEG images more than four times (from 4288 x 2848 to less then 2000 x 1000) before I upload them and I still know that this a really large image since, even my monitor (a 21" LCD one) will bring it down further to 1600 x 1200 if I am to avoid scrolling. So, what's the use of the bulky NEF file? Needless to say, if you want to make A3 prints of your shots, it may be necessary.. but not for me. On the other hand, if I needed optimal quality then I would have omitted JPEG altogether. In this case, I would decode the NEF file in TIFF file format and keep all the information intact. Keep in mind that TIFF files are also huge, in some cases they can be even larger than the NEF files they came from and this depends on the settings, filters and other editing you have applied to them. If you take this into account you will realize that storage is a key issue and you have to opt for many hundreds of GB if you take many pictures as I do. Anyway, Frank, my webmate, has a radically different approach on this issue so you will read the other side of the same coin below..
And.. the other side...
My view on the RAW-JPG story is the one of perfection. I was a very happy man when I started using the Nikon D70 after the Fuji 4900Z. I took all of my shots in JPG but odd results white balance that I sometimes obtained made me look for more. The free Nikon Editor that came with Nikon View allowed me to set my first steps into RAW editing. I was able to change exposure and white balance within certain limits and this worked well for some time. I kept on reading about photo editing on various forums as I still wanted more. Then I bought the much more powerful Nikon Capture editor as I had read that it "understands" the Nikon colours the best. I learned to use this editing software bit by bit and tried to combine it with an older version of Photoshop to get optimal results. In this learning process I first used some options and tools that I abandoned later because they almost deteriorated the quality of my pictures. My favorite editing routine became converting the NEF files in Capture editor first. Here the sharpening was set to none and adjustments for white balance, contrast, color mode was adjusted when needed. I also learned to use curves to adjust exposure and contrast. From there the images were sent to photoshop in 16bit where additional editing and sharpening was done after a possible crop or resize. Then the D2X came into my life... This beast with it's 20Mb uncompressed RAW files really made clear that I needed to "upgrade" my storage capacity and computing power. The storage problem was fixed very soon, as I bought some additional fast CF cards, so I had 3 of them available. I also bought a new 200Gb HDD for my "old" computer. Another problem was that the 20Mb D2X NEF files really forced the current 1.7Ghz rated Athlon with 768Mb RAM to it's knees, so I should buy a new PC too with at least a 3Ghz CPU with 2Gb RAM, but as I'm only a hobbyist the money has to be spent very carefully. I also want a portable storage device, photoshop CS2, a better tripod with ballhead and some new lenses with matching filters. The PC will be delayed, but I'm not going to give in or make compromises though. Although the JPG performance of the D2X is really wonderful, I just can't stop using RAW. I'm addicted to it's HQ editing possibilities and will be eager to apply all my recently discovered knowledge to my older files if I want or need to. This way I store many many RAW files, but I'm selective though. Every 2 months I delete all photo's that are not good enough to keep essential free storage space for adding new photo's. Of course I'll be much more selective in the more technical fish/nature pictures than in the more sentimental family "snapshots", but I also count on the fact that storage gets much cheaper every year so I can keep on going for some time to come! That aside, the 4.3.1 release of Nikon Capture made editing quite a bit faster, so it's rather unlikely that the computer needs to be replaced as soon as possible.
| < Prev | Next > |
|---|