Reports about completeness & bugs

This is an overflow page for the Using DNG safely page. The latter was getting cluttered up with large numbers of reports about the completeness of the contents of DNG, and reports of problems with Adobe software, most of them duplicates or only of historical interest. This page continues to hold the full list that I have available, while the Using DNG safely page holds what I consider to be just the important, typically recent, subset.


Do DNG files contain all the original raw image data?

Detailed technical documentation about Adobe's support of DNG tends to be fragmented. Detailed information about that support's capability or about potential problems may appear in forums, and eventually disappear when the forums are tidied. This table attempts to capture such information until it is published at a better location. It also contains some information about the quality of the information within DNG files.

Topic Links Commentary
Conversion other than Sigma / Foveon cameras

DNG Workflow Article

PhotoshopNews
2005-05-23

Support of black masked pixels in 3.2 which was released 2005-09-27

Jeff Schewe: "DNG Converter 3.1 and above does indeed move all EXIF private maker note from all TIFF-EP based raw file formats, not just NEF and CR2 files. The file formats that do NOT use EXIF based private maker notes are not migrated. File formats from; Kodak, Foveon and Leaf do not use EXIF private maker notes, so those files are not supported for migration. Additionally, Canon CRW files and Fuji RAF do not completely adhere to the TIFF-EP and EXIF spec. So some metadata will be migrated and some will be stripped. Also, black masked pixels (the far outer edges usually cropped off by conversion software) for certain files, such as some CR2 files are also not preserved in DNG conversion. Future updates to the Adobe DNG Converter will continue to add to the scope of what is preserved and migrated to DNG.
" So, if you shoot NEF or CR2 you will, in essence, loose nothing by converting to DNG. Even the Nikon D2X encrypted As Shot white balance is moved to the converted DNG file. It isn't decrypted, but it is safely moved. As a result, if, at some time in the future Nikon grants Adobe permission to decrypt the WB data, it'll still be there in the DNG file waiting to be used, as will all the other undocumented metadata.
" If your Raw file format is not NEF or CR2, some or most of the proprietary, undocumented metadata found in the private maker notes of the EXIF specification will be stripped out. Is this important data? Hard to say since it's undocumented. However, it's metadata that Camera Raw can not currently use so if you are satisfied with the way Camera Raw can convert your file, then the proprietary metadata may be meaningless to you".

GroupBrowser
2005-05-23
Thomas Knoll: "The code change in ACR/DNG 3.1 that preserves private MakerNotes applies to all formats that store their private data using the Exif MakerNote format, which includes Pentax. These MakerNotes are often byte order or offset dependent, so what I'm doing to preserve this data safely is repackaging it into the DNGPrivateData tag. This repackaging stores the source file's byte order and the original file offset to the MakerNote, along with the MakerNote data itself. This allows an application that understands the MakerNote format for a particular camera to find the MakerNote data, along with the original byte order and file offset, and thus parse the MakerNote data correctly from the DNG file".
exifprobe - camera image file probe This discusses whether raw formats are TIFF-based or not. It is not clear whether the "TIFF & TIFF- derived "raw"" formats satisfy the MakerNote principles above.

Delete Proprietary files after using Adobe DNG?

GroupBrowser
2005-04-04

Bruce Fraser: "The reason to embed the original raw file is to keep obscure bits of private metadata that only the proprietary raw converter understands. Now, it's possible that the camera vendors may be persuaded to document these private tags sometime before hell freezes over, and in that case, it's conceivable that other software may be able to use them".
Barry Pearson: "Once I have at least 2 DNGs of each file in different places, I delete the original Raws, without embedding them first. And I sleep easy at night.... So, I'll say "don't try this at home", while doing exactly that myself!"

Why save raw files that have been converted to DNG? GroupBrowser
2005-05-30
DNG Workflow Part II

GroupBrowser
2005-05-25

PhotoshopNews
2005-05-25

Jeff Schewe: "Another little known benefit of using the Adobe DNG Converter is the ability to do a simple verification that the raw file you are converting is not corrupted. If the DNG Converter encounters a file that has problems, it will throw an alert error in the Conversion Status dialog as shown above.
"While rare, it's an early warning sign that there may be problems with either that file or the directory structure of your compact flash card. Generally a simple finder copy will not show an error or problem when copying a corrupted image file-it merely copies the corrupted file and you won't discover the problem until later when trying to open the file. Note that this warning is not a complete fail safe mechanism. It's merely intended to alert you to a potential problem and indicates which file or files are involved. This will give you a warning that perhaps stronger recovery may be needed before removing the card and reformatting it".
DNG Developers: Update to DNG 1.1.0.0 support now

GroupBrowser
2005-06-04

This is the solution to the "black masked pixel" problem

Thomas Knoll: "This message is a heads up for all developers of software that reads DNG files: You need to update your DNG support so your can read DNG version 1.1.0.0 files.
"Background: Adobe published the DNG version 1.1.0.0 specification in February 2005, and added READ support for DNG version 1.1.0.0 version files to Camera Raw 2.4 and later. However, the DNG Converter versions 2.4, 3.0 and 3.1 continue to WRITE DNG files with version 1.0.0.0.
"As of Camera Raw/DNG Converter version 3.2 (available later this year), Adobe is planning in switching to writing version 1.1.0.0 files, since we are going to start taking advantage of this new version's ability to preserve the black masked pixels that some raw formats (mostly Canon's) store. This change will be transparent users of Adobe software (except users of Camera Raw 2.3 will need to update to Camera Raw 2.4). However, this will require that third party software be updated to read this new DNG version. The changes required are minimal, and are described in DNG 1.1.0.0 specification".

How does DNG handle Nikon curves and Canon picture styles? GroupBrowser
2005-12-04
Thomas Knoll: "Both the Canon picture style and the Nikon curve information is stored inside the private MakerNotes of their raw formats. Starting with ACR/DNG 3.1, these private MakerNotes are copied (along with original file offset and byte order information) into the DNGPrivateData tag in DNG files. So all this private information is now preserved in DNG".
Does DNG import data from Canon THM file?

GroupBrowser
2005-12-12

David C Anderson: "But what about the Canon THM sidecar file? Does all the info (whatever that might be) contained in the THM file get transferred into a DNG file when its matching CRW is converted?"
Thomas Knoll: "Yes. Actually, the ".THM" files directly from the camera don't contain any metadata that is not already in the ".CRW" files. Some third party software writes IPTC data into the ".THM" files, which the DNG Converter does read and include in the ".DNG" file".

ORF conversion to DNG - good or bad results?

ORF vs DNG

DPReview
2006-05-26

FourThirdsPhoto Forum 2007-02-27

DPReview
2007-02-27

Michael Meissner: "Olympus uses a screwy encoding of the MakerNotes, and if you just rewrite the MakerNotes, you will write incorrect information (in some cases Olympus puts the wrong length information, in other places, it uses absolute addresses within the file, which have to be adjusted whenever the file is rewritten)."
Boardhead (Phil Harvey, ExifTool author): "The Olympus maker notes contain pointers which reference information outside the makernotes data block. This information (a few kB worth) is not copied by the DNG converter (I just tested DNG Converter version 3.4 and the problem still exists). To properly copy ORF maker notes, you need to understand the structure".
marlof: "Then I run a batch file converting the ORFs to DNG files, and using ExifTool to copy the EXIF info from the ORFs to the DNG files". (Documented here: DPReview).
Douglas Palmer: Windows & Mac scripts here.
DPReview
2007-04-27
Michael Meissner: "Given that the E-410, E-510, and presumably the P-1 use a new compressed RAW image that first showed up with the SP-550, you might have to wait a bit for the ORF->DNG converter to catch up. As far as I understand it, Olympus also fixed the bogus way they stored the MakerNotes in the new RAW file, so you won't have to depend on something like exiftool to restore the MakerNotes if you use that information".
"Quality" of DNG GroupBrowser
2006-05-30
Jim Hess: "I purchased a Fuji S9000 camera.... "
Thomas Knoll: "The DNG version of the file contains EXACTLY the same image data (bit for bit) as the original RAF file".
"DNGPrivateData and MakerNote tags" GroupBrowser
2007-01-08

John Francis: In the DNGPrivateData MakerNote copy contents are:
1) Six bytes containing the zero-terminated string "Adobe" (the DNG spec calls for the DNGPrivateData tag to start with an ASCII string identifying the creator/format). 2) An ASCII string ("MakN" for a Makernote), presumably indicating what sort of data is being stored here. Note that this is not zero-terminated. 3) A four-byte count (number of data bytes following); for a simple MakerNote copy this the length of the original MakerNote data, plus six additional bytes to store the next two data items. 4) The byte-order indicator from the original file (the usual 'MM'/4D4D or 'II'/4949). 5) The original file offset for the MakerNote tag data (stored according to the byte order given above). 6) The contents of the MakerNote tag. This appears to be a simple byte-for-byte copy, with no modification.
Thomas Knoll: The best documentation for this is the source code in the DNG SDK.

"Pecularity of WhiteLevel" GroupBrowser
2007-06-05
G Sch: "While analyzing the raw images of several cameras in DNG format, I found something strange: the WhiteLevel value is very different in images created by a Canon 30D".
Thomas Knoll: "Figuring out what the white level is Canon files is somewhat of a mystery. I do the best I can based on the model, iso, and actual image. However, without documentation from Canon, it is always somewhat of a guess".
"Canon 1D Mark III diffference" DPReview
2007-06-11
Barry Pearson: using the DNG route "A small number of pixels differ by 1 in one of the channels".
"Adobe DNG file size is smaller than RAW camera files (cr2)" GroupBrowser
2007-07-03
G Sch: "As to the statement that there is no data loss: it is simply not true".
Thomas Knoll: "You are not correct. The image data from CR2 files is preserved EXACTLY, bit for bit, including the "values less than black"".
Sigma / Foveon X3F conversion
How is Foveon sensor data held in a DNG? GroupBrowser
2005-12-04

DNGs created from a Sigma10 X3F have PhotometricInterpretation of LinearRaw, whether or not the "Convert to Linear Image" option in the DNG Converter is set. (And Silkypix, which can handle Linear DNGs but not X3Fs, can handle such DNGs).
Foveon libraries are used by ACR and the DNG Converter to handle the X3F images. The libraries embedded in early versions of ACR 2.x were later replaced with others.

Conversion of Foveon X3F files to DNG

DPReview
2007-03-26

DPReview
2007-03-26

Zalman Stern: "X3F files are similar to DNG in that the format is designed to contain all the info that a converter needs to turn the raw file into an image. Thus the Foveon provided code used in prior versions of ACR converts the SD14 files as it did with SD9/SD10 files.... The issue is that Foveon has evolved their algorithms for doing the conversion and thus can offer higher quality with their new code. This code is not in ACR 3. or 4.0".
Zalman Stern: "There is no way to support Foveon X3 data in DNG without publishing intimate details of the off-sensor data, which is Foveon's intellectual property".

What about bugs and deficiencies in key software products?

Bugs and deficiencies in DNG-handling software do occur, as illustrated in the table below. Sometimes the problem is actually with ACR rather than the DNG Converter, or even with the camera manufacturer. Typically the work-around is to not take the upgrade concerned. Typically the bug is solved in the next upgrade. Another problem is with unofficial support of new cameras, which is sometimes not up the standard of official support. Once again, typically the issue is solved in the next upgrade.

Topic Links Commentary
Problems before version 3.0
Upside down DNG conversions(?) GroupBrowser
2006-02-06
Jim Hess: "I have read reports of DNG images converted from some Fuji raw files being upside down when they are first opened in Photoshop or Camera Raw".
Thomas Knoll: "There was also a bug in Camera Raw versions before 3.0 in how they interpreted some orientations used by the Fuji cameras. If you use a post 3.0 DNG Converter and pre 3.0 version of Camera Raw, sometimes the orientation is wrong".
Problems with version 3.1 and earlier
350D Raw files and CS2+ACR 3.1 problem
This is not a DNG problem.
GroupBrowser
2005-08-19
Thomas Knoll: "Canon appears to have installed a new firmware version (1.0.2) when it serviced your camera, and the EXIF model name string is different in this firmware release. Instead of "Canon EOS 350D Digital" it is now "Canon EOS 350D" with the new firmware. Camera Raw uses the make and model EXIF tags to figure out which kind of raw file it is reading, and since the model name no longer matches the value it was expecting, it cannot read the file. You will need to wait until a future version of Camera Raw is updated to detect this new model string".
Camera raw image files have incorrect creation times (Bridge 1.0) Adobe Support Knowledgebase
2006-02-08
Problem: "Adobe Bridge lists incorrect creation times for camera raw image files from some Canon digital cameras.... Incorrect creation times appear in all files you create from the camera raw image files (for example, PSD files, JPG files, and DNG files).... The Camera Raw plug-in (version 3.1 and earlier) incorrectly reports the Date Time Original value saved with some Canon camera raw formats.
Solution: "Install the Adobe Camera Raw 3.2 update".
Problems with version 3.2
DNG dinging my data? GroupBrowser
2006-02-27

Joe W Jackson: "I converted my Olympus E-1 RAW files (ORD) to Adobe DNG files. But when I compare the ORD and the DNG in ACR, the DNG version has matrixy looking artifacts visible at any viewing resolution greater than 100%".
Thomas Knoll: "Update to Camera Raw 3.3".
Joe W Jackson: "Thanks Thomas. That took care of my problem".

Adobe Support Knowledgebase: "When you convert images to DNG using the Camera Raw 3.2 plug-in and open them in Adobe Photoshop CS2, they contain artifacts.... This issue affects DNG files, but it is a problem in the Camera Raw plug-in. The cameras affected include the Panasonic LX1 and FZ30, the Leica D-Lux2, and the Olympus E-1".
Camera Raw 3.2 DNG images open with artifacts in Photoshop CS2 Adobe Support Knowledgebase
2006-03-20
Raw (SR2) and DNG look different in Camera Raw GroupBrowser
2006-04-15
MikeKatz: "Can anyone tell me why the Sony R1 .SR2 files do not look the same in ACR as the DNG files created from that same file?"
Thomas Knoll: "The Sony R1 was not officially supported until ACR/DNG 3.3. While there was some preliminary support in ACR/DNG 3.2, it was not finished yet. Always use the most recent version of the DNG Converter (currently 3.3). When using the DNG Converter on files from a camera model that is not on Adobe's officially supported list, never discard the originals files. You should probably re-convert them when a version of the DNG Converter comes out that does officially support your camera model".
Problems with version 3.4

Camera Raw 3.4 - Problems With This Version!
This is not a DNG problem.

GroupBrowser
2006-05-10

Various: ACR 3.4 doesn't work on some Athlon processors.
Thomas Knoll: "We have confirmed that several similar Athlon processors that also don't have SSE2 instructions work with 3.4 anyway, so it appears this problem is limited to only some Athlon processors without SSE2, not all without SSE2".

Problems with version 3.5
Crashes when using "update JPEG previews" CS3 forum
2007-01-24
Thomas Knoll: "The bug with updated the embedded JPEG previews in DNG will be fixed in ACR 3.7 and the final version of ACR 4.0. This bug is specific to multiple processor (or core) Windows machines. The workaround is to disable the update embedded JPEG previews option.... The bug is also highly timing dependent. Some users can make it go away by changing the size of the embeded previews, but that fix would certainly not work all the time given the nature of the bug".
Problems with version 3.6 & 3.7 (Mac)
DNG vs NEF White Balance Mismatch GroupBrowser
2007-02-21
Cajun Man: "D2X ... I have found that my photos often have very different White Balance values -- for both Color Temperature as well as Tint -- when comparing the original NEF and a DNG created by Adobe DNG Converter".
Thomas Knoll: "There is a bug in the Macintosh version of the DNG Conveter 3.6 and 3.7 in the handling of the white balance from these Nikon cameras. Either go back to version 3.5, or wait for 4.0. Or save the DNG files directly from the Camera Raw plug-in.... If you are brave, you can try to fix this yourself. Open the package (control click and pick "Show Package Contents") for the both DNG Converter 3.7 application and the Camera Raw 3.7 plug-in. Navagate inside each package into the Contents/MacOS folder. Copy the file "NkMiniLib.dylib" from the Camera Raw package into the DNG Converter package".
Problems with version 4.5
DNG converter 4.5 command line broken GroupBrowser
2008-08-11
DNG converter 4.5 needs full path names in its command line.
Example: "Adobe DNG Converter.exe" -c d:\pictures\PICT0001.MRW
The older versions did not require the full path name and would also work like this:
"Adobe DNG Converter.exe" -c PICT0001.MRW provided that the working directory was "d:\pictures".
Problems with Lightroom 1.4 (Windows)
DNG Conversion Error (Windows Only) Lightroom Journal
2008-03-16
"Adobe has included technology to verify that the image data in a DNG file is unchanged from when it was originally converted to DNG. ... when converting to DNG using Lightroom 1.4 on Windows, the application will write an incorrect verification tag to describe the image data. When Lightroom attempts to work with those files in the Develop module, the application reads that incorrect tag, believes that there is something wrong with the raw data and will present an error. ... there is nothing wrong with the integrity of your image data or metadata".
Problems with various versions
DNG Converter buttons DPReview
2006-10-08
Erik Magnuson: "the DNG converter will cut off the bottom buttons with certain WIndows Xp font options (and the window is not resizeable!)"
DNG files with more than four channels? GroupBrowser
2007-04-15
Lawrence Taplin: "... the DNG SDK has kMaxSamplesPerPixel set at 4... Photoshop won't read other than the thumbnail".
Thomas Knoll: "... If there is a good reason, Adobe might lift the limit in its code"
Lawrence Taplin: I am working with several six-channel cameras designed for museum imaging ... It would be ideal to store the images in a publicly documented format that can be opened with standard software for colorimetric rendering".
Inconsistent DNG labeling: Metadata Browser Adobe Lightroom Forum
2007-04-22
Simon Iannelli: "DNGs I created with the multilanguage version of DNG converter are labeled as 'digital negative'. DNGs I created with the german version of Lighroom are labeled as 'digitales negativ' which is the german translation of 'digital negative'. Due to this the Metadata Browser has two categories for DNGs".
Problems with non-Adobe products
This section is not intended to be exhaustive. With over 200 non-Adobe products handling DNG files, inevitably there are bugs in some DNG-handling software! These are therefore selected because of their wider significance.
Apple Mac OS X v10.5.2,
Mac OS X Server v10.5.2

About Security Update 2008-002
(which cures this bug)

"A stack based buffer overflow exists in the handling of Adobe Digital Negative (DNG) image files. By enticing a user to open a maliciously crafted image file, an attacker may cause an unexpected application termination or arbitrary code execution. This update addresses the issue through improved validation of DNG image files".