Dialogue with Juergen Specht

(Juergen Specht was asked to check this page for errors. His reply didn't identify any errors).



"I would not join any club that would have someone like me for a member."
Groucho Marx


Support and constructive criticism of OpenRAW

I was one of the earliest supporters of OpenRAW - number 11. I was also critical of OpenRAW's blind spots. Although my criticism was constructive, offering a way for OpenRAW to remain relevant to photographers, my views were sometimes unpopular. Eventually, I was banned from the website.

My view is that there were two initiatives for achieving "Digital Image Preservation Through Open Documentation". (That is OpenRAW's motto).

  • DNG, which has a design objective of being a true archival raw file format, in addition to its other role as an interchange format. (It can achieve this whether used in-camera or via conversion from other raw formats).
  • OpenRAW, which had a good "brand". Being independent of Adobe, (as I am too), it could be an effective lobbying force to ensure that DNG was evaluated and evolved for archival purposes, and exploited by camera manufacturers, software developers, photographers, and users of photographs.

At first, OpenRAW claimed (without evidence) that its objectives could be achieved without the use of a common raw format. (Realistically, DNG was the only contender for a common raw format).

Later, OpenRAW specifically opposed various aspects of DNG, and developed an attitude of "DNG is not the solution". Its hardening opposition to DNG paralleled the steady increase in the exploitation of DNG by all concerned . If OpenRAW saw DNG as a competitor, this could be expected.


Juergen Specht can be a bit "economical with the truth"!

He (erroneously) said in the OpenRAW mailing list: "Imagine you convert your proprietary RAW formats into DNG today and throw away your originals (a procedure a self acclaimed DNG expert named Barry Pearson promotes all over the Internet)".

Needless to say, I have never promoted any such thing! The truth is here: "This should only be used by people who have sufficient knowledge to make an informed decision about the consequences".

Juergen Specht and DNG

The OpenRAW Working Group is coordinated by Juergen Specht. The OpenRAW web site is hosted and maintained by him. Several anti-DNG articles at OpenRAW and elsewhere are in his name, including those below.

The dialogue below reveals some fallacies in his arguments. These are similar to fallacies long used on the OpenRAW site:

  • Failure to distinguish between a lossy format and lossy use of a lossless format
  • Failure to distinguish between some lossy uses and majority lossless uses
  • Failure to distinguish between important data and unimportant data
  • Failure to distinguish between lost information and undocumented information
  • Failure to distinguish between "what is done now" and "what can be done in future"
  • Failure to judge alternatives by the same criteria

The OpenRAW and Juergen Specht themes "DNG is not the answer" and "DNG is unfortunately not a solution" are simply "blocking" statements to prevent examination of the ways that DNG is part of the solution.


A personal frustration

My pages here are probably the most comprehensive source of information on the web about DNG that is independent of Adobe. I publish them because I am a highly qualified engineer who knows that the raw shooting industry needs an open interface between cameras and raw shooting software products, and a photographer who wants to take advantage of the best the industry can offer.

As an engineer, I feel compelled to investigate and publish both sides of the argument. So these pages provide probably the most comprehensive material on the web of the disadvantages of DNG, such as examples of cases where the DNG Converter doesn't copy all metadata across, or (the rare) bugs in software. I publish a list of products that don't support DNG, to help people avoid DNG if they would not benefit from it. My aim is to enable people to make informed decisions to suit themselves. I don't want to make their decisions for them.

I am disappointed when people with a personal agenda select the "negative" information that I have published, rather than adopting a balanced view. It is especially disappointed when I am simultaneously criticised for being "a sales guy for DNG", being "incredible clueless", being "officially an Idiot", yet having my own detailed analysis, (that Juergen Specht is probably incompetent to perform for himself), used in a biased way against DNG!


The dialogue!

This is a chronological record of 4 days worth of dialogue between Juergen Specht and myself. It isn't vital reading! I've recorded it because sometimes threads are deleted from OpenRAW, and I've lost articles there in the past.

I've added some sidebars to my own articles. I've let Juergen Specht's articles speak for themselves. (The first article wasn't by either of us, but triggered the dialogue).


(DudeMan) Converting current RAW images to Open RAW

Please pardon my ignorance if this has been covered somewhere and I just didn't notice it.

I was wondering if there's anything in the works as far as software for converting existing RAW images into an open format? An open format right from the camera would obviously be great, but it does nothing to solve the problem of aging RAW files that may lose support in the future.

I don't know what kind of technical problems exist for this kind of software, but it seems to me that it could be made as a batch processing plugin for UFRaw. Add in the ability to do lossless compression and you'd have something that's quite practical for backup purposes.

While I'm dreaming, let's throw in the ability to create a portable RAW file, something that's reduced in detail to get the file down to under 2mb that could be more easily distributed over the web.

DudeMan – Sat, 2007/02/03 – 6:20am


(Barry Pearson) You have just described DNG!

DNG is an open raw file format. It was launched nearly two and a half years ago, and is currently supported in some way by more than 150 products from more than 140 companies. (This includes several cameras and digital backs. The number increases all the time). There are a number of DNG converters that convert from existing formats to DNG.

The reference to "lossless compression" was a response to the original post above.

Juergen Specht's response below misses the point about what lossless compression refers to!

DNG supports lossless compression. I believe it can be read by UFRaw. (I've been using it successfully for years).

DNG has excellent archival characteristics. It is identified by the US Library of Congress as the preferred alternatives to other raw formats such as NEFs, CR2s, etc. You can read a lot about it here:
http://www.barrypearson.co.uk/articles/dng/

Barry Pearson – Mon, 2007/02/05 – 2:38am


(Juergen Specht) But it is a lossy format.

But it is a lossy format. So much for your lossless compression. ;)

Juergen Specht – Mon, 2007/02/05 – 7:23pm


(Barry Pearson) DNG is a lossless format

DNG files are capable of holding all the raw image data from cameras with CFA-based sensors, including those with offset sensors such as Fujifilm cameras. They are capable of holding all EXIF data, including Makernotes. They are capable of holding lots more, for example TIFF/EP metadata and XMP metadata.

Consider an analogy. TIFF is recognised to be a lossless format. But suppose a software product sometimes wrote TIFFs with bits of the image missing for some reason.

Would TIFF then be criticised as a lossy format?


JPEG is accepted to be a lossy format. To use the term for a format like DNG that can store the entire raw image data bit-for-bit is misuse of the term.


For NEFs, CR2s, PEFs, and possibly some other formats, Adobe claim that all the information from the original raw file is copied across the DNG file. (And Juergen Specht has not shown otherwise). Calling DNG a lossy format does a disservice to users of Nikons, Canons, and Pentaxes!


The motto of OpenRAW is "Digital Image Preservation Through Open Documentation". Not "Metadata Preservation Through Open Documentation"!

Whether they actually hold all the data depends on how well the software used, (for example DNG converters, of which there are several), copies it across. The claim is that for NEFs, CR2s, PEFs, and probably some others, it is all copied across. Anyone claiming differently should provide suitable evidence, instead of exploiting "fear, uncertainty, doubt".

Two main types of problem with DNG conversion are known:

  1. One problem is that the developer of the DNG converter doesn't know enough about the original raw file format to be able to copy all the data across. Typically, the raw image data is copied across, so high quality conversions can still be performed. ("Digital Image Preservation" is achieved). But some non-essential data may be missed, for example previews, or some information that isn't actually needed by most mainstream raw converters. (A problem with ORFs was identified with version 3.4 of the DNG Converter).
  2. Another problem is that data that IS copied across into the DNG isn't understood, other than by the company that created the original raw file. As above, high quality conversions can still be performed without knowing this information. Examples include such added-value as "picture styles", "curves", perhaps lens-identification information, etc. In this respect, DNGs are obviously no worse than the original raw files, and since the rest of the DNG file is understand, and also the DNG Converter puts EXTRA data into the DNGs, DNGs are a much better archival format than the originals.

The solutions to both of these problems need better information about the original raw file formats. In the first case, that will result in better DNG converters being written. In the second case, simply documenting the Makernote automatically provides documentation for the copy of it in the DNGs. Since Adobe software and some other mainstream raw converters don't use that data anyway, solving these problems would actually make no difference to the resultant image.

DNGs generated from documented original raw files will be a better archival format than the documented original raw files themselves. This is for two main reasons:

  1. First, the DNG files contain extra camera details so that future raw converters don't have to have camera details built in. In (say) 20 years time, while documentation of an original raw file format may enable a raw converter to READ the file, without being able to test a camera of that model it may be hard to determine how to PROCESS that data. Raw files, other than DNGs, typically don't contain all the necessary information for a high quality raw conversion, so merely documenting every detail of the raw file format is insufficient.
  2. Second, DNG largely solves the problem of the proliferation of raw file formats and variations of these. Therefore, it is easier to achieve critical mass. One good implementation will support DNGs for images captured by many camera models, making it far more likely that a company will bother to implement it. Software companies support cameras according to their business case, not just because their raw file formats are documented.

When OpenRAW was launched, there were about 25 non-Adobe products that supported DNG in some way. Today there are more than 160, including several cameras and digital backs, plus some software products from some camera manufacturers. There is also more known about the contents of Makernotes, hence of copies of those in DNG files.

Barry Pearson – Mon, 2007/02/05 – 11:53pm


(Juergen Specht) So you just confirmed its lossy ;)

Barry, nobody takes you seriously if you just sounds like a sales guy for DNG. You should just stop your crusade.

Juergen Specht – Tue, 2007/02/06 – 10:00pm


(Barry Pearson) DNG beats all other raws

To someone opposed to DNG, a statement of the facts sounds like a sales pitch!


It is important to judge any alternative to DNG by the same criteria used to judge DNG!

It would be silly to say "Nikon don't publish the format of their Makernotes, therefore when it is copied across to DNG its format isn't documented, therefore we should reject DNG and stick with NEFs"!

Yet that is often the "logic" used by critics of DNG. DNG gets rejected because of detailed issues that are a small subset of the issues of other raw file formats.

Let's clear. DNG is better than ANY other raw file format for archival purposes. The DNG specification provides a lossless raw file format. Anyone with evidence to the contrary should provide it, instead of resorting to "fear, uncertainty, and doubt" to further their own agenda.

Any losses are the result of policies of camera manufacturers, not the DNG specification itself. Major formats such as NEFs, CR2s, PEFs, are believed to be converted to DNG without loss, because they are understood well enough, and are well-formed enough, that the conversion can copy everything across. (Indeed, the DNG file contains a SUPERSET of the original information). Some other formats are not (yet) understood well enough. But at least the raw image data is copied across, which is what matters to serious photographers. Anyone with an obsession about every last detail of metadata, for example perhaps which lens was used, or perhaps the JPEG preview used in-camera, may not like this. Do they expect that information to improve their results?

If camera manufacturers don't publish their formats, DNG is better than the camera manufacturer's own format, because, except for those last details, the result is openly documented. (As in "Digital Image Preservation Through Open Documentation"). If the camera manufacturers DO publish their formats, then for the cases where all the data is copied across, we will understand what all that data means. And where it isn't, the DNG Converter can be improved, and the result will still be that DNG is the better format.

It is obvious that the problem isn't just about undocumented raw file formats, but also about their proliferation. The evidence is that software developers don't support formats just because they are documented, but rather based on the business case of supporting a particular camera. Therefore, mere documentation of vast numbers of raw file formats won't enable people to be sure of being able to process their images in the future with the tools of their choice. They may have to use older tools that can't get the best from the raw image data concerned.

What OpenRAW should have done from the start was tackle the "proliferation" problem as well, and promote the need for a common raw format. I supplied a useful definition of "common raw format" to OpenRAW that could have been used. It is interesting that OpenRAW used to say: "Many have suggested (and Adobe has created) a common, open file format for RAW image files for all camera makers to use as a solution to the RAW problem". Then the bit about "Adobe has created" was deleted. Was DNG getting in the way of OpenRAW's own agenda?

If you go back through posts from OpenRAW's supporters, you will find that a proportion of them think that OpenRAW is trying to devise such a format, or else they specifically request such a format. Now that DNG is becoming so well known and used, if OpenRAW is ever to be taken seriously, it needs to adapt to DNG. It should be trying to raise the quality and documentation of DNG conversion, and encouraging companies, (camera manufacturers, software developers, etc), to use it.

OpenRAW doesn't exist in a world where DNG can be ignored for purposes of "Digital Image Preservation Through Open Documentation". The US Library of Congress identifies DNG as the preferred alternative to other raw formats. Librarians and archivists across the world are starting to watch DNG to make a judgment about when it becomes their recommendation. (I have read a number of papers on this topic that mentions DNG). Such people want identifiable formats that they can recommend, not just a proliferation of formats. They are starting to talking about DNG in the same sentences as TIFF and PDF, etc. They certainly aren't talking about NEFs, CR2s, etc, in that way! They never will.

We have TIFF (now owned by Adobe) largely as an alternative to each scanner manufacturer using their own scanned file format. Look how successful TIFF has been in opening up the industry. We need the equivalent for raw file formats, and there is now only one contender. Many photographers have voted with their feet. They weren't going to wait for some hypothetical future where camera manufacturers open up about older formats.

Barry Pearson – Wed, 2007/02/07 – 6:45pm


(Juergen Specht) You are a funny fanatic, Barry

First you try to push your own agenda by posting wrong statements ('DNG is lossless') than you back up and give camera makers the fault that DNG cannot have all data, because camera makers don't document it.

And thats exactly our point. DNG could be so much better IF camera makers would document openly their file formats.

We have absolutely nothing against DNG, but a file format which so far (at least the converted DNG) throws away data, is lossy and its too early to throw away data at this stage.

We made comments how to improve the DNG format for all the reasons you mention, with the sole intend to improve the situation and as you probably know, we are not in a competition with Adobe, far from it.

But your statement about 'anyone with an obsession about every last detail of metadata' really just shows how incredible clueless you are and how little you understand about this topic.

Juergen Specht – Wed, 2007/02/07 – 10:10pm


(Barry Pearson) OpenRAW's views on DNG

You say: "DNG could be so much better IF camera makers would document openly their file formats."

I have posted the view on the OpenRAW website (2005-11-27) that the main value of OpenRAW will be to improve the quality of DNG converters, and ensure that more products exploit DNG.


I don't recall Juergen Specht saying anything similar. He didn't respond when I posted (2005-06-24) "OpenRAW objects to the fact that is still has private data fields, but they are not vital, and it would be possible to identify a subset that didn't have this problem. (I have suggested the name DNG/OpenRAW)".


Since posting the original article, I've replaced use of "prominent.jpg" with "prominent.gif" to reduce download time:

Screen-shot of part of the OpenRAW home page

And I have been saying that for a long time, here and elsewhere. Here is what I said nearly a year ago: "OpenRAW are doing the right thing, except for their antipathy towards DNG. And if they succeed with their lobbying, the world of "raw shooting" will transform faster into the world of "DNG shooting". That will be good for the future health of top-end digital photography."
http://www.barrypearson.co.uk/articles/dng/openraw.htm#conclusion

You say "We have absolutely nothing against DNG, but a file format which so far (at least the converted DNG) throws away data, is lossy and its too early to throw away data at this stage."

But OpenRAW has shown antipathy and even hostility towards DNG. For example, it published "DNG is not the answer" very prominently:
http://www.barrypearson.co.uk/articles/dng/commentary1.htm
http://www.barrypearson.co.uk/articles/dng/assets/prominent.gif

And I repeat: "DNG files are capable of holding all the raw image data from cameras with CFA-based sensors, including those with offset sensors such as Fujifilm cameras. They are capable of holding all EXIF data, including Makernotes. They are capable of holding lots more, for example TIFF/EP metadata and XMP metadata". Do you accept that statement?

More recently, in response to "Is the Adobe DNG format the pragmatic solution or maybe the same old song?", you responded:

"I wish it would be a solution, but Adobe decided for marketing reasons that they allow a backdoor to camera makers to store information in so called private tags, which remain undocumented. So even if some cameras can save DNG formats natively, some of the information a camera decides to conceal can be saved inside the DNG format and it becomes another undocumented RAW format after all. Plus Adobe stopped documenting their PDF and PSD format after it reached a certain market share, so they can at any time release a DNG V2.0 format and decide not to document it. No, DNG is unfortunately not a solution".
http://www.barrypearson.co.uk/articles/dng/commentary2.htm#juergenspecht

Note how inaccurate some your response was.

At various times, OpenRAW has expressed "fear, uncertainty, and doubt" about DNG. OpenRAW could have stated a policy towards DNG that included helping to improve the specification, and/or helping to improve the quality of DNG conversions, and/or helping to promote the use of DNG. It has done none of these. (More precisely, I have done some of those things on the OpenRAW website, but the originators of OpenRAW didn't).

For example, OpenRAW disparages the idea of using a common, openly documented RAW format for solving this problem, in favour of a proposal for which there is no supporting evidence:

"Many have suggested a common, open file format for RAW image files for all camera makers to use as a solution to the RAW problem. A common, openly documented RAW format would fulfill many of the goals of OpenRAW, but is likely to face significant resistance from manufacturers who feel their "creativity" and ability to innovate would be constrained. Open documentation of all RAW file formats by manufacturers is the quickest and most satisfactory way for OpenRAW's goals to be reached".

Where is the evidence that "Open documentation of all RAW file formats by manufacturers" will achieve "Digital Image Preservation"? The latter requires software products in future which can extract and exploit the raw image data from image files. That requires knowledge of both RAW file formats and of the details of the cameras that created that image data so that it can be processed. And it requires the software company to believe there is a suitable business case. Where will those camera details come from in future? What will be the business case? It is just wishful thinking!

You say: "But your statement about 'anyone with an obsession about every last detail of metadata' really just shows how incredible clueless you are and how little you understand about this topic."

Chuckle! Some of the world's experts on digital imaging and asset management for image archiving agree with the views I express, rather than the views you express. More precisely, I agree with the views they express!

But I have made an open offer (to any readers) on my website: "If you know of errors in my pages, please let me know, with suitable supporting material, and I'll correct them". And another: "You may not agree with all of my opinions. If you publish an alternative position in some consolidated form, such as a web page, (rather than responses in forums), please let me know and I'll link to it". Given that I have published more than 20 pages on this topic there, where is your evidence that I am "clueless"? (I wonder if you have even read those pages? Or made any significant use of DNG? Or analysed DNG and other raw files using tools such as the DNG SDK and others?)

NEFs, CR2s, PEFs, etc, are NOT suitable for "Digital Image Preservation Through Open Documentation". They never will be (and Pentax at least accepts that). Mere documentation of camera manufacturers' raw file formats is not the solution to "The RAW Problem". The only raw file format specifically designed for that purpose is DNG. What is the alternative?

Barry Pearson – Thu, 2007/02/08 – 12:37am


(Juergen Specht) With all due respect

With all due respect, Mr. Barry 'DNG is lossless, except the little bits it throws away nobody needs anyway' Pearson, you are officially an Idiot.

And you can quote me on that :)

Juergen Specht – Thu, 2007/02/08 – 5:59pm


(Barry Pearson) OpenRAW rejection of DNG

This is probably the article which got me banned!

I have witnessed, and recorded, the bewildering trend on the OpenRAW website from dismissal of DNG as part of the solution towards open hostility towards DNG.

This is more than rejection of DNG as a specific file format. It is also rejection of the value of a "common raw format". (Or other terms such as "common, open file format", or "universal RAW format").

I suspect that such a concept is seen as competition with the primary target of OpenRAW - persuading camera manufacturers to publish their proprietary raw file formats. If so, that would explain why the concept was downplayed in the OpenRAW version of "The RAW Flaw".

Photographers need better! They need advise on what to do now, and reassurance that they will see tangible tools and methods that they will be able to adopt.

OpenRAW once prominently displayed a document called "The RAW Flaw". The authors were Michael Reichmann (of Luminous Landscape) and Juergen Specht (of OpenRAW). It was published on both sites:
http://www.openraw.org/actnow/
http://www.luminous-landscape.com/essays/raw-flaw.shtml

There were illuminating differences in the OpenRAW version:

The Luminous Landscape version said:

"The Solution? There really is only one solution – the adoption by the camera industry of...
A. Public documentation of RAW formats; past, present and future
or, more likely...
B. Adoption of a universal RAW format"

The OpenRAW version didn't have "or, more likely..."! This increases the emphasis on public documentation, and reduces the emphasis on a universal RAW format.

Another difference in the OpenRAW version was lack of the paragraph:

"Finally, consider the problems of digital asset management and the cataloging of files. Many pros as well as amateurs have literally Gigabytes of RAW files. The various asset management programs can't hope to keep up with the ever increasing number of proprietary formats. And as time passes and these programs are enhanced, what are the chances that they will still be able to read your older RAW files?"

That paragraph made the point that it isn't sufficient to have public documentation, but it is also important to have a universal RAW format to reduce the proliferation. This was not the message that OpenRAW wanted to give.

Other parts of the page DO mention the use of a a universal RAW format, and mention DNG by name. That page is no longer prominently displayed at OpenRAW. People who didn't know it was there would probably not find it. The concepts of "a universal RAW file format" and "Adobe has put forward the DNG format as an open standard" have been hidden by OpenRAW. Whereas Michael Reichmann (of Luminous Landscape) is enthusiastic about DNG!

----------

OpenRAW said on the page "The RAW Problem", up to end March 2006, "Many have suggested (and Adobe has created) a common, open file format for RAW image files for all camera makers to use as a solution to the RAW problem".

From April 2006 it omitted "(and Adobe has created)". In effect, this retracted their 11-month position that DNG is indeed a common, open file format.

----------

Just after OpenRAW redesigned their web site at the start of April 2006, they prominently published a large article "DNG is not the answer" by Stuart Nixon:
http://www.barrypearson.co.uk/articles/dng/assets/prominent.gif

This article was edited and renamed "Notes on the future of Open RAW formats, and a look at DNG" a few days later. The theme remained the same, including the words "DNG IS NOT THE ANSWER".
http://www.openraw.org/node/1482

That article has been much-quoted in forums, with no attempt at a retraction by OpenRAW or Stuart Nixon. It has promoted "fear, uncertainty, and doubt" about DNG, using the "OpenRAW" brand. In fact, it does not stand up to scrutiny:
http://www.barrypearson.co.uk/articles/dng/commentary1.htm

That article, and the way that OpenRAW CHOSE initially to display it in that manner, (Stuart Nixon couldn't have done that by himself), indicates the antipathy, perhaps hostility, of OpenRAW towards DNG.

----------

More recently, Juergen Specht has been quoted as responding to "Is the Adobe DNG format the pragmatic solution or maybe the same old song?":
http://www.ladinamo.org/english/raw-format-the-captive-photo.php

"I wish it would be a solution, but Adobe decided for marketing reasons that they allow a backdoor to camera makers to store information in so called private tags, which remain undocumented. So even if some cameras can save DNG formats natively, some of the information a camera decides to conceal can be saved inside the DNG format and it becomes another undocumented RAW format after all. Plus Adobe stopped documenting their PDF and PSD format after it reached a certain market share, so they can at any time release a DNG V2.0 format and decide not to document it. No, DNG is unfortunately not a solution".

That is simply "fear, uncertainty, and doubt":

  • The fully documented information is sufficient to render high quality images. (Typically, products that support DNG need only the fully documented parts of DNG files). If a camera manufacturer puts in private data, the DNG file becomes "a file with documented data capable of rendering a high quality image, plus some other stuff which may or may not be useful".
  • Adobe intends to release the full PDF 1.7 specification to AIIM for the purpose of publication by ISO. PDF for Archive (PDF/A) and PDF for Exchange (PDF/X) are already ISO standards, and PDF for Engineering (PDF/E) and PDF for Universal Access (PDF/UA) are proposed standards. Additionally, PDF for Healthcare (PDF/H) is an AIIM proposed Best Practice Guide.
  • TIFF has similar status to DNG - an openly specified file format owned by Adobe. Who worries about what Adobe may do to TIFF?
  • From the license: ""DNG Specification" means any version of the Adobe Digital Negative (DNG) Specification made publicly available by Adobe". If it isn't documented, it isn't a DNG specification!

----------

OpenRAW never had a balanced attitude towards DNG. Over time, its antipathy towards DNG has hardened, even while DNG has been making significant progress towards "Digital Image Preservation Through Open Documentation". Why?

Barry Pearson – Thu, 2007/02/08 – 6:57pm


I was banned!

(Juergen Specht) OpenRAW's rejection of Barry Pearson

I like this headline better ;)

Juergen Specht – Thu, 2007/02/08 – 11:25pm


Continuation in my absence

After banning me from the OpenRAW website, Juergen Specht has continued the "dialogue", without having to worry that I will correct him! He continues with the same fallacies.

The article below was triggered by another article: batsys – Sat, 2007/02/10 – 6:38am.


Partial response:

I am consistent. My position is that DNG is part of the solution to "Digital Image Preservation Through Open Documentation". (Not what he said). I never try to convince people that deleting original RAW's is safe. I help people to make informed decisions.

I've omitted the quote from Adobe's 2004-09-20 document. It doesn't contradict what I say, but it's showing its age! It was partly superseded by ACR 3.x and the 3.x DNG Converter which copied across lots more metadata.

He is obviously wrong to claim that there cannot be a lossless conversion when the metadata isn't documented. It can be copied as undocumented metadata!

"... comparable to your analog negative" obscures the difference between the raw file itself, and the raw image data within the file. It is the latter that he is exploiting years later, and he could do so from the DNG file instead.

(Juergen Specht) The whole topic is a can of worms, you are entirely correct about it

Barry is unfortunately not consistent with his comments, and his advise is dangerous, because he tries to convince people that the DNG format is the solution to the 'undocumented RAW formats' and that deleting your original RAW's is save.

However, if you look into the current Adobe DNG specs, you find this warning there:
[Quote from September 2004 snipped]
The source: Introducing the Digital Negative specification: Information for Manufacturers (PDF, Page 5)

So we have the same chicken/egg problem here also for DNG, it cannot convert all meta-data, because they are not documented, ergo there cannot be a lossless conversion from one file format to the other, if the data which should be converted is not understood.

To your other question, the RAW file is comparable to your analog negative. If you keep your negative for 10 years and then develop a print with the newest chemicals and papers, it will look way better than it was possible 10 years ago.

The same with digital RAW files. Since 7 years I store all my Nikon NEF files as 'straight from the camera' and as converted JPG files with the current RAW converter. If I today look back at the 7 year old JPG conversations and compare them with a todays conversion of an actual RAW file, I can squeeze way more detail out of the same RAW files than I was able 7 years ago.

So its good advise to keep your RAW files and NOT delete them, you will bite yourself if you value your photos and do so.

And note, even Adobe advises against it.

Juergen Specht – Sat, 2007/02/10 – 11:16am