Lightroom not seeing .dng output as unique

Announcement of new releases, bugs, support, suggestions
Post Reply
GracieAllen
Posts: 67
Joined: 08.04.2015 00:19

Lightroom not seeing .dng output as unique

Post by GracieAllen »

I’m hoping I have a preference incorrect or something wrong in my process…
Windows 10 Pro, Lightroom current, graphics drivers current, Helicon focus 7.6.4.

I take a set of images to render in Helicon Focus. I send the raw images using “original” for the export.
Render in method B with default 8,4, save the output as a .dng to Lightroom, STAY in Helicon and render the same set in method C, save as a .dng to Lightroom.

The two files have different names and SHOULD appear very different. When viewed in Faststone viewer, Adobe Bridge, Photoshop or even Affinity Photo, they DO… THEY LOOK, AND ARE, DIFFERENT.

However, when in Lightroom Classic Library , opened in Loupe view, they are the same. There is no visible difference.
If I save .tif or .jpg files, the difference is very visible.

Exporting from Lightroom as .dng instead of Nikon RAW images makes no difference.

Building 1:1 previews makes no difference.

DELETING the preview folder tree for the entire CATALOG, forcing Lightroom to rebuild the previews makes NO difference.

Rendering the FULL stack, then re-rendering a subset of the first several images MAKES NO DIFFERENCE. So, a 19 shot stack output and an 8 shot output of the first 8 images in the stack look THE SAME.
If, however, I render 8 images FROM THE MIDDLE of the stack, NOT including the first image in the stack, the outputs DO show up as different.

A painfully involved interaction on another forum (Lightroom), resulted in MANY suggestions but no solution.

I copied three output .dng files to my website so people can download them. On THEIR systems a couple people said they SEE the differences. I can provide the file links if anyone wants to download them, but the three .dng files are close to 400 MB.

I COPIED the output files to another computer and on THAT computer THEY ALSO LOOK EXACTLY THE SAME.

Is there a preference or setting in Helicon I can have wrong? If not, I’m out of ideas.
akiecker
Posts: 10
Joined: 19.05.2020 20:36

Re: Lightroom not seeing .dng output as unique

Post by akiecker »

I have experienced the EXACT SAME PROBLEM!

I have performed the following:

I rendered a 22 image stack with both methods B and C. I then unchecked 14 images, leaving 8 images checked. These 8 images were rendered with method C. The three rendered images are then saved to Lightroom. When I view all the images (grid view) they looked the same. On my second monitor they all looked the same (seemed to be method B, the first one that I saved from HF to LR). In loupe view they all looked the same.

I then used File Explorer to find the DNG files and double clicked (Note: I did NOT export them from LR, I just accessed the image directly from the hard drive are stored on). This brought them up in PhotoShop. They are all different as they should be. I then went back to LR and exported the images to PS and the CORRECT image was sent to PS!

So I tried one more thing. I removed them from the LR catalog, then imported them back in. Same thing, all look the same.

As another test, I did the following:

• Export 21 image stack to HF. Render with method B, saved to LR. Quit HF.
• Export same 21 image stack to HF. Render with method C, saved to LR. Quit HF.
• Export same 21 image stack to HF. Uncheck several images so that only use 8 images. Render with method C, saved to LR. Quit HF.

At this point I quite LR. Got an idea, so I opened LR and did this:
Export 8 images to HF. Render with method C, saved to LR. Quite HF.

I now have 4 images that have been rendered by HF and saved to LR. All 4 of the images are the same! PS shows the correct image when you export to PS.

Can we PLEASE have an explanation of what is happening!
akiecker
Posts: 10
Joined: 19.05.2020 20:36

Re: Lightroom not seeing .dng output as unique

Post by akiecker »

I just had an idea ... In Lightroom, I examined the image in the Develop module. I saw the correct image!!

I don't know what the format of a DNG file is, but lets assume that it contains an embedded JPEG that Lightroom is displaying in the grid and loupe views. If this is the case then it looks to me like Helicon Focus is putting the incorrect JPEG into the DNG file when it is saved to Lightroom!
GracieAllen
Posts: 67
Joined: 08.04.2015 00:19

Re: Lightroom not seeing .dng output as unique

Post by GracieAllen »

I've occasionally, PARTLY had the same experience in Develop... Moving from image to image, for a split second, the correct image displays, then is replaced by the image that's the same as the others.

This morning I opened two .dng files in Word. You can read the “header” xmp stuff. I took a 19 image stack and a 7 image subset and did a compare. They were done in TWO different sessions so I’d expect SOME differences. The ONLY three fields that differed were the date at the top, the xmp:ModifyDate and the xmp:MetadataDate. One is 19:44:09, the other is 19:45:30.

As near as I (and the compare) can tell, EVERYTHING ELSE in there is identical. In Lightroom they look the same.

I took a .NEF file, in Lightroom, changed the exposure to about -3 and exported as a .dng. Changed the exposure to about +4 and exported a .dng. Different names. Opened THOSE and compared:
The three dates are different (expected)… BUT, there are OTHER differences:
xmpMM:DocumentID
xmpMM:InstanceID
crs:Exposure2012
stEvt:InstanceID
stEvt:when

And when imported into Lightroom, they look VERY different.
The Exposure is -3.xx vs +4.xx so crs:Exposure2012 isn’t important. I suspect stEvt:when isn’t either.
xmpMM:DocumentID, xmpMM:InstanceID and stEvt:InstanceID ALL CONTAIN THE SAME VALUE.

Back to Helicon…

I ran a NEW stack. Ran 16 images in method B, then ran the first 4 images in method B. Saved each as it was created. Exited from Helicon then exported the first 8 images from Lightroom and ran them method B, and saved. So, it’s a different time, a different number of images, and all the filenames are different.

In Lightroom loupe, ALL THREE images look identical. Comparing the .dng files, the 16 image run and the 4 image subset are identical – as near as I can tell there are NO fields in the xmp text that differ. There’s nothing I know of to tell Lightroom these are anything but two identical files with different names.

Then I compared the 16 image and the SEPARATE 8 image subset renders. This time the three dates differed, and the xmpMM:DocumentID, xmpMM:InstanceID, stEvt:InstanceID, stEvt:when differed as they did in the two .dng files I created from the .NEF. Further down in the xmpMM:History section there are repeated occurrences of the stEvt:instanceID that repeat earlier values, but there are TWO different values of stEvt:instanceID in various places in the text… All differ from the values in the stack of the 8 image subset.

So, there is data in the .dng that SEEMS to be unique when I run a SEPARATE session, yet in Lightroom, ALL 3 .dng files, 16 images, 4 image subset, 8 image subset, all look the same.

What DOES appear to be the same in all of the .dng images from these runs is the xmpMM:OriginalDocumentID, and in the “xmpMMDerivedFrom” section, the stRef:documentID, and the stRef:originalDocumentID, which match the xmpMM:OriginalDocumentID earlier in the text.

What is LIGHTROOM using to decide there are two different files? If it’s InstanceID, that only appears to change if a different Helicon Focus session is run. But, if it’s stRef:documentID or stRef:originalDocumentID, those are the same in all 3 .dng files.

I took a subset of images from the middle of the stack, and ran those. All the above differ, PLUS the crs:RawFileName, aux:ImageNumber and the stRef:documentID, and stRef:originalDocumentID that were the same previously.

From all this fiddling around, I have more data, but have no idea what is and isn’t meaningful. The question “How does Lightroom determine two .dng files are the same (or different)?” hasn’t been answered. As far as I can tell, Helicon doesn’t do it’s own .dng conversion. It uses the Adobe .dng converter – when you install HF, it appears to install an Adobe .dng converter. So, is it Lightroom, Adobe dng converter, or Helicon that’s not doing what’s needed to make Lightroom treat the .dng files as unique?
GracieAllen
Posts: 67
Joined: 08.04.2015 00:19

Re: Lightroom not seeing .dng output as unique

Post by GracieAllen »

From a topic started in the Adobe Lightroom Queen forum I received the following:

Your sample Helicon .dngs have the same RawDataUniqueID, and the DNG specification says:
This tag contains a 16-byte unique identifier for the raw image data in the DNG file. DNG readers can use this tag to recognize a particular raw image, even if the file's name or the metadata contained in the file has been changed.

If a DNG writer creates such an identifier, it should do so using an algorithm that will ensure that it is very unlikely two different images will end up having the same identifier.

This is the proximate cause of this misbehavior. I used Exiftool to delete the RawDataUniqueID field from the three samples, and after reimporting them into my LR 10, they now show three different previews:
exiftool -rawdatauniqueid= *.dng

I'm not a .dng expert, but it sounds like the folks who posted this may be, so it appears that the problem may be that Lightroom is getting .dng files that are not compliant with the .dng standards?
User avatar
Catherine
Posts: 1163
Joined: 29.04.2019 22:38

Re: Lightroom not seeing .dng output as unique

Post by Catherine »

Thank you for findings and sharing this info and details. There is no setting in Helicon Focus that affects any of this, and it looks like we'll need to address this problem by updating the software. We're already looking into it and will post back as soon as there's any positive result.
Post Reply