Helicon Focus 7.6.6 cross-output retouching and other code defects

Announcement of new releases, bugs, support, suggestions
Post Reply
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by umbel »

Hi, hit another project bug on the fourth or fifth stack I've attempted with 7.6.6. Problem is the method B output isn't coming available for retouching the method C stack and vice versa so something's preventing Focus from seeing the stacks as compatible. I hit similar trouble occasionally in 7.6.1 but this is the first stack with a repro which persists for me on a clean restacking after restarting Focus. The "Use another output as a source" dropdown should be populated for both outputs but instead it just has a greyed out "no compatible outputs".

Frames and log file are uploading to the ftp server while I'm typing this (directory is P1080486) with another half hour or so to go. The .log is weird since it only has the second, method C, stack and not the preceding method B stack, so perhaps there's a clue there. I saved out .hproj files and uploaded those too.
Last edited by umbel on 22.02.2021 23:01, edited 2 times in total.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Helicon Focus 7.6.6 cross-output retouching

Post by umbel »

Hit this same issue with cross-output retouching on a few more stacks, so tried ditching 7.6.6 and rolled back to 7.6.4. Unfortunately this issue also reproduces on 7.6.4. I've uploaded a second set of frames to the ftp server as well.

So far, I've identified two mitigations.
  1. Disable adjust brightness (Edit -> Preferences -> Autoadjustments).
  2. Keep the rotational alignment allowance at its default of zero even on images where it may be beneficial.
Since the optimal alignment frames is presumably a property of only the frames, and therefore constant across different choices in stacking methods, it's unclear to me why rotational alignment would influence the availability of outputs as retouching sources. My expectation is the alignment would be calculated once and then reused across different choices of stacking options but, given the large number of issues encountered in this space, perhaps this doesn't occur, perhaps numerical tolerances aren't considered, or perhaps there is an issue in creating a transform aligning the outputs to each other.

It seems more natural that optimal brightness transforms might vary with stacking parameters since, for example, the contrast increase associated with pyramid stacking would favor increased highlight protection. However, it's unclear why this would affect retouching availability of outputs. The purpose of cross-output retouching is, after all, to merge differences between outputs.
Last edited by umbel on 22.02.2021 23:01, edited 2 times in total.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by umbel »

Hi Catherine, any update? Been over a month since the issue was reported and I keep hitting more failures to link outputs for retouching even with rotation and brightness adjustment disabled. Since an additional workaround appears to be disabling OpenCL it would seem there are perhaps some issues with numerical stability on that codepath in addition to the issues around project design. This is problematic since the purpose of OpenCL support is to allow faster stacking than with just the CPU. But if you have to do two OpenCL stacks with different methods to discover cross output retouching won't work for the stack, disable OpenCL, restart Helicon, and then restack with CPU only that's considerably slower than just doing the two stacks CPU only begin with.

Additionally, Focus ignores two finger zooms and three finger touchpad presses for middle button clicks on one of my machines when the stylus is within range of the USB tablet I use. This unnecessarily complicates retouching as the stylus has to be lifted out of range to use zoom or jump to 100%. Other programs, such as GIMP, aren't affected by the issue and neither is Focus on my other machine with the same tablet, so this behavior appears to be due to event filtering within Focus that's handling concurrent input devices inconsistently. It might be related to the touchpad using Windows 10's precision touchpad support.

There's also an issue with 7.6.6 getting stuck near the end of stacking. Instead of the progress bar jumping the last bit to completion link normal, a single CPU core remains active apparently indefinitely (the GPU's idle). I've only hit this once (with method B) but it suggests exit from some loop near the end of of the stacking process is not entirely reliable.
Last edited by umbel on 22.02.2021 23:02, edited 1 time in total.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Helicon Focus 7.6.6 cross-output retouching

Post by umbel »

Another variation of this issue: with OpenCL and rotational alignment some outputs don't get enabled as retouching sources unless automatic output cropping is also disabled. However, since method B and method C are somehow producing substantially different alignments from the exact same set of input frames, the sizes of the outputs differ by enough to defeat the purpose of cross-output retouching. While I can circumvent the cross-output block in this case by disabling or restricting alignment scaling this is not helpful as cross-output retouching only remains enabled when the scaling allowance is low enough that mismatched outputs result. Allowing sufficient scaling to match the outputs results in cross-output retouching switching to disabled when it would become useful. This suggests further design issues within the alignment process and indicates output-to-output registration defects resulting in the opposite of desirable cross-output retouching behavior.

The workaround for this Helicon Focus defect is thus Zerene.

For completeness, in some cases an additional thing to try within Helicon method A as it can pair with method B or C for cross-output retouching even when B and C outputs fail to pair as expected. For the images I work with A consistently performs enough worse than B that this hasn't proved helpful in the few cases I've encountered so far where A has paired. But maybe it'll help someone else with different kinds of images or help with implementing a fix.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Re: Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by umbel »

Just a bump to note I continue to hit images where outputs aren't compatible for retouching. In some cases the misalignment between method B and method C stacking is so large it's obvious when toggling between the outputs since the image jumps in the output pane.

I've developed a workflow which falls back to Zerene when Helicon fails. It's less than ideal due to Zerene's slowness but so far Zerene has always produced compatible outputs in these cases. Net result it's faster and definitely easier than trying to use F9 retouching to compensate for Helicon's failures to produce compatible outputs.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Re: Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by umbel »

I've been monitoring Helicon's success rate in creating compatible outputs between methods B and C and it's been only in the 20-33% range on recent stacks. There is no apparent correlation with the subject of the stack, lenses used, or number of frames in the stack. These are all on tripod, autofocus based stacks without subject motion. So there's nothing unusual or demanding in their alignment requirements.

However, in the cases I've checked so far, moving stacks which fail to produce compatible outputs on a quad core processor back to an older dual core processor results in the creation of compatible outputs even though nothing's changed in the input stack or Helicon's alignment settings. OpenCL is disabled and both processors I've tested on implement AVX2 but not AVX512. So Helicon 7.6.6 is presumably using the same instruction set extensions in both cases. This suggests there is an alignment code defect which is dual core latent and quad core active. Since the core throttle previously suggested hasn't been implemented there doesn't appear to be a way for users to confirm or work around. The behavior observed so far as been wholly deterministic, so it seems most likely a task scheduling defect rather than a race condition.

With at least some of the stacks the choice of front to back versus back to front frame ordering differs between methods B and C on quad core but is identical on dual core. Since Helicon lacks other stackers' ability for users to control the frame ordering it doesn't appear users have a way of forcing matching ordering to see if that helps. One component of this issue may be the checking process for output compatibility isn't able to recognize alignments which are in opposing orders but otherwise identical. However, that still leaves the cases where visibly different alignments are produced from identical sets of frames.

I hope someone at Helicon will have look at the stacks uploaded 2-3 months ago when this thread started and implement a fix allowing Helicon to work properly on current processors and GPUs. A fast stacker requiring older hardware be used for what seems to be the most common form of retouching for deeper stacks does leave something to be desired.
User avatar
Catherine
Posts: 1163
Joined: 29.04.2019 22:38

Re: Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by Catherine »

umbel wrote: 04.03.2021 04:58 Just a bump to note I continue to hit images where outputs aren't compatible for retouching.
We'll be looking into this, but I don't think we've actually managed to reproduce this problem yet (I'll check with my colleagues and will post an update later). It could be that you're doing something specific that triggers the issue that we didn't find during testing.
User avatar
Catherine
Posts: 1163
Joined: 29.04.2019 22:38

Re: Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by Catherine »

umbel wrote: 22.03.2021 19:39This suggests there is an alignment code defect which is dual core latent and quad core active.
1. You could be right on that, but that seems highly unlikely. Much more likely is that it's purely deterministic, doesn't depend on the CPU, but depends on the Helicon Focus settings. Since you're testing on different machines, it's quite possible that some setting differs and causes this.
A simple test: clear the settings completely on both machines (Open preferences, press Ctrl + Alt + Shift + W and proceed as prompted). Then run the same stack with the same method an parameters, and see if there's a difference now - including the sorting difference, there shouldn't be any since sorting is deterministic as well.

P. S. Before clearing the settings, send a bug report (menu - Help - Report a bug) from each of the two computers. That way we'll get a copy of your current settings on each computer that we could then use if it turns out that the problem is indeed related to your particular settings.

2. You can very well control the sorting behavior: right-click any one of the source images, un-check "Sort automatically", and then you can choose between ascending and descending sorting.

3. You can force Helicon Focus to only use two cores on the the quad-core computer: open Helicon Focus, then open Task Manager. Go to Details, right-click HeliconFocus.exe - Set affinity, and only select two cores. Press OK. Now run a render and it will only utilize two cores.
Although this is not a perfect simulation of a 2-core CPU operation because Helicon Focus will still see the CPU as quad-core and will spawn 4 threads, which will then be scheduled to only 2 cores by Windows. But as I said, so far this seems a less likely scenario that's not worth putting time into just yet. Instead, it would be great if you could test my suggestion #1.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Re: Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by umbel »

Hi Catherine! After going through Ctrl+Alt+Shift+W, manually setting the sorting order, and changing affinities it doesn't look like any of those are the issue. What the logs show is messages like
0.099: Abnormal magnification 1.00094 detected at image: 1 "P1090750-006.jpg" new order: Qt::DescendingOrder
0.348: Abnormal magnification 1.00063 detected at image: 1 "P1090750-044.jpg" new order: Qt::AscendingOrder
which presumably explains why the image sort order changes between the method B and method C stacks.

Based on nearly two hours' experimenting, there appear to be two workarounds:
  1. Exactly repeating the broken stack. For example if stacking with methods B+C produces incompatible outputs continuing to B+C+B by saying yes to "Render task again with same parameters?" results in the second B output being compatible with the C output. This enables cross-output retouching.
  2. Removing additional frames at the end of the stack to avoid the abnormal magnification entries in the log. Given a camera body whose autofocus bracketing implementation reverses directions at the ends of the stack, for certain magnification shifts just as the direction is changing, the changes are too small to be visible because they are less than a pixel and round to zero. Since current workflow requires manually trimming these frames at the ends of the stack, the rounding means a human isn't always going to be as precise as Helicon expects.
My quick mitigation suggestion would be to put a warning icon on outputs where abnormal magnifications are detected with hovertext indicating which frames reverse magnification. As far as I can tell Helicon's behavior is numerically correct but, with some of the stacks, I can't see which way the magnification goes between the frames even when toggling between their corners at 200%. So having some machine assistance with detecting the issue would be helpful (particularly as Zerene and Picolay are insensitive). Incremental structural improvements would include sharing alignments between outputs using the same frames, considering more information when determining whether to reverse a previously chosen frame order, and automating bracket trimming.

I would like to check this behavior with OpenCL as well but the Intel UHD Graphics 620 on the quad core are no longer seen as an OpenCL device. How do I get it to reappear in the performance tab of the preferences dialog? I did a complete uninstall and reinstall, including removing registry keys and re-registering 7.6.6. C:\WINDOWS\{system32, SysWow64}\OpenCL.dll appear in the adapter's driver files list and it's reported by clinfo for versions 1.0-3.0, so it's definitely OpenCL enabled.

For a given stack, the magnifications reported in the two machines' abnormal magnification log entries are identical so I can't explain the machine to machine differences in behavior. I'll monitor and see what additional issues I can turn up, if any, since if it was just abnormal magnifications, then disabling OpenCL, turning off brightness matching, and restricting certain alignment transforms shouldn't have produced improvement. The behavior with repeating a broken stack might explain why it looked like brightness and alignment made a difference. But definitely not OpenCL since changing that requires restarting Helicon.
User avatar
Catherine
Posts: 1163
Joined: 29.04.2019 22:38

Re: Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by Catherine »

Hi! Good job investigating the problem, it makes perfect sense now.
Indeed, the stacks must be rendered in the same direction to be compatible for retouching. The simplest fix is to un-check "Sort automatically". Then you can pick the direction that works best for your particular stack(s).

The preferred direction for auto mode is always computed on the relative magnification between the first and the second images in the stack. Increasing magnification is preferred because the alignment is calculated based on the first image of the stack, and it's better if the image is growing outwards. If it's shrinking inwards you will end up with the region of incomplete information at the perimeter of the final image and it will have to be cropped out - which Focus does automatically, but this means that the resolution of your final image will be slightly less.

If your stack is not sorted consistently (the ordering of file names does not match the ordering by focal plane / magnification) in such a way that the first two images on both ends are in the decreasing (unfavorable) order, then the auto function will switch direction every time you press "Render". Which, again, you should combat either by cleaning up your stack before hand (renaming the files or removing the redundant once at the ends), or by disabling automatic sorting. Maybe this is what was causing the variation you've been observing.

Helicon Focus should be no more sensitive to file ordering than Zerene and Picolay - especially with the C method, which you should prefer for inconsistently sorted stacks.

As for the OpenCL - that's quite weird, we've never heard of such problems before, and I've just tested 7.6.6 on Windows 10 with Intel UHD 630 - no problems. Please send a bug report from Helicon Focus (menu - Help - Report a bug), that might give us some insight.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Re: Helicon Focus 7.6.6 cross-output retouching and other code defects

Post by umbel »

Er, except stacks don't actually have to be rendered in the same direction to be marked as compatible. Because automatic alignment flips the direction for each new output, doing method B+C+B produces two B outputs having the same direction, a C output having the opposite direction, and the last B and C are seen as compatible for retouching. The opposite directions mean probably there is some difference in their alignments and retouching won't work very well, but Helicon is thinking they are compatible even though it decided the C and the first B are incompatible. This is a variation on other forms of incorrect output compatibility detection I mentioned earlier in the thread. So there are bugs in the compatibility detection logic.

The stack is sorted consistently. It is, after all, an autofocus bracket. I'd have to manually renumber the frames to put them out of order. The difference is Helicon has strong reactions to minor deviation in the first frame where the other stackers are relaxed about it. For the example plotted below, the magnification reversal results in a 0.65 pixel stretch at the extreme corners of a single frame. Which, for most stacks (I'm tempted to say basically all of them), will have negligible effect on the depth map.
P1090750B alignment.png
P1090750B alignment.png (11.27 KiB) Viewed 1151 times
This is why I suggested making automatic sorting more robust by considering the balance of evidence and not flipping orderings between outputs using the same frames. I mean, I know exactly what I'm looking for between the first two frames and there's no way to see it at 100% as the only bit that's in focus is halfway between the center and corner. It is therefore moving only a third of a pixel on the LCD.

Sure, I now know how to diagnose, avoid, and work around this and that helps people who happen to find and read this thread. But probably not many will and I would be surprised if many Helicon users have the ability to measure frame scaling. Since the code paths already exist to display a warning on unexpected scalings of, say, a couple pixels and errors on larger ones the possibility of more user friendly behavior seemed worth mentioning.

Sent in a bug report on OpenCL detection. Definitely agree it's weird as the initial install of 7.6.4 on the machine found the 620 graphics but doing a complete uninstall and reinstall to either 7.6.4 or 7.6.6 now fails to display it the dropdown. Thanks for double checking on the 630.
Post Reply