Guidance for massive batch processing
Posted: 17.07.2016 23:35
I love that Helicon has a Batch process feature, but I may be asking a little too much of it and I'm experiencing some issues in return.
I'm attempting to batch-stack 1500 folders, each of which contain 6 images to stack. In theory, the batch UI should be able to do this. In practice, here are the issues I'm encountering.
1) On one of my machines, the UI hangs and never recovers when I try to add these folders to the batch stack list. On another machine, the UI freezes up for a minute but finally recovers. If I divide the folders into sets of, say, 200, I can get it to work. But that kinda sucks.
2) Helicon keeps filling my hard drive and then failing. It looks like the cache is only disposed when Helicon is closed - not once a stack is complete. As a result, I ended up generating a few hundred GB in cached files, filled up c:\ and stopped. Unfortunately, the UI doesn't provide a good way to recover the stacks that failed. I have to figure out the last stacked image I got and go reload the job starting from there.
3) The Multiple Folder picker UI lets me select all (Ctrl + a) or lets me click-select multiple folders, but it doesn't let me shift-select a continuous range of folders. So, when I discover that folders 1-148 out of 1500 stacked correctly, I don't have an easy way to go select 149-1500 without changing my folder structure on the disk.
4) The job progress window is pretty bad. By default, it scrolls to the bottom, but the first job to run is at the top. It doesn't provide any overall progress bar of (Job 18 of 1500), time elapsed, or time remaining. If you were batch stacking 4 folders, this wouldn't be a problem. But, since my stacking jobs will routinely run for 8 hours or more, it'd be a nice feature to get some indication of time remaining.
Again, I'd assume I'm pushing the batch stacking functionality way past its intended use, but hopefully the feedback is helpful.
Ideally, I'd love to run batch stacking through a command-line interface that I can control via PowerShell or some other scripting interface. That way, I can control my workflow on top of the stacking process to, for example, automatically generate stacks with different stacking methods and file output types, automatically start stacking watch folders, notify me on errors, etc.
By any chance, does Helicon offer a type of command line interface like what I described?
Thanks in advance! Every time I use your software I'm still amazed by the magic of focus stacking! (and THANK YOU for DNG output!)
I'm attempting to batch-stack 1500 folders, each of which contain 6 images to stack. In theory, the batch UI should be able to do this. In practice, here are the issues I'm encountering.
1) On one of my machines, the UI hangs and never recovers when I try to add these folders to the batch stack list. On another machine, the UI freezes up for a minute but finally recovers. If I divide the folders into sets of, say, 200, I can get it to work. But that kinda sucks.
2) Helicon keeps filling my hard drive and then failing. It looks like the cache is only disposed when Helicon is closed - not once a stack is complete. As a result, I ended up generating a few hundred GB in cached files, filled up c:\ and stopped. Unfortunately, the UI doesn't provide a good way to recover the stacks that failed. I have to figure out the last stacked image I got and go reload the job starting from there.
3) The Multiple Folder picker UI lets me select all (Ctrl + a) or lets me click-select multiple folders, but it doesn't let me shift-select a continuous range of folders. So, when I discover that folders 1-148 out of 1500 stacked correctly, I don't have an easy way to go select 149-1500 without changing my folder structure on the disk.
4) The job progress window is pretty bad. By default, it scrolls to the bottom, but the first job to run is at the top. It doesn't provide any overall progress bar of (Job 18 of 1500), time elapsed, or time remaining. If you were batch stacking 4 folders, this wouldn't be a problem. But, since my stacking jobs will routinely run for 8 hours or more, it'd be a nice feature to get some indication of time remaining.
Again, I'd assume I'm pushing the batch stacking functionality way past its intended use, but hopefully the feedback is helpful.
Ideally, I'd love to run batch stacking through a command-line interface that I can control via PowerShell or some other scripting interface. That way, I can control my workflow on top of the stacking process to, for example, automatically generate stacks with different stacking methods and file output types, automatically start stacking watch folders, notify me on errors, etc.
By any chance, does Helicon offer a type of command line interface like what I described?
Thanks in advance! Every time I use your software I'm still amazed by the magic of focus stacking! (and THANK YOU for DNG output!)