understanding performance benchmark results

Announcement of new releases, bugs, support, suggestions
Post Reply
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

understanding performance benchmark results

Post by umbel »

Hi, I've a workstation purchase coming up and possibly a laptop as well. Stacking performance on these machines is a consideration so I've been looking at CPU and GPU entries in the performance table (https://www.heliconsoft.com/helicon_focus_benchmark/table_view.php), running the built in benchmark (Edit -> Preferences -> Performance), and timing some of my own stacks.

I'm unable to replicate the built in benchmark with my own stacks on the hardware I currently have. For example, the machine I just tested reports 24 MP/s from the built in benchmark but the two stacking passes come in around 18.5 MP/s and 11.5 MP/s, respectively, for an overall speed near 7.1 MP/s using method B. I presume the first pass is alignment and excluded from the benchmark, which results in the 12 MP/s of the second pass rather than the 24 MP/s benchmark. The machine has GB of memory free and SSD activity is negligible both during my stacking and the builtin benchmark, both tests keep CPU at 99-100%, and the results are repeatable within 1%. So the discrepancy doesn't appear to be attributable to anything external to the stacking process. (I also checked C:\Program Files\Helicon Software\Helicon Focus 7 to see if source files for the builtin benchmark might be available to run like a normal stack but it appears the 100 images are purely in memory.)

While I wouldn't expect an exact match between benchmark and any particular stack, a factor of two is a substantial discrepancy. So I was wondering if mismatches of this size between actual stack and benchmark results are expected and if the causes might be known.

I'm also curious if GPU use accelerates alignment and method C the way it does method B. But I haven't, as yet, been able to locate a machine with a supported GPU to test on.
User avatar
Catherine
Posts: 1163
Joined: 29.04.2019 22:38

Re: understanding performance benchmark results

Post by Catherine »

Such a discrepancy is not unexpected. First of all, the benchmark only times pure processing, while the actual rendering of an actual stack also includes reading the source images and unpacking them, so it's not an apples to apples comparison. Also, you have to process the images of the same resolution because the processing time does not scale exactly linearly with the picture size in megapixels. It's close, but not exactly, so 2 x 50 MP images will not be processed in the same time as 5 x 20 MP images.

Yes, method C is also GPU-accelerated.
User avatar
umbel
Posts: 34
Joined: 12.12.2019 08:26

Re: understanding performance benchmark results

Post by umbel »

Hi Catherine, thank you. I'm not set up to test across picture size but, having measured a few more stacks, I think I can explain some of the other differences. The largest source of variation appears to be the stack itself, though I haven't (yet) done enough measurements to make a regression on magnification and departure from telecentricity (presumably both measures of alignment cost) versus differences in the image content. In addition to the stack mentioned above that's well below benchmark, I've found stacks which are faster than benchmark with loading and alignment included.

On my current primary system (Haswell i5, EVO SSD),
  • Repeating a stack is 15% faster than performing the stack the first time. Since the first stacking populates the memory cache this suggests the cost of reading and unpacking the source images can be measured as the difference between first and repeat stacking performance.
  • Displaying intermediary results creates a 39% slowdown on a first stack and about 45% for restacking. Both are higher than the 30% indicated in Edit -> Preferences -> General.
Combined, these two effects create in a factor of two difference in stacking speed.

I also get a near exact O(N) for the number of frames in a stack, which is probably expected.
Jamesaddicted
Posts: 2
Joined: 24.12.2020 18:41

Re: understanding performance benchmark results

Post by Jamesaddicted »

because it starts to consume so much memory, I had 45 GB free and after doing tests all the space on my hard drive was consumed, I could find the cache folder and delete all the tif, now I have only 25 GB !!! where are my other 20 gb disappear by Helicon Focus
Post Reply