Heap size larger than expected

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
TedBaker
Posts: 46
Joined: 2017-10-10T14:14:55-07:00
Authentication code: 1151

Heap size larger than expected

Post by TedBaker » 2017-11-24T16:23:42-07:00

I have the following 2.3905GB grayscale file.

Code: Select all

$ identify n10_757_a_full4800_exp1_001.tif 
n10_757_a_full4800_exp1_001.tif TIFF 27360x43680 27360x43680+0+0 16-bit Grayscale Gray 2.3905GB 0.010u 0:00.000
a simple rotate 90 results seems to require 8.9G?

Code: Select all

convert -define registry:temporary-path=. -debug cache n10_757_a_full4800_exp1_001.tif -rotate 90 test.tif
2017-11-24T22:19:43+00:00 0:00.000 0.000u 6.9.7 Cache convert-im6.q16[16740]: cache.c/DestroyPixelCache/1123/Cache
  destroy 
2017-11-24T22:19:43+00:00 0:00.000 0.000u 6.9.7 Cache convert-im6.q16[16740]: cache.c/DestroyPixelCache/1123/Cache
  destroy 
2017-11-24T22:19:43+00:00 0:00.000 0.000u 6.9.7 Cache convert-im6.q16[16740]: cache.c/OpenPixelCache/3873/Cache
  open n10_757_a_full4800_exp1_001.tif[0] (Heap Memory, 27360x43680 8.9041GiB)
2017-11-24T22:19:48+00:00 0:04.550 4.440u 6.9.7 Cache convert-im6.q16[16740]: cache.c/SetPixelCacheExtent/3737/Cache
  extend n10_757_a_full4800_exp1_001.tif[0] (./magick-167400e1kgOGdG18u[3], disk, 9.5607GB)
2017-11-24T22:19:48+00:00 0:04.550 4.440u 6.9.7 Cache convert-im6.q16[16740]: cache.c/OpenPixelCache/4022/Cache
  open n10_757_a_full4800_exp1_001.tif[0] (./magick-167400e1kgOGdG18u[-1], Map, 43680x27360 8.9041GiB)
2017-11-24T22:20:40+00:00 0:56.410 25.240u 6.9.7 Cache convert-im6.q16[16740]: cache.c/DestroyPixelCache/1123/Cache
  destroy n10_757_a_full4800_exp1_001.tif[0]
2017-11-24T22:21:13+00:00 1:30.070 41.670u 6.9.7 Cache convert-im6.q16[16740]: cache.c/DestroyPixelCache/1123/Cache
  destroy n10_757_a_full4800_exp1_001.tif[0]
Is this because IM allocates room for three channels? A color file with the same number of pixels using the same heap size.

If so Is there any parameters to reduce the heap size if only one channel is in use?

User avatar
dlemstra
Posts: 1434
Joined: 2013-05-04T15:28:54-07:00
Authentication code: 6789
Contact:

Re: Heap size larger than expected

Post by dlemstra » 2017-11-24T16:32:30-07:00

ImageMagick 6 will allocate room for 5 channels. You might get a lower memory footprint when you switch to ImageMagick 7.

p.s. The size on disk will not be the same amount of memory when decoded because the pixels are compressed.
.NET + ImageMagick = Magick.NET https://github.com/dlemstra/Magick.NET, @MagickNET, Donate

TedBaker
Posts: 46
Joined: 2017-10-10T14:14:55-07:00
Authentication code: 1151

Re: Heap size larger than expected

Post by TedBaker » 2017-11-24T19:26:55-07:00

Thanks, in which case it seems to make sense to increase the memory limit beyond the physical ram, I have 12G, and

identify -list resource shows Memory: 11.445GiB, changed them convert command too:

convert -limit 20G infile outfile

and no need to for any temporary files.

User avatar
fmw42
Posts: 22988
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Heap size larger than expected

Post by fmw42 » 2017-11-24T20:03:16-07:00

As dlemstra mentioned, IM 6 will need memory for 5 channels even for a gray level image. But in IM 7 a gray level image only needs memory for one channel. So you might consider upgrading to IM 7.

TedBaker
Posts: 46
Joined: 2017-10-10T14:14:55-07:00
Authentication code: 1151

Re: Heap size larger than expected

Post by TedBaker » 2017-11-25T12:24:15-07:00

Thanks fred, I will upgrade at some point. My follow up was just offered as 'way around' the 'issue' by just allocating more memory, than ideal, for grayscale images only.

I assume the default limit for memory is set at the physical limit, by increasing it to some sensible amount above the amount of RAM, you can avoid the use of disk, as I assume the pages for the 4 unused channels are never or hardly actually used.

User avatar
fmw42
Posts: 22988
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Heap size larger than expected

Post by fmw42 » 2017-11-25T13:45:59-07:00

For a grayscale to grayscale processing in IM 6, you will need to allocate at least RAM for 10 times the image size (5 channels for the input and 5 channels for the output), plus any clones or mpr: copies for intermediate processing. In IM 7 you would only need 2 times the image size (1 for input and 1 for output), etc.

Post Reply