Welcome Guest! To enable all features, please Login or Register.

Notification

Icon
Error

Options
View
Last Go to last post Unread Go to first unread post
#1 Posted : Monday, June 17, 2019 9:47:28 AM(UTC)
ramvn3003

Groups: Registered
Posts: 3


We are running the following code in Linux container in a cloud cluster. Using v20.0.0.4 Leadtools. Developed using C#.

Our container is frequently crashing due to resource issues. After going though Leadtools site I got to know about this memory threshold setting options and Diskmemory. Following is our code after I included those changes. Section inside 'Newly added to get rid of memory issue", I recently added.

Is this correct? Will it help fixing the issues? Please give your valuable suggestions on this.

using (var codecs = new RasterCodecs())
{
/* Newly added to get rid of memory issue
RasterMemoryThreshold rasterMemoryThreshold = RasterDefaults.MemoryThreshold;
rasterMemoryThreshold.MaximumGlobalRasterImageMemory = 20 * 1000 * 1024;
RasterDefaults.MemoryThreshold = rasterMemoryThreshold;
RasterDefaults.TempFileMode = LeadTempFileMode.Auto;
codecs.Options.Load.DiskMemory = true;
Newly added to get rid of memory issue*/

codecs.ThrowExceptionsOnInvalidImages = true;
codecs.Options.Txt.Load.Enabled = true;
codecs.Options.Load.Compressed = true;
codecs.Options.Load.AllPages = true;

if (settings.Resolution > 0)
{
codecs.Options.RasterizeDocument.Load.Resolution = settings.Resolution;
}
//using (ILeadStream leadStream = LeadStream.Factory.FromStream(file.OpenReadStream()))
using (RasterImage image = codecs.Load(inputStream))
{
pageCount = image.PageCount;
image.ApplyColorResolution(settings.BitsPerPixel, settings.ApplyGrayScale);
if (settings.Resolution > 0)
{
image.XResolution = settings.Resolution;
image.YResolution = settings.Resolution;
}
if (!settings.MergeImages && image.PageCount > 1)
{
using (var zipArchive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
//save each page as individual tiff
for (int pageNumber = 1; pageNumber <= pageCount; pageNumber++)
{
var zipEntry = zipArchive.CreateEntry($"{fileNameWithoutExt}_{pageNumber}.{targetFileExt}", CompressionLevel.Fastest);

using (var zipStream = zipEntry.Open())
codecs.Save(image, zipStream, settings.RasterImageFormat, settings.BitsPerPixel, pageNumber, pageNumber, 1, CodecsSavePageMode.Overwrite);
}
}
}
else
{
//save as multipage tiff
codecs.Save(image, outStream, settings.RasterImageFormat, settings.BitsPerPixel, 1, -1, 1, CodecsSavePageMode.Overwrite);
}
}
}
 

Try the latest version of LEADTOOLS for free for 60 days by downloading the evaluation: https://www.leadtools.com/downloads

Wanna join the discussion? Login to your LEADTOOLS Support accountor Register a new forum account.

#2 Posted : Wednesday, June 19, 2019 12:49:37 PM(UTC)
Josh Clark

Groups: Registered, Tech Support, Administrators
Posts: 30

Was thanked: 4 time(s) in 4 post(s)

These settings would help if your application is low on memory. You might also want to consider setting RasterDefaults.TempFileMode to LeadTempFileMode.Disk as this will make all temp files created be saved to the disk instead of to memory even if they are small files. LeadTempFileMode.Auto is the default which will only save temp files to the disk if they are large or tiled. If you are encountering crashes, this will help save files from being lost.
Josh Clark
Developer Support Engineer
LEAD Technologies, Inc.

LEAD Logo
 
#3 Posted : Wednesday, June 19, 2019 1:46:22 PM(UTC)
ramvn3003

Groups: Registered
Posts: 3


If I go for LeadTempFileMode.Disk, should I take care of removing the files from temp folder? I can see Leadtools automatically clearing the temp files getting created.
Using our service, we are downloading the stream from a blob store, do the conversion and uploading the converted file back. We don't have intentions to store it in disk. But the suggestion you gave here does not require any explicit IO operations to be added from our code, I think I can give it a try. Can you please confirm?

Another issue we are facing with PDF conversion is, for 24BPP, getting the following error consistently.
"You have exceeded the amount of memory allowed for RasterImage allocations.\nSee RasterDefaults::MemoryThreshold::MaximumGlobalRasterImageMemory"
 
#4 Posted : Friday, June 21, 2019 4:50:18 PM(UTC)
Josh Clark

Groups: Registered, Tech Support, Administrators
Posts: 30

Was thanked: 4 time(s) in 4 post(s)

The error that you are getting is due to restricting the maximumGlobalRasterImageMemory to 20 MB. Any file that you load that exceeds this limit that you set, will now instead throw an error ensuring that do not use more than what you specified. I would recommend expanding this to accommodate for larger files if you wish to process them. You can additionally set RasterCodecs.Options.Load.AllPages to false to only load one page at a time from a given document. You can then specify what page you would like in the load method as documented here:

https://www.leadtools.com/help/leadtools/v20/dh/co/rastercodecs-load(stream,int).html

Furthermore, if you are creating temporary files to disk, these are saved to the RasterDefaults.TemporaryDirectory. This will help to prevent you from losing data should your application or environment crash, as you can simply reload the temporary files. If you save your temporary files to memory, you will lose this data in the event of a crash or power outage. You can find documentation on the TemporaryDirectory here:

https://www.leadtools.com/help/leadtools/v20/dh/l/rasterdefaults-temporarydirectory.html
Josh Clark
Developer Support Engineer
LEAD Technologies, Inc.

LEAD Logo
 
#5 Posted : Monday, June 24, 2019 11:00:11 AM(UTC)
ramvn3003

Groups: Registered
Posts: 3


Ok. I will set the threshold to a higher allowed limit and try.
Loading all pages to false I will go through and make necessary changes. Will keep you posted on this.

Thanks
 
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Powered by YAF.NET | YAF.NET © 2003-2019, Yet Another Forum.NET
This page was generated in 0.250 seconds.