The LEADTOOLS Distributed Computing SDK provides developers with a complete framework that can be used to create powerful task-based applications. Utilizing a network of servers and worker computers, developers can easily create any distributed, grid, or parallel computing application.
All of the necessary framework components for developing applications with the LEADTOOLS Distributed Computing SDK are included with the Main LEADTOOLS installation. The downloads and instructions below contain supplemental demos that highlight the features of the LEADTOOLS Distributed Computing SDK.
LEADTOOLS Job Processor Server
This setup will install all of the necessary server components. This includes:
- Job Processor Server Configuration Demo - The central server hosts the WCF service classes for communication between clients/workers and the central server. The Job Processor Configuration Demo can add a new virtual directory to host the WCF service classes, define worker computers, and automatically update configuration files.
- Job Processor Administrator Demo - The Administrator Demo provides common administrative features such as a real-time status of all workers, control of jobs on each worker, and search for jobs in the database based on specified criteria.
- Job Processor Client Demos - The Job Processor Client Demos are ASP.NET Web Forms projects that may be used as job processor clients. With these demos, clients may view, add, abort, and reset jobs. Once jobs have been processed, the results may be downloaded by the client.
To set up the LEADTOOLS Job Processor Server demonstration, follow these steps:
- Download and install the LEADTOOLS Job Processor Server Setup.
Once installed, run the Job Processor Server Configuration Demo:
(\Shortcuts\IIS Config Demo\Job Processor Server Config 64-bit).
Next, IIS must be configured to host the WCF service classes. To create the application in IIS, click the 'Create' Button on the 'Main' tab. The IIS application will be created and point to the 'LEADTOOLS Job Processor Server 19\JobProcessorServices' folder. If you encounter any errors, you may not have the necessary IIS features enabled. Click the 'Troubleshoot' button under the 'Help' tab for more information on the required IIS features.
In most cases, there will be a central location used to store jobs. This could be a location on the web, local drive, network share, etc. For demonstration purposes, we will use a network share on the central server that all workers can access for read and write. During the creation of the IIS application, you may be prompted about this share. If you choose to not allow the creation of the share, you can manually create one at a later time. If you wait and manually create the share, be sure to update the web.config file of the web demos with the correct path (see below). You will also want to copy your source files into the path pointed to by
Once the virtual directories are created, the address to the WCF server will automatically be updated in the web demo web.config file. You can test both the WCF service and web demo virtual directories by clicking the 'Test' button on the 'Job Processor Properties and Test' tab.
<!--Location where all input files are stored on the server. These files are enumerated and shown in the web demo --> <setting name="InputFilesUrl" serializeAs="String"> <value>\\TEST1\JobProcessorFiles\Input</value> </setting> <!--Location where all output files are stored on the server. When jobs are processed, the web demo will allow users to download the converted file. --> <setting name="OutputFilesUrl" serializeAs="String"> <value>\\TEST1\JobProcessorFiles\Output</value> </setting>
- The next step is to add the LEADTOOLS Job Processor database to a SQL server instance and will require SQL server 2005 or newer be installed. To add the Job Processor database, click the 'Configure' button in the 'Database Configuration' section of the 'Main' tab. The utility will automatically update the connection string setting of the WCF and administrator configuration files.
The next step is to add workers. Workers are defined in Workers.xml which can be found in the App_Data directory of the WCF service. For each worker defined, you can specify:
- The frequency this worker should check for new jobs
- The types of jobs this worker should accept
- The maximum number of concurrent jobs this worker should accept
- The CPU threshold this worker must have to accept to jobs
- Whether the CPU threshold or maximum job setting should be used to determine how many concurrent jobs are processed
- The frequency this worker will report progress for each job
- The maximum amount of time allowed without a progress updated before a job is considered 'hung'
- The maximum number of times to attempt each job
Now you have successfully installed all of the necessary server components.
LEADTOOLS Job Processor Worker (LEADTOOLS Job Processor OCR Worker, LEADTOOLS Job Processor Multimedia Worker)
This setup will install all of the required worker components. This includes:
- Worker Windows service - The worker Windows service is responsible for polling the server for new jobs and accordingly process those jobs.
To set up the LEADTOOLS Job Processor Worker demonstration, follow these steps:
- Download and install the LEADTOOLS Job Processor Worker Setup. To create an OCR worker, use the LEADTOOLS Job Processor OCR Worker. To create a multimedia conversion worker, use the LEADTOOLS Job Processor Multimedia Worker During installation, you will be asked to enter the IP address or machine name of the computer that has the server components installed. Each worker needs to communicate with the server via WCF web services to retrieve new jobs, report job status, etc. The installation will update the worker configuration files (Bin\Dotnet\<Win32 or x64>\Leadtools.JobProcessor.WorkerService.exe.config) with the address or name of the server.
- The installation will install a single Windows service to handle OCR (LEADTOOLSOcrJobProcessorService) and Multimedia (LEADTOOLSMultimediaJobProcessorService) jobs. If the service is not already running, start the service from the services console (services.msc) in administrator tools.
At this point, you are ready to watch the LEADTOOLS Distributed Computing SDK work! From any computer that can access the server, you can browse to the web demo previously installed on the server. The default page will ask for a username which is used to associate users and jobs. To add a job from the web application, select a source file (the files are enumerated from the 'InputFilesUrl' directory on the server), select a target format and click 'Add Job'. All jobs submitted by the same user will be shown in the grid which is updated every few seconds. Once jobs are successfully converted, click the 'Output File' link to download the file. Other options include job deletion, cancellation, and reset.
The administrator demo (installed on the server in \Shortcuts\Administrator Demo\Job Processor Administrator 32-bit) may be used to view the real-time state of each worker, start/stop job processor services on each worker, and query jobs in the database. Jobs listed in the grid have an associated context menu to delete or reset the job. If a critical error occurs (any machine in the Job Processor framework loses power, network failure, etc), any job with a status of 'Queried' or 'Started' could be left in an unstable state. The 'Analyze Database' feature can be used in this case to locate these jobs and ensure their status is appropriately updated (status reset to 'New'). All worker machine services should be stopped before using this feature to ensure jobs are not double processed.