(ATS4-PLAT08) Server Job Pool Management

Steven Bush

R&D, AEP Core Infrastructure

steven.bush@accelrys.com

Job Pooling

• Each job works in a single scisvr process

– Isolated memory

– One bad protocol cannot crash the server

• Without pools, each job spawns a new process

• With pools, jobs with the same pool ID can reuse idle processes

– Ideal for quick running applications

Performance

• Prevent reloading system files and configuration data

• Reuse allocated memory

• Skip initialization

• Fast running protocols see substantial improvement

• Longer protocols do not see much improvement

Performance

Fast running protocol (0.1 seconds) 16 simultaneous clients against 8 core laptop

Performance

Longer running protocol (20 seconds) 16 simultaneous clients against 8 core laptop

Performance

ZOOMED: Longer running protocol (20 seconds) 16 simultaneous clients against 8 core laptop

Disadvantages

• Some components may not reinitialize themselves correctly – Can be difficult to track down these errors

• Stale resources can cause subsequent protocol failure – Example: persistent DB connections that have timed out at the DB

• Ties up memory resources – Later: how this is controlled

• Can tie up 3rd party licenses if they are not properly released

• Hard to get a good grasp of how much memory is really being used

How it happens

• Each pool is controlled by an ID – Passed in as a parameter by the client during launch

• http://myserver:9944/auth/launchjob?_protocol=Protocols/Foo&__poolid=MyPool

– Professional client creates a pool for each new job • Uses job ID as pool ID

• Links the pool to its specific set of scisvr processes • Pool behavior is controlled by server configuration

– Configurations are created by packages – Pool ID is matched to pool configuration.

• If a specific configuration does not exist, the _default_pool configuration is used

Configuration

• <InstallDir>/apps/myCompany/myPackage/xml/Objects/SciSvrPoolConfig.xml

– Maximum processes allowed in the pool

– Minimum/maximum number of idle processes to keep

– How quickly to trim the servers when the maximum is exceeded

– How to queue excess job requests in this pool

– How long is a process allowed to sit idle

– Memory thresholds for idle processes

• See apps/scitegic/core/xml/Objects/SciSvrPoolConfig.xml for example – You rarely need to set up a specific configuration!

• The _default_pool configuration is fine for most pools

Maximum Servers

• Determines how many processes can reside in that pool

• Requests past the maximum will follow the queueing behavior for the pool – Queue, spawn new process, reject request

• 8.5 always spawned new processes

• Maximum servers is a soft limit – Simultaneous job requests can push the number of processes

past the maximum • The excess will subside as jobs finish

Queueing Within Pools

• Added in 9.0

– In 8.5, requests past the maximum for a pool spawned new processes

• Improves overall throughput under heavy load

• Queued items can time out for longer running jobs

• Separate from the queuing that takes place when the entire server reaches its job limit

Queueing Within Pools

Example which reads 500 molecules and returns a bit of JSON text. Maximum servers configured for pool: 32

Elapsed Time/Throughput vs Number of Simultaneous Clients

Spare Servers

• Scisvr processes that are idle

– Minimum • Server will spawn new processes to reach minimum

– Maximum

– How quickly to trim extras • Trimming 1 server every N seconds avoids sloshing

– Idle timeout

Warmup protocols

• Pools that have StartServers or MinSpareServers set can configure a protocol that runs when a new server is spawned – Only affects processes that are spawned to meet the minimum

• Processes launched as a job request will simply run the request

– Initialize DB connections and memory use

– Preload expected protocols and shortcuts

• Example: Components/Data Access and Manipulation/Utilities/Internals/Reference Components/Warmup – Initializes calculable properties, reporting, java components

Memory limits

• Under heavy memory usage, pooled processes will shut down

– Part of pool configuration

– _default_pool • 80% total RAM usage

• 15% total RAM usage for an individual process

• Example: A server has 8 GB of RAM

– Idle pooled processes will shut down when RAM usage reaches 6.4 GB

– If an individual idle process reaches 1.2 GB, it will shut down

Impersonation

• Windows – Restricted

• Impersonation occurs after process launch • Different user can reuse the same process

– Full • Impersonation occurs during process launch • Different user cannot reuse the same process

– Queueing within pools is disabled – May wish to reduce idle timeout of pool configurations (300 to 30)

• Linux – Impersonation occurs after process launch – Different user can reuse the same process

• Connection Timeout

– Keeps the connection open while scisvr is idle

– Supported by ODBC and JDBC data sources

Data Sources

Debugging

• http://<server>:<port>/scitegic/managepools?action=debug

– Shows each pool by ID. • Configuration

• Processes that belong to the pool

– PID

– Owner (impersonation only)

– Number of times the server has executed jobs (including warm ups)

– State

• Queue

– Apache Process/Threads that are waiting for a server in this pool

Debugging

Using Pools From Clients

• Pro Client – Automatic based on jobID

• Create Protocol Link… – Add __poolID as a parameter to your URL

• http://<server>:<port>/auth/launchjob?_protocol=ABC&__poolID=MyPool

• Reporting Forms – Add __poolID using “Hidden Form Data”

• Protocol Function – use “Application ID” or “Pool ID”

• Web Port and Reporting Protocol Links – Add __poolID as a parameter to your protocol

• Client SDKs – Pass in __poolID as a parameter when you call the LaunchXXX() methods

Special pools

• Windows

– Warmup • Jobs launched without a poolID will use a single use server from the

warmup pool if one is available

– KeepWarm • A single scisvr process that runs a small warmup job every 300 seconds

to keep AEP libraries from swapping to disk

• Helps prevent cold server delays

The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.

Disclaimer