For optimal Caché performance, you need to calculate proper values
for certain Caché system parameters. These values allow you to determine
whether you need to adjust certain OpenVMS system parameters. The values you
choose should minimize swapping and paging that require disk accesses, and
thus improve system performance.
Review this section carefully and calculate the proper values for both
your operating system and Caché before proceeding. Use the
OpenVMS Parameter Calculator and
the tables provided here to record the current and calculated values for your
system parameters. You can then refer to these tables when you install Caché.
After your system is running, you may need to adjust these values to gain
optimal performance.
If you are not already familiar with the memory organization at your
operating system level, consult the appropriate system documentation and read
the overviews of process and physical memory allocation provided in the next
few sections:
OpenVMS Process Memory Organization
An OpenVMS process is any entity that can be scheduled. It may be interactive
or run as a batch process. If the OpenVMS process creates subprocesses, that
process and all its subprocesses together are called a job.
Many operating system tasks are OpenVMS processes. All processes, whether
they are interactive, batch, or operating system-specific, share the structures
that this section discusses. OpenVMS can run several processes concurrently.
Depending on the size of your computer, OpenVMS could be running between 4
and 1,000 or more simultaneous processes.
Each process uses a share of physical memory, where it stores certain
process-specific operating system information, any code the process is using,
and any data that it uses or generates. For example, if a programmer edits
a routine with an OpenVMS editor, OpenVMS loads the editor into memory, allocates
space for the editor's buffers and variables and loads all or a portion of
what is being edited into memory.
Sometimes code or data is available to more than one process. The editor
is one example of this type of code. Another example is the code for Caché
and some of the data created with Caché.
Any piece of code or data that several processes can share is stored
in a shared section of memory. Shared sections are created by the first user
of a system, or during system initialization. They are then made available
to subsequent users of that system rather than being duplicated and stored
with each individual process.
Caché system code, global buffers, and routine buffers are stored
in shared sections of memory.
How OpenVMS Measures Process Memory
OpenVMS measures the code and data area that a process requires in
pages.
Shared sections are also measured in pages, sometimes called shared or global
pages.
On OpenVMS Alpha systems, page size is not fixed. Normally, it is 8192
bytes. Each page is divided into
pagelets that are 512
bytes long.
OpenVMS Page Organization
Sometimes a process needs to access many more pages than can fit within
the physical memory allocated to the process. When this situation arises,
OpenVMS stores the extra pages on disk. The operating system keeps only the
most active, or most recently used, pages in memory. The total of all the
pages of a process, whether they are stored in memory or on disk, is called
the
virtual memory set of the process. The pages currently
stored in memory are called the
physical working set of
the process, often called
working set. The status of
each page is kept in the process's working set list.
OpenVMS keeps track of free pages available to processes in a free page
list. It also keeps a modified page list, which tracks pages that have been
changed by a process and must be written to disk before being made available
on the free page list.
Sometimes a process needs access to a page not currently in its physical
working set. When this happens, OpenVMS searches two places for that page.
First, OpenVMS looks in the modified and free page lists, because the page
might still be available in physical memory. If the page does not appear on
either of these lists, OpenVMS then retrieves the page from disk.
When OpenVMS must retrieve a page, the process has incurred a
page
fault. When the new page is in the modified or free page list,
the page fault is called a
soft page fault. When the
page is only available on disk, the page fault is called a
hard
page fault. While OpenVMS retrieves the page from disk, the process
becomes inactive. It remains in a page fault wait state until OpenVMS has
successfully retrieved the necessary page. The length of time that this procedure
requires varies according to the size of the disk, its rotation speed, and
the load on the system.
When OpenVMS retrieves the new page, it places one of the current working
set pages in the modified or free page list. OpenVMS chooses a page that the
process has not used recently.
You can minimize soft page faults by setting the OpenVMS process parameters
WSDEFAULT,
WSQUOTA,
and
WSEXTENT appropriately for each process.
Note:
The Caché
SETQUOTA utility lets you modify
the default values of quota parameters for jobbed processes, including the
network daemons and job servers.
SETQUOTA can be set
to run automatically during multiuser startup, or interactively after startup.
OpenVMS Physical Memory Allocation
In OpenVMS, you use the
SYSGEN utility to set system
limits on process memory use. You can then set process-by-process (user-by-user)
limits with the
AUTHORIZE utility.
The table below describes the OpenVMS system parameters that affect
the memory available to a process's working set.
OpenVMS Working Set Parameters
Parameter |
Definition |
WSDEFAULT |
Initial size of the working set in 512-byte pagelets. |
WSQUOTA |
Maximum guaranteed amount of physical memory, in pagelets,
that OpenVMS allocates to a given process. WSQUOTA is limited by the value
of the SYSGEN parameter WSMAX. If WSQUOTA is larger than WSMAX, the user only
has access to the amount of memory in WSMAX. |
WSEXTENT |
Indicates the maximum amount of physical memory which
the process can own. The amount WSEXTENT less WSQUOTA is called borrowed memory,
which is only allocated to the process if the system's current free list size
exceeds the amount specified in the parameter BORROWLIM. |
WSINC |
The number of pagelets by which the working set size
of a process is increased when it needs to grow. |
WSDEC |
The number of pagelets by which the working set size
of a process is decreased when it needs to be reduced in size. |
BORROWLIM |
Minimum number of pagelets that must be in the free
page list before a process can borrow pages in WSINC amounts up to WSEXTENT
pages. |
PFRATH (Page Fault Rate High) |
Indicates the number of page faults that must occur
within the period specified by AWSTIME, before OpenVMS makes an automatic
working set adjustment to increase the working set by WSINC. |
PFRATL (Page Fault Rate Low) |
If the number of page faults is less than this value,
OpenVMS makes an automatic working set adjustment to decrease
the working set by WSDEC, but not below the point set by AWSMIN. |
Note:
While parameters have units of pagelets, OpenVMS allocates memory in
pages. For this reason, the parameter values you choose should be even multiples
of the number of pagelets per page. OpenVMS automatically rounds these parameters
if you do not follow this guideline.
Default Memory Allocation
The following steps summarize the algorithm OpenVMS uses to determine
the amount of physical memory available to each process. Each capitalized
term represents a system parameter that OpenVMS uses in allocating physical
memory.
-
The process begins executing an image. OpenVMS allocates to
it the number of pagelets specified by the quota parameter
WSDEFAULT.
Page Allocation:
0--->WSDEFAULT
-
If the process has a higher page fault rate than
PFRATH,
OpenVMS allocates more pagelets to it. OpenVMS allocates these pagelets in
increments of
WSINC until the process has
WSQUOTA total
pagelets.
Page Allocation:
0--->WSDEFAULT-->WSINC-->WSINC-->WSQUOTA
-
If the process continues to have a high page fault rate and
the free list size exceeds
BORROWLIM, OpenVMS will continue
to allocate pagelets in
WSINC blocks until the free list
size is inadequate or
WSEXTENT is reached.
Page Allocation:
0-->WSDEFAULT-->WSINC-->WSQUOTA-->WSINC-->WSEXTENT
-
If the process's page fault rate drops below
PFRATL,
OpenVMS removes pages from the working set in increments of
WSDEC until
the page fault rate exceeds
PFRATL.
-
When the process exits the image, it loses these additional
pages. For example, consider a monthly batch payroll job. When you run it
in July, it begins execution at
WSDEFAULT, and gains pages
until it reaches
WSEXTENT. When you run it in August, it
again begins execution at the default value of
WSDEFAULT.
See your OpenVMS documentation for more on the OpenVMS page allocation
scheme.
Process Control of Memory Allocation
OpenVMS provides several facilities to alter working set parameters
on a process-by-process or user-by-user basis. The table below summarizes
these methods:
Working Set Control
Type of Process |
Method of Working Set Control |
Users |
The AUTHORIZE utility lets you
change the default working set on a user-by-user basis. |
Interactive Processes |
The DCL command SET WORKING_SET allows
interactive processes to change working set parameters. |
Batch Processes |
The SUBMIT command qualifiers for
working sets allow batch processes to alter physical memory allocations. |
Batch Queues |
You can use the DCL command INITIALIZE/QUEUE to
alter working set parameters for batch queues. |
Keeping Memory Free for New Processes
Memory is finite. Every time a new process begins under OpenVMS, it
takes
WSDEFAULT pages away from the total number of available
pages. The remaining memory is available for the free page list and for
WSINC additions
to each process's allocation.
Heavily loaded systems can run so many concurrent processes that the
number of pages remaining for use in the free page list becomes very small.
OpenVMS includes a parameter called
FREELIM that sets a
lower limit for the number of pages in the free page list.
The size of the free and modified page lists directly affects the ratio
of hard page faults to soft page faults. Hard page faults cause a process
to experience a resource wait state, resulting in slow execution. For better
performance, it is important to tune the system to minimize hard page faults.
While soft faults are less expensive, an excessive soft page fault rate can
also lead to poor performance.
Whenever OpenVMS detects that the size of the free page list falls below
FREELIM,
OpenVMS starts a mechanism to bring the free list size up to
FREEGOAL.
To achieve this, if all other mechanisms are inadequate, OpenVMS may move
the entire working set of an inactive process out to disk. This procedure
is called
swapping. The process placed on disk is swapped
out. When the process is brought back into memory, it is swapped in. Swapping
has a detrimental effect on overall system performance.
Allocation for Shared Sections
OpenVMS uses a slightly different approach for allocating memory for
shared code and data. The
GBLSECTIONS parameter sets the
number of shared sections to be allocated when the system is started. The
GBLPAGES parameter
sets the number of global page table entries. Every group of 128 page entries
requires 4 bytes of resident memory. In addition, OpenVMS uses the
GBLPAGFIL parameter
to set the maximum number of page file blocks available for global pages.
How Caché Uses OpenVMS Memory
Caché uses both shared memory and memory private to each process
when running on OpenVMS.
Balance Memory Locking and Paging
All Caché code is shared, and can be physically locked in memory.
The same is true for the global buffer pool.
Routine buffers work similarly, though with a slight difference: routine
buffers are shared, but you can specify how much of each routine buffer is
locked into memory. If a routine is larger than the locked portion of a routine
buffer, only a portion of the routine in the routine buffer is locked in memory.
The remainder of the routine is loaded into unlocked physical memory.
Locking shared data and routines in memory allows better response time
since memory access is quicker than disk access. As a result, the more globals
and routines that are kept in memory, the better. Memory, however, is a finite
resource. The more global and routine buffers that are allocated, the less
memory is available for OpenVMS processes. When less memory is available,
more OpenVMS paging occurs.
The goal, therefore, is to choose a number of global and routine buffers
that lets you keep enough globals and routines in memory without negatively
affecting OpenVMS paging. This document provides guidelines for selecting
these values.
Use Non-Paged Dynamic Memory to Reduce Soft Page Faults
In OpenVMS, Caché allows you to locate the global buffers in
a type of system memory called non-paged dynamic memory.
The advantage of placing global buffers in non-paged dynamic memory
is that this type of memory is indexed via the OpenVMS working set list, not
individual process's working set lists. This method reduces soft page faulting
related to accessing global buffers. InterSystems recommends that you use
non-paged dynamic memory for your global buffers.
Use Process-Private Space to Reduce Paging
Every Caché process maps to the shared memory sections, but it
also has access to a private area of memory called process-private space.
This private area includes variables, arrays, stacks, and other data structures
that belong to a particular process.
A portion of this private area of memory may be locked into the process's
working set to reduce paging. None of the private area, however, is physically
locked in memory.
Calculating OpenVMS Parameters
The accompanying OpenVMS Parameter Calculator offers good starting values
for both Caché and OpenVMS parameters. Later, if your system is not
functioning as well as you would like, you can adjust these parameters to
achieve optimal performance. The following sections describe the process of
determining OpenVMS parameters:
You may want to print out the tables that follow so that you can record
the relevant values.
Determine Parameter Calculator Input Values
This section helps you find appropriate values for the fields that control
the number of processes and the number of global and routine buffers used
by Caché. You need to determine these values, so that you can use the
Parameter Calculator to compute both Caché and OpenVMS parameters accurately.
During installation you can set these and other Caché parameters,
or you can retain default values. If you find later that your system needs
tuning, you can adjust the values at any time using the Caché Configuration
Manager. You need to restart Caché for parameter changes to take effect.
Determine Number of Processes
Before you can determine how many global and routine buffers Caché
should allocate at startup, you must first determine the maximum number of
Caché processes that will be running on your system at one time.
Every user in Caché constitutes one Caché process. Every
Caché
Job command creates a process. You should
consider any process that appears on the %SS system status display, except
the system processes
Garbage Collector and
Write
daemon. As a rule of thumb, use the number of processes called
for in your Caché license for this value; it cannot exceed your license
limit.
Determine Size of Routine Cache
Each routine buffer holds one and only one routine at a time. An OpenVMS
Alpha system can have up to 524 MB of routine cache.
Many processes can share a routine buffer. Thus, when a process switches
from one Caché routine to another, it merely accesses the other routine
in another buffer. When no process is currently accessing a buffer, the buffer
is returned to the pool of free buffers, and a new routine can be loaded into
the empty buffer as it is required by another Caché process. If all
buffers are occupied when Caché needs to load a routine, it chooses
the least recently used (LRU) buffer, not the least frequently used (LFU)
buffer. Allowing more buffers enhances performance.
The nature of the application influences the appropriate number of buffers.
If many users access a small number of routines, a relatively small number
of buffers will suffice for optimum response time. Conversely, an application
with a large number of routines will benefit from a large number of buffers.
The parameter calculator starts with a default value for the number
of routine buffers. This default is based upon a medium-sized site. You can
enter a different value for this parameter in the
Routine Buffer
Pool field in the parameter calculator based on the specifics of
your site.
Note:
If you prefer to enter a value in this field with an MB unit of measure,
this value is used for the
Memory for Routine Cache (MB) field
in the
Calculation Result section and also in calculations
involving routine buffers; it is not recalculated.
Determine Number of Global Buffers Per Process
The parameter calculator assumes the global buffers are 8 KB in size.
The calculator starts with recommended
rule of thumb values
for clustered and non-clustered systems.
If your system is memory-poor, you can reduce this value. Do not, however,
decrease it below a multiple of 32 KB per process.
Using more global buffers will help the performance of most sites. You
can use the statistics produced by the
GLOSTAT utility
to determine if adding more global buffers will reduce disk access and thereby
improve performance.
Record Current OpenVMS Parameter Values
Every system has memory needs other than those for Caché. For
example, there are memory requirements for FORTRAN and COBOL layered products
as well as for OpenVMS itself. To analyze these requirements for your system,
follow the procedures below to display the values of the indicated parameters
prior to installing Caché.
Note:
If you are performing an upgrade, shut down Caché to get accurate
numbers.
-
$ SET DEF SYS$SYSTEM
$ RUN SYSGEN
SYSGEN> USE CURRENT
SYSGEN> SHOW parameter-name
-
$ SET DEF SYS$SYSTEM
$ RUN SYSGEN
SYSGEN> USE CURRENT
SYSGEN> SHOW/PQL
Alternatively, you may run the OpenVMS
AUTHORIZE utility
to display the current value of the user authorization file (UAF) parameters
for each user account that uses Caché:
$ SET DEF SYS$SYSTEM
$ RUN AUTHORIZE
UAF> show user-name
You must then set the UAF records for all of the user accounts individually
once the calculator returns the minimum recommendations for these values.
Use the Caché OpenVMS Parameter Calculator
This section describes how to use the Caché OpenVMS Parameter
Calculator. This tool is an interactive HTML document that provides up-to-date
calculations for the OpenVMS parameters required by Caché. The default
values provide reasonable numbers for a medium sized installation. InterSystems
experts are available to provide additional help configuring and tuning Caché
for your site.
Perform the following steps to retrieve parameter values:
-
Choose whether you are calculating parameters for a clustered
system. If you select
Yes, you receive both the cluster
master values and values for any additional nodes; otherwise, the tool calculates
values for a single node.
-
As input to the OpenVMS Parameter Calculator use the values
you determined in the previous section for the following fields:
Input to OpenVMS Parameter Calculator
Parameter |
User-supplied value |
Number of Processes |
|
Routine Buffer Pool |
|
Global Buffers per Process |
|
-
-
Fill in the values returned by the calculator in the appropriate
tables below.
Analyze the Calculation Results
The results from the calculator are suggested values; treat them as
a guideline. You may need to adjust settings in order to balance and maximize
system performance, depending on your site's hardware and software configuration.
The following values are for Caché parameters that you can update
during the installation or by using the
Configuration Manager from
a Windows client after you install Caché.
Output Caché Parameters
Parameter |
Calculated value |
Number of Global Buffers |
|
Memory for Database Cache (MB) |
|
Memory for Routine Cache (MB) |
|
Update OpenVMS System Parameters
The system parameter values computed by the OpenVMS Parameter Calculator
are the amounts you need to add to your current OpenVMS system in order to
make it ready to run Caché. Caché will consume this much memory,
so evaluate your system to make sure that there is the proper amount of memory
available.
Use this table to help you combine the required amounts returned by
the calculator with your current system values. Update these parameters by
using the
SET command of the OpenVMS
SYSGEN utility.
OpenVMS System Parameter Values
Parameter |
Current value |
Required Caché amount |
Resulting value |
BALSETCNT |
|
|
|
MAXPROCESSCNT |
|
|
|
GBLPAGES |
|
|
|
GBLPAGFIL |
|
|
|
GBLSECTIONS |
|
|
|
LOCKIDTBL |
|
|
|
RESHASHTBL |
|
|
|
SYSMWCNT |
|
|
|
Update Special OpenVMS Parameters
The parameter calculator provides a few special parameters used to configure
Caché on your OpenVMS system:
-
-
The values for the special parameters
PRIORITY_OFFSET and
PE1 are
fixed values recommended by InterSystems; set these parameters to the recommended
values.
Verify the values returned by the calculator match those of your OpenVMS
system.
Special OpenVMS Parameter Values
Parameter |
Returned value |
Resident Memory (MB) |
|
PRIORITY_OFFSET |
|
PE1 |
|
Update OpenVMS Process Parameters
InterSystems recommends minimum values for some PQL process quota parameters.
This is because Caché maintains a table,
GJOBQ,
which holds default values for many of these parameters. You must have authorized
these values in order for Caché processes to be able to use the values
in the
GJOBQ table.
You do not need to calculate values for these parameters. The minimum
suggested values appear in the OpenVMS Parameter Calculator. Enter these values
in the
Recommended minimum value column in the table below.
OpenVMS Process Parameter Values
Parameter |
Current value |
Recommended minimum value |
ASTLM |
|
|
BIOLM |
|
|
BYTLM |
|
|
DIOLM |
|
|
FILLM |
|
|
PGFLQUOTA |
|
|
TQELM |
|
|
WSQUOTA |
|
|
WSEXTENT |
|
|
ENQLM |
|
|
If your current value for any of these parameters is less than the recommended
minimum value, you should adjust the values during installation using the
SET command
of the OpenVMS
SYSGEN utility or the OpenVMS
AUTHORIZE utility
for each user account.
The Alpha OpenVMS platform makes use of the Resident Memory Section
facility. All OpenVMS Caché users are encouraged to make use of this
facility.
Alpha systems running OpenVMS v7.1 and later have a memory mechanism
for allocating global sections (memory which can be shared between processes).
The two features that Caché supports, memory resident global sections
and shared page tables, are always used as a pair.
The advantages of using a memory-resident global section mapped via
a shared page table for the global buffer pool are:
-
Access to the pages in a memory-resident global section is
not charged against the process's pagefile quota nor the working set quota.
Also, when a process maps to a memory-resident global section and references
the pages, it does not use the process's working set list, so process quotas
may often be reduced.
-
Shared page tables enable two or more processes to map to
the same physical pages without each process incurring the overhead of page
table construction, page file accounting, and working set quota accounting.
Internally, shared page tables are treated as a special type of global section
and are specifically used to map pages that are part of a memory-resident
global section.
-
There is only one copy of the global buffer pool page table
on the system, which conserves physical memory and speeds up mapping the global
section into a new process. Shared page tables dramatically reduce the database
server startup time because server processes can map memory-resident global
sections hundreds of times faster than traditional global sections. This increases
overall system capacity and decreases response time to client requests.
The drawback to using these structures is that if you want to increase
the size of your global buffer pool, you may have to restart your OpenVMS
system in order to reconfigure the amount of space reserved for the memory-resident
global section. This space can be reserved when the system starts (via SYSMAN
and AUTOGEN) or it can be allocated dynamically. If it is not reserved then
there may be insufficient space available when the request is made and the
request will fail. If Caché is asked to use a memory-resident global
section and it cannot, it allocates the global buffer pool out of a nonmemory-resident
global section.
Create a Named Resident Memory Section
To use these structures you need to do the following:
-
Use the value returned by the OpenVMS Parameter Calculator
for
Resident Memory as the reserved memory size.
-
Use
SYSMAN to reserve a section of memory.
You must supply a name for the named section. You can use any name that consists
of alphanumeric characters (and the underscore) and is not longer than 43
characters. The syntax for
SYSMAN is:
MCR SYSMAN ! run sysman
SYSMAN> RESERVED_MEMORY ADD "Resident_Memory_Name" -
/ALLOCATE/PAGE_TABLES/ZERO/SIZE=<size in MB>
SYSMAN> EXIT
-
Run
AUTOGEN to process the Reserved Memory
Registry data file which contains the information to establish your memory-resident
global section. After the system is restarted the
SYSMAN command
displays the shared memory section you reserved:
MCR SYSMAN ! run sysman
SYSMAN> RESERVED_MEMORY SHOW
-
Modify the Caché configuration files (
config.def,
cache.cpf,
etc.) to use the reserved section. This is done by setting the reserved memory
parameter in the [config] section so that it reads:
useresidentmem=1[,<resident memory section name>]
Alternatively, you can also modify the configuration files by using
the Caché Configuration Manager on a Windows client:
-
On the
General tab, verify you have the
appropriate OpenVMS configuration file open. If not, on the
Advanced tab,
click
Files, select the correct configuration file,
click
Open, and then click
OK.
-
-
-
-
Restart Caché. If it cannot use the reserved memory
section, Caché displays a message and stores an error code in the
SYSLOG.
If you use resident memory to map your global buffer pool, you may be
able to reduce some of the system parameters, in particular,
WSMAX.
Typically Alpha OpenVMS comes with a
WSMAX default value
much larger than is required for running Caché.