Nowadays, with inexpensive hardware cost, large memory is more and more popular then before. In best practice, for the large memory, there are some recommendations.
HugePages is a method to have larger pages where it is useful for working with very large memory. It is both useful in 32- and 64-bit configurations. By default, Linux kernel manage memory by dividing 4K pages, with Hugepage, the pagesize is increased to 2MB, therefore, reduce total number of pages by Linux kernel, meanwhile, it reduce the amount of memory for managing the page table. Hugepages are not swappable, which avoiding performance issue by swapping.
From the Oracle Database perspective, with HugePages, the Linux kernel will use less memory to create pagetables to maintain virtual to physical mappings for SGA address range, in comparison to regular size pages. This makes more memory to be available for process-private computations or PGA usage.
The Hugapage concept is introduced in 2.6.23 kernel.
Be aware, Hugepage is conflict with AMM, DO NOT use AMM while Hugepage is enabled.
There's a script to calculate the Hugepage size in MOS: Oracle Linux: Shell Script to Calculate Values Recommended Linux HugePages / HugeTLB Configuration (Doc ID 401749.1)
The output should be like this:
Recommended setting: vm.nr_hugepages = 67
Add above values to
/etc/sysctl.conf, and execute
sysctl -p to enable the configuration.
- Setting memlock in
Set the value (in KB) slightly smaller than installed RAM, example for 64G RAM:
oracle soft memlock 60397977
- Verify Hugepage is set
grep -i huge /proc/meminfo
DO NOT forget to disable AMM if enabled. Remove
memory_max_size from spfile is recommended.
Huge page on AIX is called largepages.
MOS: How to enable Large Page Feature on AIX-Based Systems (Doc ID 372157.1)
- Verify current large pages setting
[email protected]:/# vmo -L lgpg_size
- Setting large pages
Give the Oracle user ID the CAP_BYPASS_RAC_VMM and CAP_PROPAGATE capabilities by following these steps:
--First check the current capabilities:
Configure the number and size of large pages:
vmo -p -o lgpg_regions=3800 -o lgpg_size=16777216
num_of_large_pages = INT((total_SGA_size-1)/16MB)+1
Allocate 16777216 bytes to provide large pages, with 3800 actual large pages.
Change lru_file_repage, the default is 1:
vmo -o lru_file_repage=0
Load the configuration into the kernel and reboot the server
- Setting parameters in instance level
alter system set lock_sga=TRUE scope=spfile;
On AIX databases, USE_LARGE_PAGES parameter has NO impact.
These parameters are only valid for databases running on Linux, the value of this parameter even if set to FALSE will be ignored on AIX.
After starting up instance, verify large pages is using:
svmon -P SMON_PID
vmstat -P all
For Linux, Oracle strongly recommend to disable transparent Hugepages, especially in RAC, transparent Hugepages will occur node reboot.
- Verify current setting
cat /sys/kernel/mm/transparent_hugepage/enabled #OEL
If "enabled" is NOT set to "[never]", the Transparent HugePages are being used.
grep AnonHugePages /proc/meminfo
If the AnonHugePages is not equal 0, the kernel is using Transparent HugePages.
Add "transparent_hugepage=never" to the kernel boot line in the "/boot/grub/grub.conf" file.
RHEL7: edit the "/boot/grub2/grub.cfg" file using the grubby command.
Server must be rebooted for this to take effect
Alternatively, add the following lines into the "/etc/rc.local" file and reboot the server.
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
For RAC system which SGA is over 100G, Oracle provide a best practice: Best Practices and Recommendations for RAC databases with SGA size over 100GB (Doc ID 1619155.1)
a. Set _lm_sync_timeout to 1200
Setting this will prevent some timeouts during reconfiguration and DRM. It's a static parameter and rolling restart is supported.
b. Set shared_pool_size to 15% or larger of the total SGA size.
For example, if SGA size is 1 TB, the shared pool size should be at least 150 GB. It's a dynamic parameter.
c. Set _gc_policy_minimum to 15000
There is no need to set
_gc_policy_minimumif DRM is disabled by setting
_gc_policy_time = 0.
_gc_policy_minimumis a dynamic parameter,
_gc_policy_timeis a static parameter and rolling restart is not supported. To disable DRM, instead of
_lm_drm_disableshould be used as it's dynamic.
d. Set _lm_tickets to 5000
Default is 1000. Allocating more tickets (used for sending messages) avoids issues where we ran out of tickets during the reconfiguration. It's a static parameter and rolling restart is supported. When increasing the parameter, rolling restart is fine but a cold restart can be necessary when decreasing.
e. Set gcs_server_processes to the twice the default number of lms processes that are allocated.
The default number of lms processes depends on the number of CPUs/cores that the server has, so please refer to the gcs_server_processes init.ora parameter section in the Oracle Database Reference Guide for the default number of lms processes for your server. Please make sure that the total number of lms processes of all databases on the server is less than the total number of CPUs/cores on the server. Please refer to the Document 558185.1 It's a static parameter and rolling restart is supported.