Many times I come across the configurations where the servers physically have 4GB memory installed but OS does not see it all. OS usually reports between 512 to 768 MB less!

Running Exchange 2003 on such server with recommended memory tuning parameters as documented in my previous post, makes it even worse because doing this starves the system of available memory. Available MB is the counter to watch on such systems along with Free System PTEs.

So, what causes this to happen? Is it not ok to use the memory tuning in such configurations?

The answer is, why is the system not reporting correct memory and what can you do to address that.

Typically only system above 4GB requires the use of /PAE switch in order for OS to see correct amount of memory by loading PAE kernel. Why would it work in a system with only 4GB RAM installed? The answer lies in PCI-Express based chipsets and how they use MMIO. Microsoft Exchange Team has published a detailed article explaining this.

If you are used to the original guidance of not using /PAE switch on your Exchange 2003 servers, it is time to reconsider.

As organizations refresh their old hardware and are not ready yet to deploy Exchange 2007, they tend to follow their documented build process and overlook the impact of new hardware components on the memory. When they notice it, it is usually because they have a problem with the server and they dig deeper.

So go look at your Exchange 2003 servers and see if they are reporting less memory than actually installed. Maybe it’s time to use /PAE in boot.ini!

Tags: Exchange 2003

When troubleshooting performance of Exchange 2003, most overlooked performance counter is Free System PTEs. Very simply put PTE is basically an I/O partition table.

Why is PTE important?

If a system is running out of PTEs, it can cause system hangs, sporadic lockups, general unresponsiveness. All is bad for a server running enterprise messaging.

Why is it more apparent in Exchange 2003 servers? It is due to the fact that the underlying 32-bit architecture memory limitations and resource requirements of Exchange 2003. There is published guidance on memory tuning and affects of various configurations on System PTEs. The Microsoft KB articles are 325044, 815372 and 311901.

Simply put, most Exchange 2003 servers have 4GB memory installed and /3GB switch configured in boot.ini. This significantly reduces system PTEs. The balance is shifting to provide more resources for Exchange to run smoothly at a cost. If not tuned properly, this can backfire and put your server at a risk ok being unhealthy and prone to crashes.

What can you do to fix this?

Follow the best practices documented in the Microsoft KB articles above. I have listed some obvious steps below:

Remove unused drivers. Drives reserve System PTEs. If not used, removing them can free up PTEs for system usage. The amount of PTEs depends on driver.

Remove display drivers and install System VGA driver shipped with Windows version you are running. This can be as significant as 15000 PTEs from my testing and real world results. Also, use /basevideo in boot.ini (with system VGA driver) which can free up additional ~1000 PTEs. Using /basevideo with OEM video driver can actually use more PTEs.

Tune /userva values. In most cases /userva=3030 should be sufficient. You can tune it in steps and try /userva=2970 and /userva=2900. Understand that this is a balancing act. I suggest don’t change it from 3030 if not needed.

Make sure SystemPages registry entry is set to 0 if you are running Windows Server 2003 SP1 or later.

Use /PAE switch in boot.ini if you are running Windows Server 2003 SP1 or later. (Why use /PAE switch is a subject for a different post)

Always test your configuration in lab before deploying it to production.

If any of the above steps are not clear, refer to KB article 815372.

What is healthy for Free System PTEs counter?

If System PTEs are lower than 5000, the server is in critical condition and you should address the situation immediately.

If System PTEs are above 5000 and below 10,000, there is some memory tuning necessary.

If System PTEs are in 10,000-15,000 range, it is healthy (this is subjective statement, other factors must be considered, we are talking about memory and System PTEs specifically and there may be issues with disk, processor which are not being considered in this post).

If System PTEs are above 20,000 you may be treading Exchange performance for System Performance. Verify that the configuration items mentioned above are correctly set.

Further reading:

System Pages set too High

Memory Management | SystemPages

Clint Huffman | prescriptive guidance on how to troubleshoot PTEs

Jeff Stokes | PTE Depletion, Handle Leaks

wikipedia | Page Table

Quote of the day:
Reading is to the mind what exercise is to the body. – Sir Richard Steele

Tags: Exchange 2003