This is part 2 of troubleshooting SharePoint performance problems. In order to properly diagnose and troubleshoot SharePoint performance problems, Microsoft CSS support often depends on a performance monitor (perfmon) log during problem period. In addition, we might request a baseline perfmon of healthy behavior.
In this case, it’s clear that the PID information is missing so you must fix the counters as this is what enables a mapping of w3wp to a process ID.
a) Click Start, click Run, type cmd, and then click OK.
b) At the command prompt, type unlodctr .NetFramework, and then press ENTER.
c) Use the cd command to change to the Windows\Microsoft.Net\Framework\v2.0.50727 folder.
d) At the command prompt, type lodctr corperfmonsymbols.ini, and then press ENTER
Reference the following article for more information: http://support.microsoft.com/kb/922775
Steps are below:
1. Open Performance Monitor, click the + button. Under Performance object, select .NET CLR Memory. Under “Select counters from list”, verify that you see Process ID
2. Add this registry key so that Performance Monitor will list the PID of the W3WP as part of the counter detail.
The following KB article explains how to do this: http://support.microsoft.com/kb/281884
3. Open Performance Monitor
4. Expand Performance Logs and Alerts
5. Right click on Counter Logs, click New Log Settings and type a name for the Perfmon log and click OK
6. Click on the Add Objects button. Then add the following objects and click Close
Note: Add the Threads object when troubleshooting CPU Spikes/CPU high utilization.
7. Make sure to use “Run As” so we can accurately capture the .NET counters. This should be run as the same account as the application pool.
8. Setup performance logging at 5 second intervals.
9. Select the Log Files tab and Configure button and set the preferred directory path and file name. Limit the log file size to 250 MB and hit OK.
Example screenshot below:
10. Select the Schedule tab and the start up option is your decision on whether or not you want to automatically start the log or manually.
The Stop Log section settings need to match the following screenshot:
The goal is that no information is going to be overwritten. So when the buffer limit of 250 MB is reached, a new perfmon log will be created and the existing one is preserved.
Hit okay and if you set it to start automatically based on a specified time, then you can check Performance Monitor and verify its running. If you set the Start Log option to manual, then you need to go into Performance Monitor and manually start the log.
Great, now perfmon log is up and running but we still need one more piece of information. The PID information is helpful but we need to be able to map which PID or PID’s are mapped to a specific application pool. This requires the use of IISAPP. Steps for running IISAPP are the following:
11. Go to the same server you configured perfmon and launch cmd prompt. Type the following:
iisapp
After hitting enter, you might get prompted to register cscript. Hit yes and run iisapp again.
You will see an output like the following:
You can see the value in the screen shot. Now when I launch a perfmon to review and look at process information, I can map the PID to the associated application pool.
For data collection purposes, I recommend piping this out to a txt file like running iisapp > c:\iisappout.txt.
I would run this at least every 30 minutes and you can easily configure a script or batch file to automate this process.
Note: IISAPP doesn’t work in Windows 2008. In order get PID <–> Application Pool mapping you must run the following:
Launch cmd prompt and run the following command from the c:\Windows\System32\inetsrv directory:
appcmd.exe list wp
Question/Answer Section
Q: Why do I suggest 5 second interval for sampling period?
A: It really depends on how often the performance problem reproduces and how long you decide to leave the log running. If your capture period is 8 hours or less, then I recommend 5 seconds. If your capture period is more than 8 hours, I assume that it’s unknown when the problem will occur which requires a long running perfmon. In this case, I recommend a sample period of 120 second interval. The key is managing how rapidly the perfmon log (BLG file) grows in size.
Q: Shouldn’t I set the log to overwrite instead of creating new BLG’s during long running perfmon sample?
A: This is personal preference but several conditions exists where you probably want to set the log to overwrite
Conditions:
1. You are limited on available free space on directory holding the perfmon output (BLG files)
2. Your not sure when the performance problem will reproduce and it could take 2/3 days or even a few weeks.
I will cover a nice in depth look of SPDiag in part 3 of Troubleshooting SharePoint Performance series.
Part 2 of this blog is dedicated to walking you through setting up performance monitor. Why is that necessary? We have a very specific method in collecting this data. A shout out is in store for Jason at MSFT for originally documenting most of these steps.