Intro to Get-SRxTopologyReports
It has some dust and time to blow it off. My name is Russ Maxwell and I’m blogging again and excited to share something I’ve been working on for a few months. As you may or may not know, my colleague Brian Pendergrass developed the core of Search Dashboard. Eric Dixon and Brent Groom were also part of the team and they developed the UI portion. It was a tremendous team effort and I was glad to be a part of it. If you’re not aware of Search Dashboard and its capabilities, please review Brian’s post here.
I wanted to contribute and put my stamp on Search Dashboard. I’m excited to produce the first major enhancement to Search Dashboard with a Topology Report. This was more work than I expected and I really appreciate the contributions made by Brian Pendergrass that helped make this a really rock solid report. Internally working with our customers, We’ve often encountered Search performance issues related with Crawl, Content Processing, Indexing, or Query processing. Many times during the discovery phase of these Search performance issues, we found Search Topologies in a questionable state. At times, we made suggestions to update the Search Topology and this resulted in improved performance while optimizing for future growth.
An optimal Search Topology is one that will provide the best Crawl freshness and Query timings. The Search Topology Report is a module I developed which is an extended feature of Search Dashboard. It’s a PowerShell command-let that outputs a report based on a variety of categories. It’s all PowerShell driven and initially designed for SharePoint 2013 On-Premises environments. I’ll discuss the different categories outputted in more details below.
Topology Type Detection
Topology Type Detection determines the size of a Search Topology. This is based on how many index partitions are provisioned for the active search topology.
- Small Search Topology is detected with 1 Index Partition
- Medium Search Topology detected with (2-4) Index Partitions
- Large Search Topology detected with 5 or more Index Partitions
For Example:
Depending on what type of Topology is detected determines how the Search Topology is evaluated.
Redundancy Checks
This check ensures that all components are redundant. For Index Components, it ensures that at least two index replicas/components exist for a given index partition.
If a component fails a redundancy check, it will look like the following:
Duplicate Component Check
The duplicate component will report if more than one component of the same type is running on a given server. For Example, two Analytic Processing Components are running on Server A. In this case, Search Topology Report will flag this. In my case, I had no duplicate components running so it came back clean.
Search Component Compatibility Check
We’ve reviewed Search Topologies over the years and know what mix of components run well together and what mix of components may not run as well together. The Search Component Compatibility check evaluate components running on each server and will provide feedback if components are running together that traditionally shouldn’t be.
Search Topology and Scale Report
The Search Topology and Scale Report contains a ton of checks to evaluate a Search Topology based on the size of the Search Topology. So again, we determine the Search Topology based on how many Index partitions exists. The outline for the specific checks based on Search Topology are the following:
Small Search Topology
- Flags if running more than 2 Admin components
- Flags if running more than 2 Analytic Processing Components
- Flags if running more Crawl Components than Content Processing Components provisioned
Medium and Large Search Topology
- Flags if running more than 4 Admin Components
- Running more than 4 Analytic Processing Components
- Flags if running more Crawl Components than Content Processing Components provisioned
Question: What about Crawl and Index Component Scale?
Answer: These checks require more details which I outlined in their own sections below.
Crawl Component and Scale
First, we ensure that the number of Crawl Components doesn’t exceed the supported limit as well as the number of Crawl Databases. If either of these are exceeded, we return back without checking anything further. Assuming a Search Topology is under the supported limits with crawl components we proceed further. With Crawl Component check, more goes into checking how many components are recommended based on a Search Topology size. We must take into account the index performance level as it is the control mechanism that tells the crawler how many gathering threads are available for a given crawl. Some math is required to determine if you’re scaled properly based on the following setup:
- Index Performance Level
- # of Unique Hosts (total amount of hosts defined in all Content Sources)
- Crawler Impact Rules
- Total # of Crawl Components
Based on a Crawl setup, we established baselines using calculations. If baselines are beneath the baseline or exceed the baseline, the report will suggest the following:
- Over Sized – Recommend a lower number of crawl components
- Under Sized – Recommend a higher number of crawl components
Before this functionality existed this report, it was very difficult for customers to determine if a crawl setup was scaled properly. Also, many assume that the # of crawl components is based on the size of a Search Topology which is established as the number of index replicas. That is not an accurate statement.
Index Component and Scale
Quite a few checks take place within the index and scale portion. We’ve encountered several index performance issues in the past that can index both Crawl freshness and Query timings. Some of these issues were directly correlated to an Index configuration within the active Search Topology. The index component and scale portion will go through a few checks and report if irregularities are discovered. First, Microsoft does leverage a supported boundary when it comes to number of replicas based on a given Index Partition. If less than four million exists per partition, we recommend a max of six replicas per index partition. If more than 4 million items exist per index partition, we flag if more than 3 replicas per index partition is detected.
Another check evaluates if more than ten million items exist in a given index partition. If true, we flag it and provide feedback. As part of the Index Component and Scale evaluation, this command-let also test if an uneven number of index replicas exists across index partitions. That is, it scans the number of index replicas per index partition across the Search Topology and flags if an uneven number of partitions exists. For Example, if Index Partition 1 hosts two replicas while Index Partition 2 hosts three replicas it will be flagged. Another example of a check performed is if more than 25 index partitions exists in the active Search Topology, we’ll flag that the supported limit has been reached. Finally, the index component and scale portion checks if a large number of partitions with a low number, less than 50%, of active docs in each partition. This could potentially suggest an environment is over scaled and provides a heads up. However, it’s quite possible the index corpus will grow into this Index setup which is perfectly okay.
Setup Get-SRxTopologyReports
- SRx Dashboard is required to be installed on the machine. Follow guidance here if needed.
- Download the Enable-SRxTopologyDiag.zip file near the bottom of this blog and unzip into the SRx\Modules\Search Folder:
Note: The above screenshot shows the new module Enable-SRxTopologyDiag.psm1 is extracted.
Running SRx Topology Reports
1. Launch SharePoint 2013 Management shell and navigate to the SRx directory and run the following:
.\initSRx.ps1
2. Next, run the command-let: Get-SRxTopologyReports
Here is an example of the output:
Disclaimer: We released the Search Health Reports (SRx) to help others get more out of Search, but the SRx should not be considered an official product . ALL information in this blog is provided “AS IS” with no warranties and confers no rights. This blog does not represent the thoughts, intentions, plans or strategies of my employer. All content is solely my opinion and provided with a best effort to be based in reality.
All examples, code samples, demonstrations, or anything resembling a “how-to” are provided “AS IS” without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose. Inappropriate comments will be deleted at the authors discretion.