Real Zero-Downtime Patching in SharePoint 2016

It is exciting to have Zero-Downtime Patching (ZDP) capability in SharePoint 2016. However, it requires more effort than most of us might have initially thought. While it is straightforward for other components, it requires a bit more care on Distributed Cache.

Distributed Cache doesn’t support High Availability the way that other services do. While multiple Distributed Cache servers in your SharePoint farm can help distribute the load, the data cached on each Distributed Cache server is NOT replicated to the other Distributed Cache servers. If a Distributed Cache server unexpectedly goes down, the data cached on that server will be lost. That means if you install patches and upgrades a Distributed Cache server without gracefully shutting it down, you will cause data lost!

One may argue that if there are three Distributed Cache hosts on the farm, there will be high availability for the Distributed Cache as AppFabric has as cluster quorum model. That is not true as SharePoint is not using that model!

Therefore, if you have workloads that heavily depends on Distributed Cache and have high availability requirements, add Gracefull Shutdown of the Distributed Cache Service into the patching process to have true Zero-Downtime.

Gracefully shutdown DCS: https://technet.microsoft.com/en-us/library/jj219613.aspx#graceful

Monitor SharePoint 2013 Search Components with PowerShell

This is a prototype for a PowerShell script that monitors the status of each component of Search Service Application in SharePoint 2013. This script can be saved  to a .ps1 file and run by the Window Task scheduler periodically. If it detects that any of the component is not in the “Active” state, it automatically sends an email to the administrator.


Add-PSSnapin Microsoft.SharePoint.PowerShell
#Declare variables for later use.
$ssa = Get-SPEnterpriseSearchServiceApplication
$status = Get-SPEnterpriseSearchStatus -SearchApplication $ssa
#Create an empty array to store any component that is not active.
$unhealthy = @()
#Loop through each component status, and store any one that is not active to the array.
$number = 0
$status | foreach {
if ($_.state -ne "active"){
$number++
$unhealthy +=$number.ToString() + ". " + $_.name + "`n"
}
}

#If there is any component that is not active, send an email to the admin with the component name in the email body.
if ($unhealthy.count -gt 0) {
$result = "The components below are not active:`n " + $unhealthy
$params = @{'To'='whomitmayconcern@company.com'
'From'='admin@company.com'
'Subject'='Attention! Search Service Components Unhealthy'
'Body'=$result
'SMTPServer'='smtp.contoso.com'}
Send-MailMessage @params
}

The script has been tested with a single server farm with a single Search Service Application. If your scenario is different, you should adjust the script accordingly.

Batch Enabling Auditing across Many SharePoint Sites

If you are only looking for the script and are not interested what else I say, just grab them here:

#Define the function Enable-Auditing. The URL parameter accepts pipeline input. It also enables log trimming, and log retention time is set to 30 days. This part is kind of "hardcoded", but it should not be too difficult to change it. 
function Enable-Auditing {
param([Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]$Url,
[Parameter(Mandatory=$True)]$AuditedActions);
$site = Get-SPSite $Url;
$site.TrimAuditLog = $true;
$site.AuditLogTrimmingRetention = 30;
$site.Audit.AuditFlags = $AuditedActions;
$site.Audit.Update();
$site.Dispose();
}
<# 
Run the commands to apply the settings to specific Site Collections. For the -AuditedAction parameter, input any of the following:
"All" to audit all auditable actions.
"None" to disable auditing
An array of action names to enable auditing a specific set of actions to audit, e.g. "Update", "Delete", "Search". 
Check MSDN documentation for a complete list of auditable actions: https://msdn.microsoft.com/en-us/library/microsoft.sharepoint.spauditmasktype.aspx
#>
Enable-Auditing -URL https://teamsite.contoso.com -AuditedActions "Update", "Delete"

As the -URL parameter accepts pipeline inputs, batch action can be done to multiple site collection with one line of command such as the one below:

#The command below enables auditing Update and Delete actions on all Site Collections whose URL contains "hr".
Get-SPSite -WebApplication http://teamsite.contoso.com -Limit All | ? {$_.url -like "*hr*"} | ForEach-Object {Enable-Auditing -Url $_.url -AuditedActions "Update", "Delete"}

OK, if you are interested in my monologue discussing this function, please read on. Otherwise, the content above is all you need.

How do you make sure auditing is enabled all the time? How may Site Collections do you need to manage? Does each Site Collection has its own Site Collection Administrator?

Site Collection Administrators has the permission to change site audit settings. That’s a potential risk since they can intentionally or unintentionally change the audit settings, while auditing is usually an organization-wide policy that needs to be enforced. Sometimes, turning on unnecessary auditing is bad as well as it will make the content DB grow faster. A single piece of audit log is about 1KB. Imagine, 1000 people are visiting 100 locations in a day!

One solution is to create a PowerShell scripts running under a task scheduler that enforces auditing policies, including:

  • Actions to audit
  • Whether to enable audit log trimming
  • If log trimming enabled, how many days of log to retain

In SharePoint Management Shell, there is no direct cmdlet for this purpose yet. We can define a function to make batch operations easier.

If you run Get-Member on a SPSite object, you will find that there are a few properties related to auditing:

  • Audit
  • AuditLogTrimmingCallout
  • AuditLogTrimmingRetention

To enable/disable auditing, the trick is to set the value of the SPSite.Audit.AuditFlags property. Based on tests, it accepts strings or array of strings. So there comes the code at the beginning of this post.

How to hyperlink to a specific location on a long web page?

For those looking for a quick answer, here it is:

Put in the pattern in the address bar: <URL>#<HTML Element ID>

For example:

https://technet.microsoft.com/en-SG/library/cc262787.aspx#ContentDB

https://msdn.microsoft.com/en-sg/library/ff877884.aspx#AvailabilityModes

If you would like to read a discussion on this topic, feel free to move on. Otherwise, the answer above is all you need to know. 😉

When you are sharing a webpage with others, it may be frustrating for the reader to find the exact content you are sharing when the webpage is long. What if you can direct the reader to the exact location through the hyperlink you are sharing? For example, on a lengthy TechNet Article about SharePoint limitations, the reader is directed to the Content Database limitations directly when opening the page with this hyperlink: https://technet.microsoft.com/en-SG/library/cc262787.aspx#ContentDB

The trick lies in the suffix “#ContentDB” in the URL. So the question this post is trying to answer is how to determine what to add to the end of the URL for navigating the users to a specific location on the webpage directly?

We know in HTML we can assign an ID to a tag such as <p id=”something”></p>, which is the unique identifier of the this is specific element. You can then use this ID to locate the content to share. Not every element on a webpage has an ID attribute though. So having an ID is the prerequisite for locating the content directly.

How to find the ID of the location you are sharing if any? There are two ways, the easy way and the hard way.

The easy way exists when there are internal hyperlinks on a webpage, i.e. the hyperlink points to a location on the same page. In this case you can copy the hyperlink directly and share with others. For example, on TechNet articles, you constantly see hyperlinks to the same page.

 Hyperlinks

The hard way comes when there is no internal Hyperlink on the webpage. You will need to check the source code of the webpage for any ID that can be used.

One-pic-4-a-thousand-words

Move SharePoint 2013 Search Index Location

Task: Move search index from D: drive to E: drive on the existing Index Servers.
Search topology:

Search_Topology

Preparation:

#Discover current search topology information with documentation.

$ssa = Get-SPEnterpriseSearchServiceApplication
$old = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
Get-SPEnterpriseSearchServiceApplication >C:\SSA_Info.txt
Get-SPEnterpriseSearchComponent -SearchTopology $old >c:\oldtopology.txt
Get-SPEnterpriseSearchStatus -SearchApplication $ssa

#Define parameters for the move.

<# $hostA = Get-SPEnterpriseSearchServiceInstance -Identity "Server1" $hostB = Get-SPEnterpriseSearchServiceInstance -Identity "Server2" #>
$hostC = Get-SPEnterpriseSearchServiceInstance -Identity "Server3"
$hostD = Get-SPEnterpriseSearchServiceInstance -Identity "Server4"
$hostE = Get-SPEnterpriseSearchServiceInstance -Identity "Server5"
$hostF = Get-SPEnterpriseSearchServiceInstance -Identity "Server6"
$indexPath = "E:\Search_Index01"

#Clone the current active topology

$clone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone -SearchTopology $old

#To the cloned topology, add new Index components using new path.

New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance $hostC -IndexPartition 0 -RootDirectory $indexPath
New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance $hostD -IndexPartition 0 -RootDirectory $indexPath
New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance $hostE -IndexPartition 1 -RootDirectory $indexPath
New-SPEnterpriseSearchIndexComponent -SearchTopology $clone -SearchServiceInstance $hostF -IndexPartition 1 -RootDirectory $indexPath

#Set the new (cloned) topology as active. This step takes the most time.

Set-SPEnterpriseSearchTopology -Identity $clone

#Discover the component info in the new topology.

Get-SPEnterpriseSearchComponent -SearchTopology $clone >c:\InterimTopology.txt

#Confirm that all the components are ready. Also check in the Central Admin Site for the status.

Get-SPEnterpriseSearchStatus -SearchApplication $ssa

Remove the older index components. ONLY DO THIS AFTER NEWLY ADDED INDEX COMPONENTS ARE ACTIVE. You need to clone the latest topology and modify the new cloned topology. Then set the new clone as active when modification is done. Double Check in the InterimTopology.txt the info on the components to identify the components to remove.

IndexPartitionOrdinal : 0
RootDirectory : E:\Search_Index01
ComponentId :
TopologyId :
ServerId :
Name : IndexComponent4
ServerName : SearchServer04

$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$newclone = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone -SearchTopology $active

$comp1=Get-SPEnterpriseSearchComponent -SearchTopology $newclone | ? {$_.Name -eq "IndexComponent1"}
$comp2=Get-SPEnterpriseSearchComponent -SearchTopology $newclone | ? {$_.Name -eq "IndexComponent2"}
$comp3=Get-SPEnterpriseSearchComponent -SearchTopology $newclone | ? {$_.Name -eq "IndexComponent3"}
$comp4=Get-SPEnterpriseSearchComponent -SearchTopology $newclone | ? {$_.Name -eq "IndexComponent4"}

Remove-SPEnterpriseSearchComponent -Identity $comp1 -SearchTopology $newclone
Remove-SPEnterpriseSearchComponent -Identity $comp2 -SearchTopology $newclone
Remove-SPEnterpriseSearchComponent -Identity $comp3 -SearchTopology $newclone
Remove-SPEnterpriseSearchComponent -Identity $comp4 -SearchTopology $newclone

Set-SPEnterpriseSearchTopology -Identity $newclone

# Optionally, you can use a for loop to run through a list of components to remove. But I prefer the lines above as it shows explicitly the actual action to take.

#Confirm that all the components are ready. Also check in the Central Admin Site for the status.

Get-SPEnterpriseSearchStatus -SearchApplication $ssa

#Discover the component info in the final topology.

Get-SPEnterpriseSearchComponent -SearchTopology $newclone >c:\FinalTopology.txt

#Remove the old topologies

Get-SPEnterpriseSearchTopology -SearchApplication $ssa
Remove-SPEnterpriseSearchTopology -Identity < > 

Getting Rid of GUIDs in DB names in an Exiting SharePoint Farm

DBAs hate the GUID appended to a database name. Everyone does! But that will happen if the farm and the Service Applications within are created with a wizard.

Example:

With_GUIDs

To make sure the database names are “clean”, farm admins can create the farm using psconfig.exe and create the Service Applications and Content Catabases with PowerShell. But what if a farm has been created and the databases already have GUID in their names? When you just take over a new farm, this may be the case, and you may be lucky enough to be assigned the task to change the names. 🙂

Just do the steps below to get rid of the GUIDs:

For the Central Administration site content database, you need to move the site to a new Content Database. Follow the steps:

  1. Create a new databases with proper names and configuration.
  2. Move the sites with Move-SPSite PowerShell cmdlet to move the sites to the new databases.
  3. Verify the sites are working correctly after the move.
  4. Delete the old databases.

For the user Content Databases, you can rename the databases. Follow the steps:

  1. Dismount the database from SharePoint Web Applications with Dismount-SPContentDatabase PowerShell cmdlet.
  2. Rename the dismounted database in SQL Server with ALTER DATABASE T-SQL command or in the Object Explorer.
  3. Mount the renamed database to SharePoint Web Applications with Mount-SPContentDatabase PowerShell cmdlet.

For Service Application databases, you need to rename the databases with additional actions and it is specific to each service application. Check this TechNet article to find information for the service applications you are using.

Example:

No_GUIDs

Done!

Getting Prompted for Login Repeatedly

If you encounter repeated login prompt even when the credential entered is correct, there are some common solutions that can help. If the issue persists after all these solutions are in place, systematic troubleshooting will be needed.

Here are the common areas to check, on the clients and the servers.

Client Side Solutions

Add the team Web site to the list of trusted intranet sites

To do this, complete the following steps:

  1. On the Internet Explorer toolbar, click Tools, and then click Internet Options.
  2. In the Internet Options dialog box, click the Security tab, and then select Local intranet.
  3. Click Sites, and then click Advanced.
  4. Type the URL of the team Web site in the Add this Web site to the zone box, click Add, and then click OK.

Bypass proxy server for local addresses

To bypass your Internet proxy for local addresses in Microsoft Internet Explorer 5 or later, complete the following steps:

  1. On the Internet Explorer toolbar, clickTools, and then click Internet Options.
  2. In theInternet Options dialog box, click the Connections tab, and then click LAN Settings.
  3. UnderProxy server, select the Bypass proxy server for local addresses check box, and then click OK.

Make sure Integrated Windows Authentication is enabled.

Make sure Integrated Windows authentication is enabled in IE. (Tools >> Internet Options >> Advanced >> under security, enable integrated authentication)

Add the entry to the Credentials Manager

  1. Go to Start > Run and type in control keymgr.dll to open the Windows key manager.
    Alternatevily: navigate to Contorl Panel > User Accounts > Manager Windows Credentials
  2. Select Add a generic credential
  3. Add yourSharePoint site URLlogin and password to the corresponding fields. If this entry already exists, edit it to have your login credentials.
  4. Reboot the computer.

If you are missing the Add button, you may want to modify Windows Registry to be able to save the password.

Note: for editing Windows Registry, administrator rights are required. Editing Windows Registry is not safe and users will perform it at their own risk.

  1. In Windows, go to Start > Run and enter regedit.
  2. Navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\.
  3. Find the DisableDomainCreds A value of 1(enabled) will prevent you from saving new credentials.
    Change the value to 0 and reboot. Now you should have the Add button available. Note that 0 is the default value.
  4. Also check the LmCompatibilityLevel It should be set to 3, which is the default value. If you have another value, change it to 3. If it does not work with 3, then also try it with 2.
  5. Reboot the computer to apply changes.

If the client PCs are using Windows 7, some hotfixes may be needed:

https://support.microsoft.com/en-us/kb/943280

The solutions below are specific to the “Open with Explorer” function. If you get prompted for login repeatedly when using “Open with Explorer” after trying the configurations above, you can try the solutions below.

Restart WebClient Service

  1. Click Start > Run
  2. Enter ‘services.msc’
  3. Find the WebCient service and select Restart

Check Internet Explorer Version

  • Windows 7: Internet Explorer 10 is not yet compatible with the You will need to revert to Internet Explorer 9.
  • Windows 8: Internet Explorer 10 is compatible, so you should not have an issue with this OS and browser version combination.

Server Side solutions

Specify Host Names on each SharePoint Web Front End. (Preferred method over disabling loopback check)

To do this, follow these steps for all the nodes on the client computer:

  1. Click Start, click Run, type regedit, and then click OK.
  2. In Registry Editor, locate and then click the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0

  1. Right-click MSV1_0, point to New, and then click Multi-String Value.
  2. Type BackConnectionHostNames, and then press ENTER.
  3. Right-click BackConnectionHostNames, and then click Modify.
  4. In the Value data box, type the host name or the host names for the sites that are on the local computer, and then click OK.
  5. Quit Registry Editor, and then restart the IISAdmin service.

Client and Server Coordination

Make sure the NTLM Level is the same among Domain Controllers, SharePoint Servers and Clients

This can be pushed down from a Domain Controller using GPO (Group Policy Object).

  1. Use Group Policy Editor (GPE) to open the Group Policy Object (GPO) you want to modify. You can create a policy that applies to the OUs that contains the DCs, SharePoint Servers, and user clients.
  2. Navigate to Computer Configuration, Windows Settings, Security Settings, Local Policies, Security Options.
  3. Double-click the “Network security: LAN Manager authentication level” policy.

You can choose a level that is acceptable to your internal security policy, Just make sure you use the same level across DCs, SharePoint Servers and clients.

  1. Select “Define this policy setting” and from the drop-down menu select the desired level.
  2. Click OK.
  3. Close the GPO.
  4. To make the modification effective immediately, run gpupdate /force on all the servers and clients.

One Single IP:port for Multiple SSL Enabled Sites, each with a Different SSL Certificate

In a real-world production environments, there can be multiple web applications in the same farm that use different host headers but the same port, which is often 80/443. This is not an issue when SSL is not implemented. When the request reach the web server, IIS will direct the request to specific sites based on the host header. However, if SSL is to be implemented and all the sites will be using the same port, there will be a problem with the certificate as network end-point can only be identified with IP:Port binding. In the old days, two possible solutions are:

  • Use two IPs on the same server, so that each IIS site can use a different certificate. This is should not be too difficult when the WFEs are virtualized.
  • Use a wildcard certificate so all the web applications can use the same certificate as long as they share the same domain.  If not, you may want to choose a Unified Communications Certificate if that really is a preferred solutions in your situation.

With Windows Server 2012 or above, you have the third option that comes with the Server Name Indication feature of IIS 8. This feature makes it possible that the host name in the IIS binding be used to identify the network endpoint. In this case, different IIS sites can use the same IP address and port while using different certificates. However, it requires the client browsers to support SNI. If not, you need to create a default SSL site. Check this blog post for more information: http://blogs.msdn.com/b/kaushal/archive/2012/09/04/server-name-indication-sni-in-iis-8-windows-server-2012.aspx

Why you need to host Intranet and Internet sites in separate farms?

If you have both Intranet sites and a public facing portal, it is usually recommended to host them separately in their own farms instead of sharing the same farm. Some may argue that these two types of site can actually be hosted on the same farm as long as they are separate web applications using different application pools and even separate instances of the same service applications. However, there still are factors to be considered:

  • Security – The servers of Internet-facing website are close to external access. It is a recommended security practice to separate servers hosting intranet from those serving internet-facing websites. In this case, intranet farms servers could be placed in an internal network zone with no internet access.
  • Performance optimization – Intranet sites and internet-facing web site have different usage patterns. Internet-sites are primarily read-intensive, while intranet sites have more user updates. The configuration for these two types of environment are different. For example, in database server hosting content databases, the transaction log files of write-intensive databases should be put in a faster drive than that of the data files. For read-intensive databases, the configuration is quite the other way.
  • Scalability – The growth pattern and scaling requirement is usually different between intranet websites and internet websites. Take the sizing of one of my customers’ SharePoint farm as an example, the intranet portal has a content database of more than 50 GBs while the total size of each of the internet web applications is only around 1 GB. The user traffic growth pattern may be different as well. The intranet traffic is basically predictable based on the growth of internal users. That of the internet farm may change quite drastically if the website becomes more and more popular.
  • Maintainability – The maintenance schedule is usually different between internal intranet and internet-facing sites. Farm maintenance task often involves, solution deployment, IIS reset and even server reboot. Separating sites into two farms will ensure maintenance on Intranet farm does not impact the internet-facing websites. What is more, internet-facing sites have the most stringent SLA. Correspondingly, a dedicated high availability and maintenance schedule should be implemented.