Category Archives: Exchange Server

CreateCluster() failed with 0×5 adding members to DAG in Exchange 2013

  By Bhargav | November 26, 2012 - 1:39 pm | ,
6 Comments

UPDATE: While TechNet article “Create a Database Availability Group” only mentions Windows 2008 R2 domain controllers, I must thank Scott Schnoll for the following clarification: “Creating an Exchange 2013 DAG with Mailbox servers on Windows Server 2012? You must pre-stage CNO before adding first server!”. TechNet documentation will be updated to reflect the same I assume.

If you are in IT long enough, you know the fact that nothing will every work without throwing an issue or two you have to solve. Especially if you are dealing with recently released software such as Exchange 2013.

In my lab, I had installed all Exchange servers I needed in different sites. All my domain controllers and Exchange servers are running Windows Server 2012. Since I am not testing co-existence, it is a green field deployment.

Since everything so far was working as expected, I proceeded with creation of DAG. From EAC, creating DAG itself worked with no issues. I then went ahead and added first mailbox server to DAG. this step, however, refused to complete with error:

A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API ‘”CreateCluster() failed with 0×5. Error: Access is denied”‘ failed.. [Server: MBX1.fabrikam.int]

Looking at CNO, I noticed Exchange Trusted Subsystem had special permissions assigned and not “Full Control” on CNO that the process created automatically.

Assigning “Full Control” to Exchange Trusted Subsystem on CNO, I assumed should fix the issue, however, it actually produced a completely different error when I tried to add the mailbox server to DAG again:

An Active Manager operation failed with a transient error. Please retry the operation. Error: The fully qualified domain name for node ‘DAG1′ could not be found.

I also noticed mailbox server account with similar permissions and not “Full Control” on the object.

From reading TechNet forums I knew a solution existed where you can just delete DAG CNO and pre-stage it as described in TechNet article: Pre-Stage the Cluster Network Object for a Database Availability Group.

I was able to fix the issue using one of the following two methods:

  1. Disable CNO, assign “Full Control” to ETS on the DAG object and remove mailbox server from permissions list on CNO. Add mailbox server to DAG.
  2. Delete CNO from AD and pre-stage CNO using process described in the article mentioned above. Add mailbox server to DAG.

I am not sure if it is a bug or known issue with Server 2012 domain controllers. It remains to be seen as more guidance become available from Microsoft.

If you are wondering why the last lines above are striked out, please read the update at the top of the page for explanation.

Print Friendly

Exchange 2013 Setup – Client Access server role readiness check fails

  By Bhargav | November 16, 2012 - 4:18 pm | ,
Leave a comment

When installing Exchange 2013 servers in complex lab scenario, I decided to separate Mailbox and Client Access server roles to dedicated servers running Windows Server 2012. I installed Mailbox server role first and then proceeded to install Client Access role. Since Mailbox server role installed fine using setup UI and required windows components installed (since I had the option checked in the wizard, I did have to install office filter packs and UCMA manually), I decided to trust the setup wizard to install windows components for CAS role for me. Well, what do I know?

To my surprise, the setup halted on prerequisite analysis step complaining that UCMA wasn’t installed.

image

I did have to install UCMA myself so I was actually happy to see this and proceeded to install UCMA manually as I had also done on Mailbox server. That’s when I got bad news from UCMA setup. UCMA setup complained about missing Media Foundation which is a pre-requisite for UCMA installation and should have been installed with required windows components that I assumed setup UI installed.

image

I didn’t need to investigate as it was obvious UI didn’t install required pre-requisites. I was however perplexed that setup didn’t do what is should have done. Could it be an anomaly? Am I missing something? I didn’t have internet access on that lab VM. The setup could not check for update. Was this fixed in an update I am not downloading? To rule that out, I provided internet access to that VM and ran setup again, this time selecting update option. Setup quickly came back to tell me there were no updates available.

I haven’t tried unattended mode of setup yet. So I decided to try that with an obvious /InstallWindowsComponents switch. As you probably know by now, guess where that got me… NOWHERE!

The only option I was left with was to install pre-requisites manually as mentioned in TechNet article: http://technet.microsoft.com/en-us/library/bb691354(v=exchg.150).aspx#WS2012CAS. Everything was good after that.

It remains to be seen if this will change in future with setup updates. Till then, I gotta have a setup script like one I have for Exchange 2010. Hurry up Pat Richard. Smile

Print Friendly

Outlook must be online or connected…

  By Bhargav | November 14, 2012 - 12:08 pm | , ,
15 Comments

Testing Exchange 2013 means careful planning of what you are going to test in lab, what you need to test al given scenarios and what is required. One of my golden rule is to test multiple versions of clients against given server. So in my recent configuration testing, I created a very simple lab. It has a domain controller, two Exchange 2013 servers, a load balancer and two client machines. All testing I am performing is on intranet and no internet simulation or remote client scenarios are going to be tested. One of my client machine is running Windows 7 and Outlook 2010. The other client runs Windows 8 and Outlook 2013.

After I completed setup of my first Exchange 2013 server and load balacer, I decided to test the clients before diving into setting up second server and DAG.

First test was on Outlook 2010 client. I fired up the Outlook client and proceeded to usual profile configuration wizard at first launch. The wizard found settings and proceeded to last step of logging on to the server. To my surprise, it took longer than usual. Everything was configured correctly so I decided to wait. It took about 40 seconds before Outlook choked on an error: “the connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action.”

image

Clicking ok took me to next and somewhat familiar screen of profile setup, showing Exchange server and mailbox information. Not surprisingly, for exchange server, it shows guid@smtpdomain of user mailbox. This is new to Exchange 2013. Exchange 2013 creates a connection point comprised of mailbox GUID @ primary smtp address in effort to nearly eliminate the familiar “Your administrator has made a change to your mailbox. Please restart.” message. Refer to “Exchange 2013 Architecture section of this TechNet article (http://technet.microsoft.com/en-us/library/jj150540(v=exchg.150).aspx) for more details.

image

Now I have to figure out how to make this work since Outlook 2007 and above is supported client versions for Exchange 2013.

Since I had second client machine with Outlook 2013 handy, I proceeded to first test the differences between the clients and see if that made any difference in the initial experience. If I were to tell you that it just worked without the error I was getting in Outlook 2010, will you be surprised? Smile

image

At this point, it was clear that the issue wasn’t my server or load balancer configuration. The issue was certainly local to Outlook 2010. So on to trustee Bing.com… and what did I find? Outlook 2010 update released on April 2012. While Microsoft knowledge base article KB2596959 referenced in the update doesn’t exactly map to the issue I was having, it caught my attention as it mentioned GUID@DOMAIN.COM.

I immediately downloaded the update and applied it to Outlook 2010 client.

Now, will you be surprised if I tell you that Outlook 2010 had no problem setting up profile correctly after I applied this update? Winking smile

Print Friendly

If Retention Tags weren’t confusing…

  By Bhargav | June 8, 2012 - 12:23 pm |
2 Comments

UPDATE: Please read updated guidance due to changes introduced in SP2 RU4 here: http://blogs.technet.com/b/exchange/archive/2012/08/14/calendar-and-tasks-retention-tag-support-in-exchange-2010-sp2-ru4.aspx. It specifically addresses Calendar items and tasks.

I wouldn’t be talking about it today!

When the topic came up with my colleague, it quickly became a confusing discussion of what’s what, what works, what doesn’t and why TechNet seems to say something that doesn’t work.

So let’s start with this article: “How Retention Age is Calculated”. This clearly states how age is calculated on Calendar items, Tasks, Contacts and other items. Then it immediately mentions in the Note that retention tags aren’t supported for the Calendar and Tasks. The Managed Folder Assistant doesn’t process items in these folders.

This is really what is a start of confusion. The article “Understanding Retention Tags and Retention Policies” doesn’t mention the same fact (probably to avoid duplication).

The confusion is evident when Bharat specifically spells out in EHLO article “Prevent archiving of items in a default folder in Exchange 2010” that Calendar, Tasks and Contacts folders aren’t supported.

However, above all, the confusion prevails! So let’s clear things up a little bit.

Exchange 2007 feature Managed Folder Policies, aka MRM 1.0 functionality supports Calendars, Tasks etc. However, it has its limitations. MRM 2.0 functionality provides more flexibility so individual folders can have different retention either applied to mailboxes using policy or by user using tags on folders or messages individually. Not to mention, at the cost of being able to process Calendar and Tasks along with Notes and Contacts. In SP1, Notes are processed, however, don’t expect more than that!

So if you want to process Calendar items, Tasks or Contacts, you should use Managed Folders. It does mean you won’t be able to apply one policy to one folder (i.e. Inbox) and different policy to another (i.e. Calendar). It also means that you can’t archive enable the mailbox that has Managed Folders policy applied or you can’t apply Managed Folders Policy to mailbox that has archive mailbox. If that is clear, I assume you know by now that Managed Folder Policy can’t move anything to archive mailbox. Although, never assume, so previous statement should remove my assumption and clearly tell you what it is!

Conversely, if you apply Default Policy Tags or Retention Policy Tags to a mailbox, Managed Folder Assistant will not process unsupported items. Namely, Calendar, Tasks, Contacts and Notes in Exchange 2010 RTM and Calendar, Tasks and Contacts in Exchange 2010 SP1.

Also worth noting is that you can either apply Managed Folders Policy or Retention Policy Tags (DPT/RPT whatever you call it) but not both to a mailbox. You can apply Managed Folders Policy to one mailbox and Retention Policy Tags to another in same Exchange 2010 database as these settings are per mailbox.

Oh and that “How Retention Age is Calculated” post isn’t incorrect. It correctly states how the age is calculated. Just understand it only applies to Managed Folder Policy.

Clear a mud? Perfect!

Originally posted at http://blogs.technet.com/bshukla

Print Friendly

Exchange Management Shell Error 500 – Internal Server Error

  By Bhargav | May 3, 2012 - 9:05 pm | ,
Leave a comment

I have come across this issue enough times that even if it is documented on TechNet it deserves mention here.

When you launch Exchange Management Shell or try to connect to an Exchange 2010 Server remotely using PowerShell, you get error “500 – Internal Server Error. There is a problem with the resource you are looking for, and it cannot be displayed.”

Error details also show the following:

For more information, see the about_Remote_Troubleshooting Help topic.

    + CategoryInfo          : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [], PSRemotingTransportException

    + FullyQualifiedErrorId : PSSessionOpenFailed

The other possible errors you may see are the following:

The WinRM client cannot process the request. It cannot determine the content type of the HTTP response from the destination computer. The content type is absent or invalid. For more information, see the about_Remote_Troubleshooting Help topic.

Or

Connecting to remote server failed with the following error message: The WinRM client received an HTTP server error status (500), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic. It was running the command 'Discover-ExchangeServer -UseWIA $true -SuppressError $true'.

Or

The WinRM client received an HTTP status code of 403 from the remote WS-Management service.

All of these issues relate to a problem with PowerShell virtual directory on given server not configured properly. If you were to run Exchange Best Practices Analyzer, it alerts about this issues as well.

The resolution is well documented on TechNet article “PowerShell Virtual Directory issues cause problems with Exchange Management tools”. I will let you read the solution there, however, I wanted to mention the oddity in my case.

Looking at the error I was getting and mapping it to solution in article didn’t resolve the issue. I had to configure kerberos authentication as mentioned in the article. Once KerbAuth was registered as native module, EMS and remote PowerShell sessions started working.

What my friend mentioned seems so relevant: “Why can’t our lives be just as predictable as computers? While there are some problems in a day, most of it is logical and if you know that logic, you can predict the outcome or fix the issue!”. Amen to that my friend.

Originally posted at http://blogs.technet.com/bshukla

Print Friendly

Certificate revocation checked failed

  By Bhargav | April 30, 2012 - 1:06 pm |
Leave a comment

Recently I came across a CAS server that was rebuilt. Think of it as a new server you are introducing in your environment.

 

Everything looked good except certificate that we imported. The certificate looked good when looking at validity, issuing authority certificate and other dependencies. However, Exchange Management Console complained:

“The certificate status could not be determined because the revocation checked failed.”

Since the error seemed clear enough, we checked and verified that we can reach CRL. We could successfully access it and download CRL. We also ensured that there was no proxy servers configured or required, which they weren’t.

However, the server had its own mind.

KB979694 wasn’t applicable since there was no proxy in the environment.

The only logical thinking here was, why is “Local System” account (which the service uses to get the revocation status) unable to get to CRL. To get to the answer, we needed to check proxy settings of Local System account. How do you do that? You can’t simply start IE as different user!

That exactly is the purpose of this post. I found bits and pieces of information that helped me resolve the issue but not a one step document. In this post, I am trying to put it all together so you have one stop solution.

Here’s how you can fix the issue:

  1. Open up command prompt as Administrator
  2. Run “sc create testsvc binpath= "cmd /K start" type= own type= interact”
    • This creates testsvc service which will run as local system and allow interaction with desktop
  3. Run “sc start testsvc”
    • The error “[SC] StartService failed 1053” is expected and can be ignored safely
  4. Locate “Interactive Services Detection” icon blinking in the taskbar and click “view message”
  5. You are now in a command prompt window running as Local System and you will not see your desktop. The only other visible window is “Interactive Services Detection” window.
  6. Launch Internet Explorer using the following command:
    • "c:\Program Files (x86)\Internet Explorer\iexplore.exe"
  7. Internet Explorer may present Set up window. If it does, click “Ask me later”.
  8. We will now check proxy settings. Go to Tools -> Internet options -> Connections -> Lan Settings.
  9. Verify proxy and automatic configuration options and change them to match your environment. In my case we cleared all checkboxes since no proxy existed in environment.
    • In our case, either server build process or a setting from or a GPO was populating incorrect proxy settings.
  10. Close Internet Explorer window and return to command prompt.
  11. We will now clean certutil caches.
  12. Run “certutil -urlcache ocsp delete”
  13. Run “certutil -urlcache crl delete”
  14. We’re almost done here. We now have to close and exit out of service.
  15. Type “exit” and press enter to close command prompt that is running as Local System.
  16. Now you should have only one “Interactive Services Detection” window.
  17. Click “Return Now”.

You are now back to your desktop and we have corrected Internet Explorer settings for Local System (removing proxy configuration that was incorrect). After this, we restarted Exchange Management Console and verified certificate on CAS server in question. Certificate was no longer issuing the warning and we proceeded with assigning the certificate to appropriate services.

It is important to note that refresh time varies from immediate to more than few minutes so don’t fret over certificate still showing the same error. If, however, it takes more than 15 minutes, I would check if all steps were followed as mentioned above and configuration is correct for your environment.

Yet another issue put to bed. On to another.

Originally posted at http://blogs.technet.com/bshukla

Print Friendly

RBAC and Principle of Least Privilege

  By Bhargav | December 15, 2011 - 12:05 pm | ,
Leave a comment

Exchange 2010 introduced RBAC as a mechanism to manage access to administrative tasks at granular level which was not possible in previous versions of Exchange.

While you may know how to use RBAC to create custom roles that maps to job functions in your environment, one particular feature tends to get easily overlooked, mostly because it is least understood I believe. It is Unscoped Top Level Management Roles.

So, I wrote a blog post on it detailing what it is, and how to configure it. It went live few days ago at Hey, Scripting Guy! blog.

You can read complete article here – http://blogs.technet.com/b/heyscriptingguy/archive/2012/01/13/use-powershell-and-rbac-to-control-access-to-exchange-server-cmdlets.aspx

Enjoy!

Originally posted at http://blogs.technet.com/bshukla

Print Friendly

Updated – Verify Exchange Server Schema Version

  By Bhargav | December 15, 2011 - 5:30 am |
Leave a comment

This article was originally posted on my personal blog here. Since I don’t actively maintain it anymore, I am publishing it here.

When you run Exchange Setup to prepare schema, usually the very next question is, how do I verify schema was updated successfully? Verifying only the values of attributes as mentioned below is not a good verification of Exchange setup completion. This article is intended to only provide reference to attributes and their values.

Let’s start back at Exchange 2003 SP2.

One of the last actions setup /forestprep in Exchange 2003 is to set objectVersion attribute on Exchange organization container to a value of 6903. You can verify this using ADSIEdit and navigating to Configuration NC, Exchange organization object under services\Microsoft Exchange node.

On the other hand, when setup /domainprep is run, it sets the objectVersion attribute on Microsoft Exchange System Objects container to a value of 6936. You can verify this using ADSIEdit and navigating to Domain NC, Microsoft Exchange System Objects container.

In Exchange 2007, after successful run of Setup /PrepareSchema you will find that the attributes mentioned above are not changed! You need to verify the value of rangeUpper attribute of ms-Exch-Schema-Version-Pt object in Schema NC. The value should be 10637.

It is only when you run Setup /PrepareAD the objectVersion attribute of Organization container in Configuration NC is updated to a value of 10666. You will also find that objectVersion attribute on Microsoft Exchange System Objects container in Domain NC is set to a value of 10628.

You will also notice that Setup /PrepareDomain does not have any effect on these attribute values.

Let’s briefly review what does Exchange 2007 SP1, SP2 and Exchange 2010 setup update these attribute values to.

Exchange 2007 SP1

  • Value of rangeUpper attribute of ms-Exch-Schema-Version-Pt object in Schema NC is set to 11116 when setup /PrepareSchema is run successfully.
  • Setup /PrepareAD sets the objectVersion attribute of Organization container in Configuration NC is updated to a value of 11221. objectVersion attribute on Microsoft Exchange System Objects container in Domain NC is also set to the same value of 11221.
  • Setup /PrepareDomain does not have any effect on these attribute values.

Exchange 2007 SP2

  • Value of rangeUpper attribute of ms-Exch-Schema-Version-Pt object in Schema NC is set to 14622 when setup /PrepareSchema is run successfully.
  • Setup /PrepareAD sets objectVersion attribute of Organization container in Configuration NC to a value of 11222. objectVersion attribute on Microsoft Exchange System Objects container in Domain NC remains unchanged at value of 11221.
  • Setup /PrepareDomain does not have any effect on these attribute values.

Exchange 2007 SP3

  • Value of rangeUpper attribute of ms-Exch-Schema-Version-Pt object in Schema NC is set to 14625 when setup /PrepareSchema is run successfully.
  • objectVersion attribute of Organization container in Configuration NC remains unchanged at a value of 11222. objectVersion attribute on Microsoft Exchange System Objects container in Domain NC remains unchanged at value of 11221.
  • Setup /PrepareDomain does not have any effect on these attribute values.

Exchange 2010

  • Value of rangeUpper attribute of ms-Exch-Schema-Version-Pt object in Schema NC is not changed from 14622 when setup /PrepareSchema is run successfully.
  • Setup /PrepareAD sets objectVersion attribute of Organization container in Configuration NC to a value of 12640. objectVersion attribute on Microsoft Exchange System Objects container in Domain NC remains unchanged at value of 12639.
  • Setup /PrepareDomain does not have any effect on these attribute values.

Exchange 2010 SP1

  • Value of rangeUpper attribute of ms-Exch-Schema-Version-Pt object in Schema NC is not changed from 14726 when setup /PrepareSchema is run successfully.
  • Setup /PrepareAD sets objectVersion attribute of Organization container in Configuration NC to a value of 13214. objectVersion attribute on Microsoft Exchange System Objects container in Domain NC is changed to value of 13040.
  • Setup /PrepareDomain does not have any effect on these attribute values.

Exchange 2010 SP2

  • Value of rangeUpper attribute of ms-Exch-Schema-Version-Pt object in Schema NC is changed to 14732 when setup /PrepareSchema is run successfully.
  • Setup /PrepareAD sets objectVersion attribute of Organization container in Configuration NC to a value of 14247. objectVersion attribute on Microsoft Exchange System Objects container in Domain NC remains unchanged at value of 13040.
  • Setup /PrepareDomain does not have any effect on these attribute values.

When reading this article, consider the fact that the lab setup I used was upgraded from Exchange 2003 schema to Exchange 2007 schema and then to Exchange 2010/SP1 schema. Service Pack 2 was tested in Exchange 2003 environment with no Exchange 2007 or Exchange 2010 Service Pack 1. This should not affect any attribute values mentioned above however I cannot guarantee since I have not tested it.

Originally posted at http://blogs.technet.com/bshukla

Print Friendly

New pre-requisites for Exchange 2010 Service Pack 2 and CAS Role

  By Bhargav | December 6, 2011 - 5:56 am |
1 Comment

With release of Service Pack 2 for Exchange Server 2010, you gain few new features such as Cross-Site Silent Redirection for OWA, Address Book Policies, Mailbox Auto-Mapping and few other additions (What’s new in Exchange 2010 SP2).

With it, comes new pre-requisites if you are installing/updating Client Access Server (CAS) role.

You will need to install the following components on the server that will be running CAS role (or existing CAS you are planning to update):

ISAPI Filters – Web-ISAPI-Filter
IIS 6 WMI Compatibility – Web-WMI
ASP.Net – Web-Asp-Net

You can install them as described in Exchange 2010 Prerequisites article. If you want to install these components on existing CAS server before upgrade to SP2, you can launch PowerShell as Administrator and then run:

Import-Module ServerManager
Add-WindowsFeature Web-ISAPI-Filter,Web-WMI,Web-Asp-Net

Note this post refers only to Windows 2008 R2. I haven’t checked if requirements are different for Windows 2008 (without R2) servers. They are, however, detailed in the TechNet articles linked above.

Enjoy!

Originally posted at http://blogs.technet.com/bshukla

Print Friendly

Script to configure static ports on Exchange Server 2010

  By Bhargav | October 21, 2011 - 12:45 pm | ,
Leave a comment

There is nothing new about this. If you have been reading about Exchange Server 2010 or have it deployed with hardware load balancer, chances are, you have read how to configure static ports on Exchange Server 2010 on TechNet Social wiki for Exchange 2010. Chances are that you have also used my script (referenced in the post above) to set static ports on your servers. Lastly, chances are that you have read all about it on my previous post here.

If so, why am I even talking about it today?

Well, if you haven’t noticed a few things already, the way you change ports is different in RTM and SP1. My script didn’t account for SP1 originally when it was written. Was SP1 even existed then?

The other reason is my nature of always learning something and making things better! I noticed how my code was inefficient now that I know a few more things about PowerShell (yeah that’s not funny). I decided to write it more efficiently and that basically meant a complete overhaul of my old script.

The new script is now more user friendly! It uses cmdletbinding and comment based help. It means, for you as a user, you can just type:

Get-Help Set-StaticPorts.ps1 –examples

or

Get-Help Set-StaticPorts.ps1 –Full

The script now validates parameters using ValidateRange and ValidateScript. I think that’s cool! It also uses 59531 and 59532 by default now. How about using recommended ports instead of random ones I used in my previous script? I think that’s even more cool!

The script uses all the right write-* cmdlets now instead of write-host. So now you can use tee-object and won’t end up with empty output file. Yes you loose cool colors I used with write-host but hey, you are trying to set ports on your Exchange Server 2010. For colors you would go see Macy’s Fireworks on New Year, right? Smile

Oh and last but probably the most important change is inclusion of –auto and –whatif functionality!

-WhatIf is obvious. Script will tell you what it is doing without actually making any changes.

-Auto will automatically find all your Exchange 2010 CAS servers and Exchange 2010 Mailbox servers that are hosting Public Folders. It will then change ports on CAS Server for RPC CA service and Exchange AB service. On Mailbox servers it will only change RPC CA ports as Exchange AB service doesn’t exist on Mailbox only role.

If you combine all this with –Force, you can also silence the script. It won’t ask you for any confirmation and will change ports you specify (or use defaults) and restart the services! Isn’t that awesome!

So go download the script from here: Set-StaticPorts.ps1 and give it a go. As always, let me know if you find any issues and I will be happy to fix it.

Originally posted at http://blogs.technet.com/bshukla

Print Friendly