Connecting on-premise SharePoint to Exchange Online

My company is in the process of migrating our on-premise email system (Exchange 2003!) to BPOS/O365/Exchange Online/other acronyms here. Since we use a slightly non-standard email routing system with SharePoint, I'd been having some difficulties figuring out how all the pieces would come together to make this happen. O365 Support has been...well...there.

Before I get into the particulars, here's the basics of our configuration:

Web-frontend: PORTAL.company.com
Index/app server: INDEX.company.com

We currently receive mail at @company.com, which has been floated up to Office 365. Our email-enabled lists receive mail at @portal.company.com, and we create Contacts for those addresses in Exchange, but we don't publish them in the Global Address List. Instead, we create a Distribution Group and put the Contact in there, and then provide the Distribution Group to the users, so that if we need to change the address in SharePoint, we can do that somewhat painlessly.

PORTAL doesn't actually process the mail. As a performance measure (probably a very tiny one, but I digress), we actually process inbound mail on INDEX. PORTAL receives the mail, but a scheduled job on PORTAL shuttles the mail into the Drop folder on INDEX every minute, where the Microsoft SharePoint Foundation Incoming E-Mail service runs. Alerts and other notifications are also sent from INDEX.

Now that this is all out of the way, here's the situation. We don't publish an MX record for PORTAL.company.com - instead, our on-premises Exchange server would handle all mail for *.company.com and route PORTAL.company.com messages appropriately. This way, nobody from the outside world can spam our SharePoint farm directly.

In order to set this up, we had to set up a new inbound connector and outbound connector for SharePoint in FOPE (Forefront Online Protection for Exchange). If you're the SharePoint monkey and haven't really gotten into the depths of Office 365, you can access FOPE by logging into your O365 account, then in the top-bar (assuming you're an Office 365 Administrator), click Home, then Admin (Admin is not directly accessible from the mail window).

On the Admin Overview page, in the main content area under the header "Microsoft Office 365", look for the Exchange section, and click Manage. This opens up a new screen which is the Exchange Online administration screen. In the menu on the left, you should be on "Users & Groups". Click "Mail Control". After the page loads, on the far righthand side, you should see, under the header "Additional Security Settings" the link for FOPE. Click this link and you'll be in FOPE.

You may get a notification that your session has expired. This happens a lot. If so, *close Internet Explorer completely*. You'll need to log into Office 365 all over again, even if you just started your Office 365 session from scratch. It appears that FOPE tracks its sessions independently and doesn't clear them on exit, so when you load FOPE and it finds an expired session, it forces you to load all over again.

Anyway. Now you're in FOPE.

In the context of FOPE, "inbound connectors" mean mail inbound TO the server. So, from a SharePoint context, you'll configure an "inbound connector" for your outbound mail, and for your mail incoming to SharePoint, you'll configure an "outbound connector".

Let's do inbound first. A wrinkle, however.

In the past, SMTP on INDEX didn't need to authenticate itself to Exchange by virtue of already being inside the network. This is probably bad practice in a large organization, but since we're a small shop and only have  one Exchange server, it's easily monitored for nefarious activities.

Microsoft will obviously not be so trusting, so we need to configure a secure connection to allow for our server to deliver messages to Office 365. This actually requires us to add an additional user to Office 365 for the purposes of being the delivery proxy for SharePoint's email. So...one E1 license later, and I now have a username and password for use with the connector (I also configured this account to route all of its incoming mail to the SharePoint Administrators distribution group so that if anyone replies to an alert or such, we'll all see it).

On INDEX, IIS 6.0 Management, right-click on your SMTP server, select Properties. Go to the Delivery tab, and we'll be changing settings on all three of the buttons at the bottom.

First, Outbound Security.

If you aren't already using TLS, you will need to be now, so be sure to enable that. You'll also enter the credentials for your new Exchange Online user.

Next, Outbound Connections.

Make sure to change the TCP port since you'll now be using TLS. If you need to punch holes in facesfirewalls, now is the time to do so.

Last, Advanced Delivery.

The Smart Host field will have your client-specific smarthost. You'll probably recognize it with the "pod" prefix, but if that changes in the future, you can locate the smarthost information through the Office 365 page in the POP/IMAP/SMTP settings shown in the connection information.

 

Once you've made the SMTP changes, you'll also need to change SharePoint's mail settings to match.

Now that you've done things on the SharePoint side, time to set up FOPE. Note that FOPE doesn't have to be done last, I just saved it for last.

First, the Inbound Connector (remember - outbound from SharePoint!)

And then the Outbound Connector:

Once these connectors are configured, you will now need to Enforce them in FOPE. Enforcing the connectors means that the changes you've set up will take effect.

After you do this, test your mail setup, because you should be done!

Content approval for folders

Due to unfortunate necessity, I'm having to take our SP2010 branding apart bit-by-bit, redo the solution, and get dragged screaming and kicking into the 21st century of SP2010 development (IE - VS2010 on a server, no more STSDEV). I've certainly had my share of frustrations learning what everyone already knows here, but I discovered a new item that I'd only seen one other person mention anything about. I'm adding masterpages to the site's Master Page Gallery in the solution. Nothing big there. As someone with branding experience might already know, sometimes the masterpages won't be content-approved when you upload them, meaning that your event handler needs to be able to check them in on-the-fly. Nothing big there.

                using (SPSite site = (SPSite)properties.Feature.Parent)
                {
                    SPList masterPageGallery = site.GetCatalog(SPListTemplateType.MasterPageCatalog);
                    SPFolder masterPageTargetFolder = masterPageGallery.RootFolder.SubFolders[themeName];

                    foreach (SPFile file in masterPageTargetFolder.Files)
                    {
                        SPListItem listItem = file.Item;

                        if (!listItem.HasPublishedVersion)
                        {
                            listItem.File.CheckIn("Automatically added by installation feature.", SPCheckinType.MajorCheckIn);
                            listItem.File.Update();
                            listItem.File.Approve("Automatically added by installation feature.");
                            listItem.File.Update();
                        }
                    }
                }

Now, it turns out that since I'm putting the masterpages for this theme in their own folder, the folder also is subject to content type approval (not that this is relevant to the masterpages being accessible). So, let's say we want to be good citizens and have that folder automatically checked in when the code runs. Well, how do you do that? There's no .CheckIn() method on SPFolder or SPListItem...and if you think you'll be clever and try masterPageTargetFolder.Item.File.CheckIn(), the File member is null, so you'll find yourself quickly dashed. I found this post by Jay Noirfalise that got me part of the way there (using SPModerationInformation), but it didn't quite work.

                    SPList masterPageGallery = site.GetCatalog(SPListTemplateType.MasterPageCatalog);

                    SPFolder masterPageTargetFolder = masterPageGallery.RootFolder.SubFolders[themeName];
                    SPListItem masterPageTargetFolderItem = masterPageTargetFolder.Item;
                    SPModerationInformation moderationInformation = masterPageTargetFolderItem.ModerationInformation;
                    moderationInformation.Comment = "Automatically added by installation feature.";
                    moderationInformation.Status = SPModerationStatusType.Approved;
                    masterPageTargetFolderItem.Update();

For starters, it looked a little off to me. If I'm instantiating a SPModerationInformation object and then setting properties there, don't I need to save that object back to the SPListItem to effect the change? I tried to verify this in PowerShell and my suspicions were confirmed - nothing changed when I did this. Maybe it worked like that in WSS/MOSS? I dunno. Further, the SPListItem.ModerationInformation member is read-only. So, you just need to drill into the properties and set them inside the ModerationInformation member, like so:

                    SPList masterPageGallery = site.GetCatalog(SPListTemplateType.MasterPageCatalog);

                    SPFolder masterPageTargetFolder = masterPageGallery.RootFolder.SubFolders[themeName];
                    SPListItem masterPageTargetFolderItem = masterPageTargetFolder.Item;
                    masterPageTargetFolderItem.ModerationInformation.Comment = "Automatically added by installation feature.";
                    masterPageTargetFolderItem.ModerationInformation.Status = SPModerationStatusType.Approved;
                    masterPageTargetFolderItem.SystemUpdate(false);

But it still doesn't work. After running the matching code in PowerShell, my folder is still "Pending". You might notice one change I made which breaks this whole thing. It makes a little sense if you think about it - the approval's supposed to be done by a human, so we can't use SystemUpdate(false) to effect the change, because then there's no record of who did the change.

                    SPList masterPageGallery = site.GetCatalog(SPListTemplateType.MasterPageCatalog);

                    SPFolder masterPageTargetFolder = masterPageGallery.RootFolder.SubFolders[themeName];
                    SPListItem masterPageTargetFolderItem = masterPageTargetFolder.Item;
                    masterPageTargetFolderItem.ModerationInformation.Comment = "Automatically added by installation feature.";
                    masterPageTargetFolderItem.ModerationInformation.Status = SPModerationStatusType.Approved;
                    masterPageTargetFolderItem.Update(true);

Use Update() or SystemUpdate(true), and you're all set.

Dealing with orphaned user solutions

I'm surprised this problem isn't more common. It seems easy enough to trigger by accident, and fixing it was a puzzle.

No doubt most of us have at some point in using SharePoint 2010 created a site template, and then after gazing upon it in the Solution Gallery, determined that we've named it wrong, need to change one small thing, or any number of different issues. Normally, the correct thing to do in this circumstance is to deactivate and then delete the solution.

What if you don't deactivate it?

Create a site.
Save it as a template to the Solution Gallery.
Download the WSP to your desktop.
Delete the solution from the Solution Gallery *without* deactivating it. To do this, click the Edit button and then click Delete from there.
Upload the WSP back to the Solution Gallery.

You'll find that you're not able to activate the WSP you just uploaded, or deactivate it. And now you're just kinda stuck with a site template that's visible when you try to create a new site, but nothing that can be done about it!

So, first off, user solution WSPs still contain features. We *can* deactivate the feature that installs this site template (via PowerShell).

$site = Get-SPSite http://my.site.com
$site.Features | Where-Object {$_.FeatureDefinitionScope -eq "Site"}
Disable-SPFeature $feature -url http://my.site.com

But that's only hiding the problem, it's not actually fixing it. So, we should try to identify the user solution and remove it instead.

Uninstall-SPUserSolution -Identity "BasicProjectSite.wsp" -Site http://my.site.com

But this tells me that there is no solution with that name. But when I list $site.Solutions, it's there. Even running Get-SPUserSolution lists 0 solutions. What to do? Wait, does the object model still work?

Name                           SolutionId                           Status
----                           ----------                           ------
BasicProjectSite.wsp           af9fc924-26b1-4c4f-9250-57b26c81316f Activated

$solution = $site.Solutions[[guid]"af9fc924-26b1-4c4f-9250-57b26c81316f"]
$site.Solutions.Remove($solution)

And it's finally gone!

Odd issue with security trimming and global navigation

Just figured this one out.

So, right now, we use the SharePoint-managed nodemap in our layout, which is managed via /_layouts/AreaNavigationSettings.aspx and does automatic security trimming.

Recently, I'd noticed peculiar behavior in a couple of places. We would give a deep link to some resource to an external party, and when verifying the security on the new accounts we'd created for those individuals, most of the nodes in the tree would disappear...but not everything. The ones that remained were still not accessible to the user - clicking one would give Access Denied, but still, security trimming should remove these anyway, right?

The tricky bit is partially my fault.

Whenever I create a hyperlink target in SharePoint, I always strive to remove any unnecessary cruft from URLs. I strip off the hostname and the URI specifier, and if the target is a library or list, I remove the view name and just allow SharePoint to fall down to the default view, unless the target is supposed to be a particular list. So, when I have a link target of http://my.server.com/division/office/Lists/Contacts/AllContacts.aspx, I will usually replace that with /division/office/Lists/Contacts. Similarly, in a navigational node (where all the targets are generally subsites), I'd replace http://my.server.com/division/office with /division/office.

Usually.

Sometimes, I would inadvertently use /division/office/. That trailing slash would cause the security trimming to fail, but of course, the security is re-checked on pageload (thankfully!), so while I was violating security by accidentally exposing resources, I wasn't actually granting access to those resources. And in my current job, that's not a big big deal, but we're about to grant site access to a few companies who collaborate with us in some markets and compete in others...and one of the visible links was the name of a moderately high-profile project!

CredSSP across forests (except not)

Sorry, this post isn't going to give any insightful gleaning into how this works. I'm making a guess that it could work, but I would have to get a bunch of people to dedicate time and cycles to making it work, and only a small number of said individuals would likely think it important enough to actually do.

We're in the process of domain consolidation, and the domain my office is moving to is admin'ed by different folks. Our farm is not moving, we're opting instead to keep it in its current domain and simply stand up 2013 in the new domain and (to channel some Xzibit here) "migrate while we migrate". There's trust issues (from a certificate standpoint, not human trust) meaning that anytime I try to RDP into my own servers, Windows complains that they're not trusted anymore (don't recall getting those warnings before when I was in my old domain trying to remote into servers in the current domain, so I'm guessing their GPO doesn't have our certificate server as trusted, or some other AD nonsense). Accordingly, it's fouled up my credential delegation configuration (or perhaps this is also GPO at work).

At any rate, I have actual work to do and don't have time just yet to suss this out...so...saying goodbye to PowerShell remoting for now...

Testing owssvr.dll output

This is really a minor thing, but it annoyed me off and on over the span of a few months, and I just realized what the issue was and quickly fixed it.

I've touched on using owssvr.dll to create XMLified list dumps for consumption in InfoPath and other applications before. It was pretty straight-forward to do in SharePoint 2007, though once I migrated to SharePoint 2010, the lists weren't working exactly as I'd expect anymore. InfoPath seemed to be able to consume them fine (once I added noredirect=TRUE to the request).

If  I wanted to actually see the output I was getting from a request, however, normally I'd open up Internet Explorer, punch in the URL, and I'd see XML splashed all over my monitor. Until now. Now, IE wants to download the file instead. What a pain.

It turns out that this is related to a common problem folks have with new farms and viewing PDFs. The safe MIMEtype collection default in SharePoint doesn't contain a MIMEtype entry for application/PDF, nor does it contain one for application/xml!

Now, in Production, this is generally fine. My users won't be storing XML data in SharePoint, so I won't bother with updating the permitted MIMEtype list for the web application. In Development, however, I'm going to be using this a lot, and I'd rather not get bitten by the problem again, so...Central Administration -> Manage Web Applications...open General Settings for each of my web applications, and change Browser File Handling from Strict to Permissive. Voila, I no longer need to fuss with the extra steps of opening XML from my Downloads folder.

SharePoint DB GUID cleanup

With the help of well-publicized articles by Todd Klindt and his bro-crusher Shane Young, I had some admin time today to work on getting my environment set in order. Our Production SP2010 environment was put in place by consultants, which means, amongst other things, databases with long, frilly names.

For reference, see here. (Pictures hosted from the dead SharePoint911 server, go poke Shane about migrating his blog content to Rackspace!)

Shane's article on the Search Service Application DBs indicated that on a vanilla, non-used database, migrating your Crawl Store and Property Store DBs took a base of six minutes. When my databases were going upwards of 25 minutes, I decided to look into it more closely. As it turns out (this isn't entirely unexpected), when SharePoint creates its replacement databases, it ignores the Model DB settings and sets its growth to the agonizing standard of 1 MB increments...and then of course, it proceeds to jam all your old search terms/cache into that DB, one MB at a time.

Save yourself some agony - if you plan to do this yourself, once you apply your changes to the topology, open SQL Server Management Studio and fix the growth settings on both DBs to save yourself present and future headaches.

PowerShell 3 and SP2010, forever apart...?

It's well-documented that SP2010 is so hardcore about .Net 3.5 that it has a tattoo on its biceps of the build number in an XML element.

Of course, in my eagerness to adopt PowerShell and start to flex my muscles with it (which admittedly is rather tough to do when you don't have a sizeable farm to manage), I deployed PowerShell 3 on my development server, and now suddenly PowerShell doesn't like me anymore.

There's a small handful of posts out there (mostly related to PowerGUI) that essentially tell the story: PowerShell v3 doesn't play well with the SP2010 cmdlets (because they won't work with .Net 4.0). I tried to use a .config file for the PowerShell executeable to deny it access to .Net 4.0, but that simply didn't work as the PS engine requires 4.0. The common recommendation is to change the shortcuts for PowerShell on the upgraded servers to add "-version 2" to the launch command for PowerShell.

Well, that's great and all, but what about PS Remoting? Whenever I launch a local PS client and then remote into a server, the remote system is still running v3 (see $psversiontable), and I can't very well invoke a new instance of PowerShell from that remote session by trying the "powershell -version 2" trick from a remote session.

Luckily, PSv3 helps us out somewhat. First, we need to create a PSSessionConfigurationFile on the remote server:

New-PSSessionConfigurationFile -Path .\PowerShellv2.pssc

Now, open that file, and look for the line which contains the following:

# PowerShellVersion =

Uncomment this line and add in the target version like so:

PowerShellVersion = '2.0'

Save your changes, and now run:

Register-PSSessionConfiguration -Name Microsoft.PowerShellv2 -Path .\PowerShellv2.pssc

The configuration name is up to you, I used it simply because there's an existing configuration entry for PSv3 named "Microsoft.PowerShell". You'll get a couple of confirmation prompts and the WinRM service will be restarted (make sure you're not kicking anyone out of the system!). Once this is done, from your client:

Enter-PSSession -ComputerName MyServer -Authentication CredSSP -Credential $credentials -ConfigurationName Microsoft.PowerShellv2

Now just to check:

$psversiontable


Name                           Value
----                           -----
CLRVersion                     2.0.50727.5456
BuildVersion                   6.1.7601.17514
PSVersion                      2.0
WSManStackVersion              2.0
PSCompatibleVersions           {1.0, 2.0}
SerializationVersion           1.1.0.1
PSRemotingProtocolVersion      2.1


Success!

Missing Metadata Navigation panel

Had a bit of a brief stumper just now.

I've started using the Metadata Navigation panel more in some of my libraries. The tree view is somewhat useful for folder-level filtering, but the real juicy bits come in the Key Filters list.

Well, this is all well and good, but what if you configure Metadata Navigation, and then the panel doesn't even show up? A quick survey of the Interwebs reveals that everyone thinks that there are only two causes to this issue:

1. You turned off the feature
2. Your masterpage is borking it

Two very reasonable conclusions, but in my case, I had two libraries in the same site and one library had the navigation and the other didn't! Egad!

After doing some comparisons (and also noticing some odd webpart behavior from a week ago when I first set these libraries up), I realized that the Metadata Navigation panel appears to only show up for List webparts in something resembling their default configuration (more or less).

For the uninitiated, view pages are simply web part pages that have data view parts with the view information preconfigured based on the ViewFields collection. To a limited extent, you can modify these pages and add extra webparts and such. Apparently, the webpart in one library had its toolbar set to Summary, and another set to Full. I updated the Summary toolbar part to Full, and the nav panel reappeared. I turned it back off, and the panel stayed, so it's not a dependency on the toolbar setting at the very least, but there may have been another missing property in the web part (this library and its singular view had been untouched since our migration to 2010), and it appears that changing the toolbar mode reset that.

I hope this saves someone some agony, but then considering how narrow my particular issues tend to be, I have to laugh at myself about it all...

Moving list items with "Manage Content and Structure"

File this one under "lazy programming".

It's well-documented that you can move list items around using SharePoint's Content and Structure manager in SP2007 and higher, much in the same way that you can use Explorer View in document libraries to shuffle things around.

I was tasked with making portions of several lists visible to external entities. No problem I say, we simply break them out into folders, secure the folders, and modify the existing views to ignore folders. Like any good admin, of course, I created my folders, moved the files, and then turned off the Create Folders option, because I don't want people to start going nuts with the folders and expose things they don't intend to be exposed.

A bit of background information - One of these lists was already problematic due to some SPD editing in SP2007 that apparently did not translate well in our migration to SP2010, so I was eager to blow this list away and recreate it. One of the issues at hand was the fact that the selected view always said "Default", even though none were named Default and sometimes when selecting views, the view would change, and sometimes it wouldn't, and it almost consistently ignored whatever the settings in the Modify View page indicated. That there was a Modify View page at all should indicate that our XSLT data view part is not fully-converted. The view itself appeared to work to the end user, but it we were unable to maintain it since SPD didn't recognize its own handiwork any longer and SP2010 felt the same way. I expect that in the background, there was a separate list view web part that was being modified but it was not visible on the web part page for the view.

Anyway.

So, inevitably, after moving everything into the new list (and creating content types!), I find out that a bunch of the content is missing. Naturally, I should've checked that the list item count in View All Site Content matched, but I'm used to using List Totals to do that, and well...when you can't trust the views, that obviously doesn't work very well!

Again, anyway. (This does have a point.)

I restore the old list under a new name, and then I start to try to migrate in the content, but alas - I cannot move it into the new folder directly? When I use Content and Structure to select the subfolder to put the missing list items, the expansion doesn't occur and the + simply disappears. I had created two other similar lists, and those worked fine, so what gives? Is it an issue with the default view hiding folders? I tried to change default views in the new list to one that doesn't hide the folder content type, but that didn't help.

It turned out that some sloppiness on my part gave way to the ultimate solution here. Content and Structure will check if a list has Create Folders turned on before it tries to iterate and display the folder list. Since I failed to turn off Create Folders in the other lists, they would expand normally in the target selection dialog. Turning on Create Folders in the affected library restored my ability to select subfolders as a copy/move target in Content and Structure.

This one puzzled me for a few hours across a couple of days, and I'm not exactly sure how one would find the solution to such a problem, but if this post helps at least one person save a little time (and not just a spambot that says "your solution worked for me thanks"), then it will have been worthwhile.

Revisiting PowerShell Remoting

In a previous post, I mentioned configuring PowerShell Remoting so that I could start doing nifty PowerShell scripting stuffs without having to RDP into the server to do it. You know, The UNIX Way™. As I mentioned in a subsequent post, my remoting had suddenly and mysteriously broken such that while I could run around the filesystem all day long, executing a SharePoint cmdlet on the remote system would turn my session Broken.

Processing data for a remote command failed with the following error message: The WSMan provider host process did not return a proper response.  A provider in the host process may have behaved improperly. For more information, see the about_Remote_Troubleshooting Help topic.

Well, that's unhelpful, since I thought I had fixed this in the past. I spent an hour or so wrangling with it, disabling and re-enabling WSManCredSSP, but what finally tipped me off to the underlying cause of the problem was running winrm s winrm/config/client. I don't have the string output from the command handy since I have since closed that PS window, but in essence it told me that I was getting an underlying firewall exception because one of my NICs was set to Public Network (the VirtualBox Host-Only Network adapter). What? I thought we fixed that!

Opening up the Network and Sharing Center told me that, lo and behold, I was back on a Public network! It didn't take long for me to figure out what had happened - an upgraded version of VirtualBox likely removed and replaced the virtual NIC, and so it was set back to a Public Network. Bollocks!

Well, I fixed that problem, but I was still experiencing the same error trying to run SharePoint cmdlets (this was the point where I restarted the PowerShell session to clear my head), and then it occurred to me that I might need to restart WinRM in order to effect this change (other articles say to disable/enable the NIC, but I did not find this necessary). So, I restarted the WinRM service, and then I ran Get-SPServiceApplication about 10 times just to make sure I wasn't deceiving myself in that the commands were working...

Well, with that finally fixed, now I can set about configuring my remote profiles so that I will never, ever need to retype Add-PSSnapIn Microsoft.SharePoint.PowerShell...

IPFS permissions in SP2007 versus SP2010

The more things change, the more things stay the same. With regards to changing wantonly, I expect.

03/19/2012 09:16:04.25 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Foundation         Logging Correlation Data       xmnv Medium   Name=Request (POST:https://my.site.com:443/path/to/site/_layouts/Postback.FormServer.aspx) dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.29 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Foundation         General                       8kh7 High     <nativehr>0x80070005</nativehr><nativestack></nativestack>Access denied. dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.29 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Foundation         Logging Correlation Data       xmnv Medium   Site=/ dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.30 w3wp.exe (0x0DFC)                       0x1BF8 InfoPath Forms Services       Runtime                       f9n5 Medium   Thread was being aborted. dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.30 w3wp.exe (0x0DFC)                       0x1BF8 InfoPath Forms Services       Runtime - Business Logic       7tge Medium   Exception thrown from business logic event listener: System.Threading.ThreadAbortException: Thread was being aborted.     at NonConformanceReport.FormCode.buttonInitialSave_Clicked(Object sender, ClickedEventArgs e) in D:\2010 Projects\NonConformanceReport\NonConformanceReport\FormCode.cs:line 458     at Microsoft.Office.InfoPath.Server.SolutionLifetime.ButtonEventHost.<>c__DisplayClass6.<>c__DisplayClassa.<add_Clicked>b__3()     at Microsoft.Office.InfoPath.Server.Util.DocumentReliability.InvokeBusinessLogic(Thunk thunk)     at Microsoft.Office.InfoPath.Server.SolutionLifetime.ButtonEventHost.FireClickedEvent(Document document, ClickedEventArgs args)     at Microsoft.Office.InfoPath.Server.DocumentLifetime.OMExceptionManager.CallFormCodeWithEx... dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.30* w3wp.exe (0x0DFC)                       0x1BF8 InfoPath Forms Services       Runtime - Business Logic       7tge Medium   ...ceptionHandling(UserMessages userMessages, OMCall d) dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.30 w3wp.exe (0x0DFC)                       0x1BF8 InfoPath Forms Services       Runtime                       f9n5 Medium   Thread was being aborted. dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.30 w3wp.exe (0x0DFC)                       0x1BF8 InfoPath Forms Services       Runtime                       961x Medium   Not persisting state for request due to previous errors. Form Template: urn:schemas-microsoft-com:office:infopath:Non-Conformance-Report:-myXSD-2011-02-18T16-16-58 dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.33 w3wp.exe (0x0DFC)                       0x1BF8 InfoPath Forms Services       Runtime                       82fh Exception Unhandled exception processing request for PostbackPage Microsoft.Office.InfoPath.Server.Util.InfoPathFatalException: Exception of type 'Microsoft.Office.InfoPath.Server.Util.InfoPathFatalException' was thrown.     at Microsoft.Office.InfoPath.Server.Util.GlobalStorage.get_CurrentFormId()     at Microsoft.Office.InfoPath.Server.Util.GlobalStorage.get_CurrentContext()     at Microsoft.Office.InfoPath.Server.Util.GlobalStorage.IsDefined(GlobalItems key)     at Microsoft.Office.InfoPath.Server.Controls.DateFormattingInfo.TryGetCachedTimeZoneId(UInt16& timeZoneId)     at Microsoft.Office.InfoPath.Server.Controls.DateFormattingInfo.GetSPTimeZoneObject()     at Microsoft.Office.InfoPath.Server.Controls.DateFormattingInfo.CalculateSPLocalTimeOffset()     at Microsoft.Office.InfoPath.Server.Util.G... dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.33* w3wp.exe (0x0DFC)                       0x1BF8 InfoPath Forms Services       Runtime                       82fh Exception ...enericUtils.GetServerTimeZone()     at Microsoft.Office.InfoPath.Server.DocumentLifetime.ErrorPageRenderer.RenderResult(TextWriter writer, Document document, EventLogStart eventLogStart)     at Microsoft.Office.InfoPath.Server.DocumentLifetime.ErrorPageRenderer.RenderForException(HttpContext context, Exception exception, Document document, EventLogStart eventLogStart)     at Microsoft.Office.InfoPath.Server.Controls.PostbackPage.OnPreInit(EventArgs e)     at System.Web.UI.Page.PerformPreInit()     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.33 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Server             Unified Logging Service       c91s Monitorable Watson bucket parameters: SharePoint Server 2010, ULSException14, 41b7c39f "infopath forms services", 0e00178d "14.0.6029.0", fd34ba92 "microsoft.office.infopath.server", 0e0017dd "14.0.6109.0", 4e41890c "tue aug 09 15:22:52 2011", 00002563 "00002563", 00000014 "00000014", 33bcb602 "infopathfatalexception", 38326668 "82fh" dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.33 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Foundation         Performance                   nask High     An SPRequest object was not disposed before the end of this thread.  To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it.  This object will now be disposed.  Allocation Id: {C5772F5B-A52E-4B26-B3BE-1A21B271E91E}  To determine where this object was allocated, set Microsoft.SharePoint.Administration.SPWebService.ContentService.CollectSPRequestAllocationCallStacks = true. dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.33 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Foundation         Performance                   nask High     An SPRequest object was not disposed before the end of this thread.  To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it.  This object will now be disposed.  Allocation Id: {A0ED4F51-D0BF-44DF-A58F-C3CEC59FA780}  To determine where this object was allocated, set Microsoft.SharePoint.Administration.SPWebService.ContentService.CollectSPRequestAllocationCallStacks = true. dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.33 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Foundation         Performance                   nask High     An SPRequest object was not disposed before the end of this thread.  To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it.  This object will now be disposed.  Allocation Id: {EF332EFB-A193-4C02-BCD1-D7354B478B84}  To determine where this object was allocated, set Microsoft.SharePoint.Administration.SPWebService.ContentService.CollectSPRequestAllocationCallStacks = true. dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

03/19/2012 09:16:04.33 w3wp.exe (0x0DFC)                       0x1BF8 SharePoint Foundation         Monitoring                     b4ly Medium   Leaving Monitored Scope (Request (POST:https://my.site.com:443/path/to/site/_layouts/Postback.FormServer.aspx)). Execution Time=84.2051154546178 dd4f6a8d-fcc4-463b-b42c-f3c150b42ca6

In SharePoint 2007, using an IPFS form meant that the user account needed whatever privileges were required to:

1. Open the form template
2. Access any data sources referenced in the form template
3. Save the form back (if necessary or not handled in code, the form might not save to SP at all!)

Reasonable enough. Since the form templates are stored in the site collection root, if you had forms deployed in a subsite, you would just create a SP/AD group that holds the form users and gives them the specific access they require without compromising your model.

This translates to:

1. Read access to the site collection root (from which FormServerTemplates should be inheriting)
2. Read access to any data sources referenced in the form template
3. Add or Modify access depending on if/how the form saves data back to SharePoint

So, obviously with SharePoint 2010, things get more complicated. Some of the users of two particular applications no longer have the ability to save a specific form. Of course, this didn't come up in our migration testing. Now I find that the following is required:

1. Read access to the site collection root (from which FormServerTemplates should be inheriting)
2. Contribute access to the web root
3. Read access to any data sources referenced in the form template
4. Add or Modify access depending on if/how the form saves data back to SharePoint

I cannot fathom why this changed, but now it's forcing me to either make broad, sweeping security changes to accomodate the forms, or place them in a subsite and keep security as-is, but fix any broken path dependencies (such as Excel sheets reporting on form submissions).


Edit: I haven't had time to do isolation testing to narrow down the above, but I had another form that works without setting Contribute on the site root, the key difference being that this form uses the built-in SharePoint submit rule rather than saving via code. I'll have to take a closer look at it.

Setting up PS Remoting for all that PS/SP fun on your local box

With the boss out today and no major tasks beating down my door, I decided to take another swing at getting a sane and working PowerShell environment set up on my work PC. I was tired of having to remote into a server to do simple CLI stuff like getting internal field names and web GUIDs and the like.

The first time I was doing this, I was messing around with PowerGUI and trying to see about simple list iterations on the server, but I rapidly ran out of remote sessions and didn't know offhand how to easily reconnect to them. So, after five times, I had to start figuring out how to kill the remaining sessions before I could continue trying to figure out how to do actual things. Wasn't getting to be very fun.

Now for Take Two.

This time, I learned right away that I could easily manage the sessions by storing them in a variable and using Invoke-Command to do single-line command entry, like so:

PS > $session = New-PSSession -ComputerName TargetHost

PS > Invoke-Command -Session $session -ScriptBlock { Get-ChildItem C:\ } 

Simple enough so far. But obviously cumbersome. I'd already played around with Enter-PSSession before though, so was it easy enough to simply write Enter-PSSession $session?

PS > Enter-PSSession $session

[TargetHost]: PS > Get-ChildItem C:\

Apparently so! Exit-PSSession will return you to your local host, with $session intact, and you can run Remove-PSSession $session once you're finished to clean up after yourself.


Now then, I can at last experiment without worrying about locking myself out.


So, for the new user, it doesn't take long to figure out that while you're on a SharePoint server, the regular PowerShell just won't do. You must either launch the SP-provided PowerShell launcher, or run the basic PowerShell client and then run Add-PSSnapIn Microsoft.SharePoint.PowerShell to get access to the SharePoint-specific cmdlets. So, what happens when I run this in my session?

Cannot access the local farm. Verify that the local farm is properly configured, currently available, and that you have the appropriate permissions to access the database before trying again.

Now, I have this permssion on my own account when I run the PowerShell client via RDP on the server, so what gives? Enter this excellent article from JoshGav which explains that the culprit is our old nemesis, the double-hop authentication problem! Curses!

Following the instructions within mostly worked, but running Enable-WSManCredSSP -Role Client -DelegateComputer * -Force would always fail with the following:

Enable-WSManCredSSP : The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: "winrm quickconfig".

At line:1 char:20

+ Enable-WSManCredSSP <<<<  -Role Client -DelegateComputer CSLISP2010WFE -Force

    + CategoryInfo          : InvalidOperation: (System.String[]:String[]) [Enable-WSManCredSSP], InvalidOperationException

    + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.EnableWSManCredSSPCommand

A couple of MSDN forum postings later, and I got to the point where I confirmed that the problem was that I had enabled remoting on the server...but silly me, I still needed to enable it on the client!

PS C:\Users\Me> Enable-PSRemoting

WinRM Quick Configuration

Running command "Set-WSManQuickConfig" to enable this machine for remote management through WinRM service.

 This includes:

    1. Starting or restarting (if already started) the WinRM service

    2. Setting the WinRM service type to auto start

    3. Creating a listener to accept requests on any IP address

    4. Enabling firewall exception for WS-Management traffic (for http only).

Do you want to continue?

[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"): y

WinRM has been updated to receive requests.

WinRM service type changed successfully.

WinRM service started.

Set-WSManQuickConfig : WinRM firewall exception will not work since one of the network connection types on this machine is set to Public. Change the network connection type to either Domain or Private and try again.

At line:50 char:33

+             Set-WSManQuickConfig <<<<  -force

    + CategoryInfo          : InvalidOperation: (:) [Set-WSManQuickConfig], InvalidOperationException

    + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.SetWSManQuickConfigCommand

Aha. I had read on this earlier in my research, and I already knew the culprit - VirtualBox. Luckily, I located this helpful article by Thijs Kroesbergen which details how to exclude VirtualBox from network management so that I can enable CredSSP.

So, now we're golden right? Wrong. Now, when I try to run Enable-PSRemoting, I get sent back a simple "Access is denied." error. Well, that's just peachy. One last bit of research tells me that I must be *explicitly* placed in my machine's local Administrators group.

Windows PowerShell

Copyright (C) 2009 Microsoft Corporation. All rights reserved.

PS C:\Users\Me> Enable-PSRemoting

WinRM Quick Configuration

Running command "Set-WSManQuickConfig" to enable this machine for remote management through WinRM service.

 This includes:

    1. Starting or restarting (if already started) the WinRM service

    2. Setting the WinRM service type to auto start

    3. Creating a listener to accept requests on any IP address

    4. Enabling firewall exception for WS-Management traffic (for http only).

Do you want to continue?

[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"): a

WinRM already is set up to receive requests on this machine.

WinRM already is set up for remote management on this machine.

PS C:\Users\Me> Enable-WSManCredSSP -Role Client -DelegateComputer * -Force

cfg         : http://schemas.microsoft.com/wbem/wsman/1/config/client/auth

lang        : en-US

Basic       : true

Digest      : true

Kerberos    : true

Negotiate   : true

Certificate : true

CredSSP     : true

PS C:\Users\Me> $session = New-PSSession SPSERVER -Authentication CredSSP -Credential DOMAIN\Me

PS C:\Users\Me> Enter-PSSession $session

[spserver]: PS C:\Users\Me\Documents> Add-PSSnapIn Microsoft.SharePoint.PowerShell

[spserver]: PS C:\Users\Me\Documents> $site = Get-SPSite("https://my.site.com/")

[spserver]: PS C:\Users\Me\Documents> $site

Url

---

https://my.site.com

Success!

The other migration: STSADM to PowerShell

So, as everyone and their brother knows, while STSADM is still technically available in SP2010, it's deprecated and the general advice is to pretend it doesn't exist anymore. Fair enough, PowerShell purports to give us plenty of power to make our lives better and easier. I haven't done any serious learning with PS yet, but I've had to use it in a couple different ways during the course of this SP2010 migration.

So, that said, looks like it still needs a bit of tweaking:

This is an error message I received because I was re-adding a solution too quickly. I'm uploading the solution on my WFE, but doing deployment/retraction in Central Admin on the application server, and I'll receive the above unfriendly red text whenever I try to upload a solution that's in the middle of removal from the farm. 

Add-SPSolution : An object in the SharePoint administrative framework, "SPSolutionLanguagePack Name=0", depends on other objects which do not exist.  Ensure that all of the objects dependencies are created and retry this operation.

At line:1 char:15

+ Add-SPSolution <<<<  D:\My.New.Column.wsp

    + CategoryInfo          : InvalidData: (Microsoft.Share...dletAddSolution:SPCmdletAddSolution) [Add-SPSolution], A

   rgumentException

    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletAddSolution

Awfully cryptic, but at least it didn't take long to figure out.

The infamous SP2010 User Profile Service

One of our contractor consultants is responsible with part-time administration tasks, and he assisted to a degree with the setup of our SP2010 environment. He struggled for some time with the configuration of the import of user profiles (we have five domains to pull profiles from currently), but last week he successfully got it configured and got profiles imported.

Thing is though, I was trying to look at some information about the profile import and anytime I'd open the Connections page, I would get the well-traveled "The query returns nothing" error.

After poring over his configuration for some time, I finally got tired of fussing with it and just nuked everything he did. Using the de facto guide published by Spencer Harbar (and some incremental fixing of Event Viewer log errors in trying to fix the original morass), I was able to successfully configure the User Profile Service Application in under 3 hours (had to add a NIC to the server in order to set up an SSL instance of MySites to save for later).

If you want something done right...

IPFS in SharePoint 2010

So, in my old work environment, we had a number of rather elaborate code-behind forms hosted in IPFS. My boss there made a wise decision to keep those applications running in MOSS while gradually porting those solutions into SP2010. I was dimly aware of the fact that porting IPFS 2007 forms into SP2010 was effectively a game of roulette, so I was pleasantly surprised when the three forms we currently use at my new company ported with some degree of ease.

Ironically, the form with the most code-behind (for item uploads, sending mail notifications, rolling comments in a repeating section, and a host of other tidbits) had no problems whatsoever. The form that I experienced the biggest number of issues with had almost no code at all, and the code it did have was simply an autonumber generation system.

This form uses a drop-down category selector that, when changed, would show/hide a number of other drop-downs based on the category selected. In the past, each of these categories was located in its own section with duplicate bindings to the same parent group (I'm not a huge XML guy, so I have little doubt that this is highly offensive to someone). So, when you change the category, one section unhides, and the currently visible one disappears. The sub-selections each use a OWSSVR.dll query to filter on a specific list using the category selection as a parameter, so we have a different data source for each category. Simple enough (but not really). I could have managed this in code-behind to a degree, but I'm not a fan of using OnFormXmlChange events in IPFS, and I didn't want the user to have to click a button after selecting a category to re-run the lookups.

Anyway...

Further down this form lie a number of XHTML fields.

For some inexplicable reason, whenever the category selection changes (either in a new form, or changing the selected value in an existing form), one or more of these XHTML fields would suddenly become read-only (and any existing text would be concealed).

Flailing away at the form template like a coffee-dependent monkey eventually gave rise to the realization that in SP2010, upon form redraw (as might happen if a control is being shown or hidden), the fields are actually being reloaded and their contents re-rendered, and it seems as though having to redo this redraw effort with the extra sections caused the redraw to fail prematurely, and so some controls would not completely load.

At first I experimented with cloning the category drop-down into a series of radio selectors. Using these made the problem less prominent, but it would still occur. Finally, I removed the sections and placed each of the six sub-selects into a single section and set the conditional formatting on each control itself, rather than a governing section, and in general this made the behavior go away (though it still manifests when the Category selection changes, the redraw seems to finish when the sub-selection is changed).

I met with the owner of the system employing this form today to show him my work in redesigning the original work to conform to my new IPFS form layout (since I had to touch the form anyway to fix the 2010 issues, I modified its layout to conform to the styling I plan to employ for all my forms here). During our demo, we came across another issue.

Now, cascading drop-downs are nothing new in InfoPath, but the behavior of these in IPFS 2010 is a little different and might cause me some grief down the road. Normally, any data source queries in InfoPath lookups will always have a blank entry at the top of the query. I have a rule set to reinitialize the value in this sub-select if the category changes (since obviously that value is no longer valid). When the category changes, the drop-down does indeed clear, however the previous selection is still shown in the drop-down's values! What exactly is going on here?

Clearly there's a new renderer at work to go along with the SP2010 engine, but I'm puzzled by some of these not-particularly-complex bugs popping up (and unfixed in Service Pack 1). We're currently planning to fast-track our migration into SP15 (possibly even skipping the SP2010 visual upgrade!), so maybe I should just hold my breath for a year.

From the other side of the fence

Well, should've had our SP2010 upgrade done by now, but some delays with our setup of the User Profile Service caused us to miss our deadline for determining go-live, so we pushed it back by a couple of weeks. In the mean time, it turns out that this decision was for the best, because it turns out that our branding effort being made by the consultant team didn't cover all the SharePoint pages we'd be using (such as the special layout used for the Thumbnail view in an Asset Library).

Regarding the User Profile Service, we were having one of the consultants work on that, but he left a huge mess of extra application pools in his wake, and now the service isn't even running because of a permissions problem, so I've got to sort that out.

In the mean time, I've learned a ton about setup recommendations and application pool sharing from Todd Klindt's blogposts here and here. My setup isn't completely out of whack, but it's nice to know the reasoning behind these decisions that I'm reading other people recommend. I had to consolidate a couple of service pools on our app server, but I won't be consolidating the web apps since they're all SSL-only and thus answering on separate IPs.

After that, I just need to work around an Edit Control Block (ECB) issue in this one document library (which will be moved to an Asset Library after our migration), and then I think we've pretty much got this one in the bag. I've spent a good amount of time in the last week doing performance tuning and fixing issues on the servers, so I'm looking forward to abnormally good performance in our new environment.

Now if only IPFS would be cooperative...(but that is for a future post, guess I'll start that one right now!)

Oh hey, this thing's still on

We're almost to the end of our upgrade from MOSS to SP2010. One of the minor gotchas I've had to correct recently involves the docicon registry on the server. For whatever reason, even though Microsoft negotiated a deal to incorporate Adobe branding/trademarks for Office clients (to save to PDF), this evidently didn't extend to SharePoint, and so SharePoint 2010 is, once again, without an official PDF icon.

Accordingly, I have to complete the traditional administrator journey of looking up information on something I did years ago once and never had to worry about again:

http://www.sharepointedutech.com/2010/02/17/changing-file-associations-icons-in-sharepoint-2010/


Of course, the instructions are pretty straight-forward, but the icon insertion didn't work, and suddenly I found all the icons broken. The answer is pretty obvious once you know about it, but it's so simple to forget about. Consider the string below:

<Mapping Key="pdf" Value="icpdf.gif" OpenControl=""/>

When copied off some websites using fancy new-fangled fonts and the like, instead of the regular ASCII quotes, you'll get elegant opened/closed quotes, which will still paste as extended ASCII in Notepad, but they'll break the XML syntax and upon an IISreset, you will no longer be able to load the icon file at all. Brilliant, eh?

Webparts and STSDEV

I'm pretty used to doing 2007 development with STSDev and I've gotten really comfortable with it. However, one of the things I haven't built yet, amazingly enough, is a webpart.

So, last week, I was trying to find a world clock webpart that would properly observe DST where applicable. Amazingly, only one I was able to find does, but it sources external Flash content as part of its rendering. Instead, I decided to get the CKS release of the Bamboo World Clock and Weather webpart and put proper DST support within.

The code itself is...amusing to say the least. If I didn't know better, I'd say it was intentional to have the main namespace "CKS.Bamboo.WorldCockAndWeather".

At any rate, the CKS project makes a solution, but it's not feature-activated. STSDev has support for webpart-based projects, but crashes when parsing ClassResources. So, in my infinite wisdom OCD, I'm moving the Bamboo code into a STSDev project, and now I find myself modifying the source of STSDev to incorporate support for copying ClassResources into the Manifest, and proper storage of the data within the DDF.

2010 can't come soon enough.