Thursday, 22 July 2010

Upload multiple files into a document library using PowerShell and set metadata

This article has now been updated to include a script download with many more features, including support for almost all column types, copying files from subfolders in the source location and retaining the structure in the destination library, an overwrite option, ability to merge subfolders into a single location, and much more. Visit this page for details - http://get-spscripts.com/2010/10/bulk-upload-files-with-metadata-into.html.

A scenario I come across quite regularly is migrating data into SharePoint from a file system and tagging it with column values on its way in. The PowerShell script below provides a framework for doing exactly that. It doesn’t cover all the options that you might need from a fully featured migration platform but is a good starting point from which you can extend further.

The general order of the script is as follows:

  • Specify site, document library and local folder path variables (haven’t tested with network paths yet but will update blog when I have).
  • Enumerate through all files in the folder (not subfolders at the moment) and upload them into the document library in SharePoint. I have set it to upload the files to the root of the library, but you could change the script to copy into sub-folders or migrate an entire folder structure from the file system to SharePoint.
  • Connect to a manifest XML file and configure the column values - The columns themselves must already be present in the document library prior to running the script. The file specifies values for “Single line of text” and “Managed Metadata” (single value) column types – the only two columns types supported in the script at the moment. With Managed Metadata columns, the script will look for the value in the term set first (it will choose the first term it picks up if there is more than one in the term set with the same name) and create it if it isn’t there. An example of a manifest file is shown below:

<?xml version="1.0" encoding="utf-8"?>
<Columns>
  <Column name="Technology Area" type="TaxonomyFieldType">
    <Values>
      <Value>SharePoint</Value>
    </Values>
  </Column>
  <Column name="Subject" type="Text">
    <Values>
      <Value>Document Management</Value>
    </Values>
  </Column>
</Columns>

  • Check the files into the document library (optional depending on the document library configuration)
  • Approve the files (optional depending on the document library configuration)

Here is the script:

#Setup default variables
$metadataManifest = [xml] (Get-Content ("C:\Install\DocManifest.xml"))
$webUrl = "http://portal"
$docLibraryName = "Documents"
$localFolderPath = "c:\Install\Docs"

#Get web and document library objects
$web = Get-SPWeb $webUrl
$docLibrary = $web.Lists[$docLibraryName]
#Attach to local folder and enumerate through all files
$files = ([System.IO.DirectoryInfo] (Get-Item $localFolderPath)).GetFiles() | ForEach-Object {

    #Create file stream object from file
    $fileStream = ([System.IO.FileInfo] (Get-Item $_.FullName)).OpenRead()
    $contents = new-object byte[] $fileStream.Length
    $fileStream.Read($contents, 0, [int]$fileStream.Length);
    $fileStream.Close();

    write-host "Copying" $_.Name "to" $docLibrary.Title "in" $web.Title "..."

    #Add file
    $folder = $docLibrary.RootFolder
    $spFile = $folder.Files.Add($folder.Url + "/" + $_.Name, $contents, $true)
    $spItem = $spFile.Item

    #Walk through manifest XML file and configure column values on the file
    $metadataManifest.Columns.Column | ForEach-Object {
        #Single line of text column
        if ($_.Type -eq "Text")
        {
            $columnName = $_.Name
            write-host "Setting value on column"$columnName
            $_.Values.Value | ForEach-Object {
                $spItem[$columnName] = $_
                $spItem.Update()
                write-host "Value set to"$_
            }
        }
        #Single value Managed Metadata column
        if ($_.Type -eq "TaxonomyFieldType")
        {
            $columnName = $_.Name
            $taxonomySession = Get-SPTaxonomySession -Site $web.Site
            $termStore = $taxonomySession.DefaultSiteCollectionTermStore
            $taxonomyField = $docLibrary.Fields[$columnName]
            $termSet = $termStore.GetTermSet($taxonomyField.TermSetId)
            write-host "Setting value on column"$columnName
            $_.Values.Value | ForEach-Object {
                $termCollection = $termSet.GetTerms($_, $true)
                if ($termCollection.Count -eq 0)
                {
                    $term = $termSet.CreateTerm($_, 1033)
                    $termStore.CommitAll()
                }
                else
                {
                    $term = $termCollection[0]
                }
                $taxonomyField.SetFieldValue($spItem, $term)
                write-host "Value set to"$_
            }
        }
    }

    #Check in file to document library
    #MinorCheckIn=0, MajorCheckIn=1, OverwriteCheckIn=2
    $spFile.CheckIn("File copied from " + $filePath, 1)
    if ($spFile.CheckOutStatus -eq "None") { write-host $spfile.Name"checked in" }

    #Approve file
    $spFile.Approve("File automatically approved after copying from " + $filePath)
    if ($spItem["Approval Status"] -eq 0) { write-host $spfile.Name"approved" }
}
#Dispose of Web object
$web.Dispose()

For brevity purposes, I haven’t added any exception handling in the script posted here, so I am assuming that the site, document library, local folder, permissions, columns, etc. have all been created and assigned correctly. PowerShell is fairly good at reporting issues, and if you use something like Windows PowerShell ISE included with Windows Server 2008 R2 and Windows 7, it will report the location of the issue in the script and even supports breakpoints so that you can debug.

Once you have this basic script working, you could introduce a whole host of other features – e.g., create an XML or CSV file plan containing different folder structures on the file system and related document manifests so that you can copy larger numbers of documents up in one hit, configure specific security permissions, attach to different content types, etc. I will be adding extra functionality myself and documenting in future posts on this blog…

Wednesday, 21 July 2010

Tuesday, 20 July 2010

Modify multi value user profile properties in SharePoint with PowerShell

Following on from my previous article on how to modify single value user profile properties in SharePoint 2010 using PowerShell, this script shows how you can modify multi value properties, which are stored in the Keywords term set in the Managed Metadata service application. There is actually not a lot of difference between the two scripts – the only change being the part of the script that modifies the property. The key difference is that this script adds a term to the list of terms already present in the property rather than replacing the value as before.

The example I have used here is to add the term “SharePoint” to the list of skills for the user DOMAIN\phil.childs. If the term is already present in the Keywords term set, then it will retrieve it and add it to the user profile property. If it is not currently in the term set, then it will add it for you before populating the user profile property:

#Set up default variables
$mySiteUrl = "http://mysite"
$adAccount = "DOMAIN\phil.childs"
$upAttribute = "SPS-Skills"
$upAttributeValue = "SharePoint"

#Get site objects and connect to User Profile Manager service
$site = Get-SPSite $mySiteUrl
$context = Get-SPServiceContext $site
$profileManager = New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager($context)

#Check to see if user profile exists
if ($profileManager.UserExists($adAccount))
{
    #Get user profile
    $up = $profileManager.GetUserProfile($adAccount)
    $up[$upAttribute].Add($upAttributeValue)
    $up.Commit()
}
else
{
    write-host "Profile for user"$adAccount "cannot be found"
}

#Dispose of site object
$site.Dispose()

A few things to note with this script:

  • Make sure you have the correct permissions configured on the User Profile Service Application (see previous article for details)
  • As before, you will need to specify the My Site host site collection URL for the $mySiteUrl variable
  • You have to use the internal name of the user profile property when assigning the $upAttribute variable – e.g., “SPS-Skills” for the Skills property – this is also the same for single value properties, too (forgot to mention this in my previous article). You can find the internal name of a property by going to the Manage User Properties page of the User Profile Service Application and editing a property

Manage User Properties 

Along with adding terms to a multi value user profile property, you may also want to remove them. You can do this by using the same script above and replacing the $up[$upAttribute].Add($upAttributeValue) line with the following, which cycles through all the terms assigned to the user profile property and removes the term you have specified once found:

for ( [int]$i = 0; $i -lt $up[$upAttribute].Count; $i++ )
{
    if ($up[$upAttribute][$i] -eq $upAttributeValue) { $up[$upAttribute].RemoveAt($i) }
}

Finally, if you want to remove ALL terms from the user profile property (e.g., delete all terms from the Skills property for that user), then replace the $up[$upAttribute].Add($upAttributeValue) line with the following:

$up[$upAttribute].Clear()

Friday, 16 July 2010

Modify single value user profile properties in SharePoint with PowerShell

The two most widely used user profile property types in SharePoint 2010 are “string (Single Value)” and “string (Multi Value)”. Single value property types work in a similar way to “Single line of text” columns in lists and libraries – examples being Name, Department, Job Type, etc. Multi value property types use the Managed Metadata service to allow the multiple input of new or previously used values from the Keywords term set – examples being Skills, Schools, Responsibilities, etc.

The PowerShell script below provides an example of how to add or modify a single value user profile property for an individual user. Admittedly, this can be done quite easily by using the Central Administration UI (although it does take a few clicks), but it’s real power is when you combine the script with a CSV or XML file to bulk change hundreds of user profile properties in a matter of seconds – you can find examples of using CSV and XML files for bulk operations here and here, although I plan to publish more articles on these techniques in future posts.

Note: The user running the script (which I assume is some sort of SharePoint Administrator) must have been granted the Manage Profiles right in the Administrators section and Full Control rights in the Permissions section of the User Profile Service Application (pictured below).

UserProfileAdmin

The script below changes the value of the Web Site attribute for a user with the account name DOMAIN\Phil.Childs. You will need to specify the My Site host site collection URL for the $mySiteUrl variable:

#Set up default variables
$mySiteUrl = "http://mysite"
$adAccount = "DOMAIN\phil.childs"
$upAttribute = "WebSite"
$upAttributeValue = “http://get-spscripts.com”

#Get site objects and connect to User Profile Manager service
$site = Get-SPSite $mySiteUrl
$context = Get-SPServiceContext $site
$profileManager = New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager($context)

#Check to see if user profile exists
if ($profileManager.UserExists($adAccount))
{
    #Get user profile and change the value
    $up = $profileManager.GetUserProfile($adAccount)
    $up[$upAttribute].Value = $upAttributeValue
    $up.Commit()
}
else
{
    write-host "Profile for user"$adAccount "cannot be found"
}

#Dispose of site object
$site.Dispose()

In a future post I will run through how to use a similar approach to change a multi value user profile property…

Thursday, 15 July 2010

Replacing the Page Contact on all pages for all sites in a site collection using PowerShell

Here is the scenario: A page author at your company has decided to leave and there are a large number of publishing pages in SharePoint associated with them as a Page Contact. Your options are to a) manually replace the Page Contact on each page in each site with the person now responsible for managing publishing pages in the site collection, b) pay the old page author more money and persuade them to stay to avoid the extra management costs, or c) use this PowerShell script below to walk through each page in each site of the site collection and automatically change the Page Contact from the old page author to the new one:

#Set up default variables
$webSiteUrl = "http://portal"

#Add SharePoint assemblies
Add-Type -AssemblyName "Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
Add-Type -AssemblyName "Microsoft.SharePoint.Publishing, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"

#Get site and web objects
$site = Get-SPSite $webSiteUrl

#Set original and new Page Contact AD accounts
$originalContact = "DOMAIN\oldpageauthor"
$newContact = "DOMAIN\newpageauthor"

#Walk through each Publishing Web and change the page contact on every page
#where the original contact equals the originalContact variable set above
$site | Get-SPWeb -limit all | ForEach-Object {
    #Check to see if site is a publishing site
    if ([Microsoft.SharePoint.Publishing.PublishingWeb]::IsPublishingWeb($_)) {
        write-host "Reviewing pages in"$_.Title"site...."
        #Get the Publishing Web and pages within it
        $publishingWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($_)
        $publishingPages = $publishingWeb.GetPublishingPages()
        foreach ($publishingPage in $publishingPages)
        {
            $originalUser = $_.AllUsers[$originalContact]
            #Check if the new user exists in the web - if not, create them
            try
            {
                $newUser = $_.AllUsers[$newContact]
            }
            catch
            {
                $_.AllUsers.Add($newContact, "", "", "")
                $_.Update()
                $newUser = $_.AllUsers[$newContact]
            }
            #Check to ensure the page isn't checked out
            if ($publishingPage.ListItem.File.CheckOutStatus -eq "None")
            {
                #Check to ensure a Page Contact has been set on the page
                if ($publishingPage.Contact -ne $null)
                {
                    #If the Page Contact is set to the original user,
                    #change to the new user and check in the page
                    if ($publishingPage.Contact.ToString() -eq $originalUser.ToString())
                    {
                        $publishingPage.ListItem.File.CheckOut()
                        $publishingPage.Contact = $newUser
                        $publishingPage.Update()
                        $publishingPage.ListItem.File.CheckIn("Contact name changed", [Microsoft.SharePoint.SPCheckinType]::MajorCheckIn)
                        write-host "Page contact for"$publishingPage.Title"changed to"$publishingPage.Contact.LoginName
                    }
                    else
                    {
                        #Notify administrator that the Page Contact is someone other than the originalContact specified in this script
                        write-host "Page contact for"$publishingPage.Title"is"$publishingPage.Contact.LoginName"and shall not be modified"
                    }
                }
                else
                {
                    #Notify administrator that the Page Contact has not been set on this page
                    write-host "Page contact for"$publishingPage.Title"has not been set and shall not be modified"
                }
            }
            else
            {
                #Notify administrator that the page is checked out and cannot be modified
                write-host "Page"$publishingPage.Title"is currently checked out to"$publishingPage.ListItem.File.CheckedOutBy"and cannot be modified"
            }
        }
    }
    else
    {
        #Notify administrator that the site is not a publishing site
        write-host $_.Title"is not a publishing site"
    }
}

#Dispose of Site object
$site.Dispose()

In summary, this script does the following:

- Connects to a site collection and walks through each site
- Checks to see if the site is a publishing site before making any changes (if not it moves on to the next site)
- Checks each publishing page in the site
- If a page is already checked out then it doesn't make any changes to the page
- If a Page Contact is set to someone else other than the old author you specified then it doesn't make any changes
- If the Page Contact is set to the old author then it will check out the page, change the contact to the new author specified and check the page back in again
- Reports all activities to the console whilst reviewing the sites - see below for an example log:

Reviewing pages in PAC Portal site....
Page contact for Home has not been set and shall not be modified
Page Query Web Parts test is currently checked out to DOMAIN\spadmin and cannot be modified
Reviewing pages in Publishing Site site....
Page contact for Home changed to DOMAIN\newpageauthor
Page contact for Test is DOMAIN\phil.childs and shall not be modifed
Reviewing pages in Search site....
Page contact for Search Center has not been set and shall not be modified
Page contact for Advanced search has not been set and shall not be modified
Page contact for People Search has not been set and shall not be modified
Page contact for Search Results has not been set and shall not be modified
Page contact for People Results has not been set and shall not be modified
Team is not a publishing site

Note: If approval is required on pages in your publishing sites, then you will need to go through manually and approve any pages changed by this script to publish them (of course, this can be done in PowerShell, but I think I’ll leave that to another time!)

Tuesday, 13 July 2010

How to build a SharePoint 2010 PowerShell Cmdlet

There is a great article by Corey Roth here on what you need to start building a custom SharePoint 2010 PowerShell cmdlet in Visual Studio 2010. As he says in the article, there are a lot of resources out there on how to build general PowerShell cmdlets, but not many specifically focused on SharePoint. Another great resource to look at is Gary Lapointe’s SharePoint Automation blog. Gary has created a whole bunch of custom PowerShell cmdlets for SharePoint and released the source code for download so that you can see how it’s done…

Monday, 12 July 2010

Adding SharePoint groups with permission levels to sites using PowerShell

For this PowerShell script I have created a function called AddGroupToSite, allowing you to assign a SharePoint group (must already be created in the site collection) to a site along with a permission level by specifying one line of script. This first section sets up the SPWeb object and the function:

$web = Get-SPWeb "http://portal"

function AddGroupToSite ($web, $groupName, $permLevel)
{
    $account = $web.SiteGroups[$groupName]
    $assignment = New-Object Microsoft.SharePoint.SPRoleAssignment($account)
    $role = $web.RoleDefinitions[$permLevel]
    $assignment.RoleDefinitionBindings.Add($role);
    $web.RoleAssignments.Add($assignment)
}

Functions are very useful in PowerShell because they allow you to reduce the number of lines in your script by calling the same routine multiple times, passing in various parameters of your choosing to vary the properties of the function – in the example above, I am passing in the SPWeb object, group name and permission level. You can find more information on using functions by typing get-help about_Functions from PowerShell itself or there are plenty of tutorials on the subject around the Web or in books. Once we have our function set up, we can call it and pass the relevant parameters as follows:

AddGroupToSite -web $web -groupName "Site Admins" -permLevel "Full Control"
AddGroupToSite -web $web -groupName "Site Readers" -permLevel "Read"

These lines add two SharePoint groups – Site Admins and Site Readers – to the site http://portal and assign them Full Control and Read permissions respectively. You could also feed values from a CSV or XML file into your function to automate this for a number of sites as a bulk operation. If you need to break permission inheritance on the site before adding the groups, add one of the following lines just after the $web = Get-SPWeb http://portal line at the top of the script:

#Break permissions inheritance and copy the groups from parent site into this site
$web.BreakRoleInheritance($true)

#Break permissions inheritance and assign the current user as the only member of this site
$web.BreakRoleInheritance($false)

UPDATE - 17th February 2011: I have written an article expanding on the subject of this post describing how to use PowerShell to assign SharePoint and AD group/user permissions for all sites in a site collection. Please click here for details.

Friday, 2 July 2010

Add pre-query search suggestions for SharePoint 2010 with a CSV file and PowerShell

The pre-query search suggestions feature (“find-as-you-type” functionality in the Search Center) automatically populates a list of keywords using previously searched terms by users. Obviously, this may take a while to compile, so it is a good idea to pre-populate a list of keywords from a CSV file (providing you know what the keywords are of course). This PowerShell script does exactly that. The format of the CSV file must have the word “Suggestion” on the top line and then your keyword list – for example:

Suggestion
SharePoint
SharePoint Server 2010
Sharing Files
SharePoint Designer
SharePoint Foundation

The script itself looks like this:

#Set up default variables
$csvfile="C:\Install\SearchSuggestions.csv"
$ssa = Get-SPEnterpriseSearchServiceapplication -Identity "Search Service Application"

#Create Lists from each item in CSV file
$csvData = Import-Csv $csvfile
foreach ($line in $csvData)
{
    New-SPEnterpriseSearchLanguageResourcePhrase -SearchApplication $ssa -Language en-US -Type QuerySuggestionAlwaysSuggest -Name $line.Suggestion
}

$timerJob = Get-SPTimerJob "Prepare query suggestions"
$timerJob.RunNow()

Note that the we also run the “Prepare query suggestions” Timer job – which normally runs daily – to prepare these suggestions for use in search.

Thursday, 1 July 2010

Creating multiple SharePoint groups and adding users with PowerShell

One of the great features in PowerShell is the ability to read an XML file and traverse the DOM nodes using a very simple dot notation. An example is the script below, which uses an XML file to modify and create SharePoint groups, adding their corresponding users. First the XML file:

<?xml version="1.0"?>
<Groups>
    <Group name="Approvers" description="">
        <Users>
            <User>DOMAIN\pchilds</User>
            <User>DOMAIN\tsmith</User>
        </Users>
    </Group>
    <Group name="HR" description="Team responsible for company Human Resources">
        <Users>
            <User>DOMAIN\jdavies</User>
            <User>DOMAIN\pjackson</User>
        </Users>
    </Group>
</Groups>

In this scenario, we are not just creating new groups, but also adding users to existing groups (“Approvers” in the example above). If the users already exists in a group, the script will just continue and add the next one:

#Get Site and Web objects
$site = Get-SPSite “http://portal”
$web = $site.RootWeb

#Get XML file containing groups and associated users
$groupsXML = [xml] (Get-Content ("C:\Path\Groups.XML"))

#Walk through each group node defined in the XML file
$groupsXML.Groups.Group | ForEach-Object {
    #Check to see if SharePoint group already exists in the site collection
    if ($web.SiteGroups[$_.name] -eq $null)
    {
        #If the SharePoint group doesn't exist already - create it from the name and description values at the node
        $newGroup = $web.SiteGroups.Add($_.name, $web.CurrentUser, $null, $_.description)
    }

    #Get SharePoint group from the site collection
    $group = $web.SiteGroups[$_.name]

    #Add the users defined in the XML to the SharePoint group
    $_.Users.User | ForEach-Object {
        $group.Users.Add($_, "", "", "")
    }
}

#Dispose of Web and Site objects
$web.Dispose()
$site.Dispose()

The script connects to the <Group /> node defined in the XML file with the line $groupsXML.Groups.Group and then uses the ForEach-Object statement to walk through each one. It detects if the group already exists – if not it creates a new one using the name and description defined at the node – and then walks through each <User /> node to add the users. Note the way the dot notation is used to access various parts of the XML node tree and how the “$_” syntax is used to call the node in context and its associated values.