Archiving documents timer job

February 7, 2012

Whilst working a recent project I had to create a timer job which moved some InfoPath forms from an archive document library to a location which was dynamically generated depending on the forms modified date. I am not going to cover the basics on how to create a timer job as Andrew Connell already has a very good article on how to create a custom timer job.

In this article I will focus on how to move a InfoPath form but it could equally to word documents, PDF, etc. The same could apply to list data but it will require some code changes.

As with all my projects I first mapped out what steps, see below.

  1. Get configuration data
  2. Open the site in which my forms are stored
  3. Get the document library
  4. Get the items which meet a certain criteria
  5. Iterate through the items and for each item perform the following
    1. Get the form modified date
    2. Check and see if a document library already exists for the current year and if not create one
    3. In the document library check if a folder for the month exists and if not create it
    4. Copy the form to the new location
    5. Set the metadata to be copied across. One consideration that may apply to others is version, however this was not relevant for me.
    6. Delete the original

I will assume you have read the post above by Andrew Connell so I will jump in once we have already got our site collection.

Point 1

As with all projects you want to minimise the amount of data which is hard coded as this reduces the need for additional deployments when certain environment specific variables change. There are a few options on where to store these but I decided to store mine in the web.config of the web application. This requires a few steps to read the data from the web.config so I created a helper method which returned a custom class which is used to store the variables. 

  1. ConfigurationData config = GetConfigurationData(webApplication.Name);


  1. /// <summary>
  2.         /// Method to get the configuration data from the web.config
  3.         /// </summary>
  4.         /// <param name="webAppName">A string of the web application name</param>
  5.         /// <returns>An ConfigurationData object with the configuration details</returns>
  6.         private ConfigurationData GetConfigurationData(string webAppName)
  7.         {
  8.             ConfigurationData configData = new ConfigurationData();
  9.             Configuration config = WebConfigurationManager.OpenWebConfiguration("/", webAppName);
  10.             if (config != null)
  11.             {
  12.                 if (config.AppSettings.Settings["Property1"] != null)
  13.                 {
  14.                     configData.Property1 = config.AppSettings.Settings["Property1"].Value;
  15.                 }
  17.                 if (config.AppSettings.Settings["Property2"] != null)
  18.                 {
  19.                     configData.Property2 = config.AppSettings.Settings["Property2"].Value;
  20.                 }
  22.                 if (config.AppSettings.Settings["Property3"] != null)
  23.                 {
  24.                     configData.Property3 = Int32.Parse(config.AppSettings.Settings["Property3"].Value);
  25.                 }
  26.             }
  28.             return configData;
  29.         }

Points 2, 3 and 4

With the configuration data retrieved from the web.config the next few steps are very straightforward to anyone who has done any SharePoint development.

  1. //open site
  2.             using (var site = new SPSite(siteCollection.Url))
  3.             using (var web = site.OpenWeb(config.Property1))
  4.             {                
  5.                 web.AllowUnsafeUpdates = true;
  6.                 try
  7.                 {
  8.                     SPList list = null;
  9.                     try
  10.                     {
  11.                         //get library
  12.                         list = web.Lists[config.Property2];
  13.                     }
  14.                     catch (ArgumentException ae)
  15.                     {
  16.                         LogDetails(String.Format("There was a problem gettting the archive list, the error was {0}", ae.ToString()));
  17.                         return;
  18.                     }
  20.                     if (list == null)
  21.                     {
  22.                         LogDetails("Unable to get the archive list");
  23.                         return;
  24.                     }                    
  26.                     var query = new SPQuery();                    
  27.                     query.Query = "YOUR CAML QUERY";
  29.                     LogDetails(String.Concat("CAML Query is ", query.Query));
  31.                     //get items
  32.                     var listItems = list.GetItems(query);
  33.                     var listItemCount = listItems.Count;
  34.                     if (listItems == null || listItemCount == 0)
  35.                     {
  36.                         LogDetails(String.Concat("There were no items returned by the query {0}", query.Query));
  37.                         return;
  38.                     }
  40.                     LogDetails(String.Concat("Got items ", listItemCount.ToString()));

Point 5

At this stage we now have a list of all items items which meet the relevant archive criteria but for obvious reason the actual logic has been removed from this blog. My next step was to iterate through all items and move them to the appropriate location. At first I used a foreach loop but this didn’t work as I was adjusting the item collection and this caused a runtime error. Next I tried a for loop using the count of the number of items. While this seemed to work I found it was only iterating through half of the list and after the half way point I was getting an error saying “Specified argument was out of the range of valid values.”. For example if the count of items was 10 it could loop through items 1-5 but as soon as it reached 6 it give the error above. To get around this I changed the code to start at the last item in the count and work backwards i.e. 10, 9, 8. To keep the code contained I separated the main functionality out into a few different methods.

Loop through items
  1. //loop through items moving them
  2.                     for (int itemNumber = listItemCount; itemNumber > 0; itemNumber–)
  3.                     {
  4.                         try
  5.                         {
  6.                             LogDetails(String.Concat("Start item ", itemNumber.ToString()));
  7.                             SPListItem listItem = listItems[itemNumber -1];
  8.                             if (listItem == null)
  9.                             {
  10.                                 LogDetails(String.Format("There was a problem getting the list item at index {0} returned by the query {1}",
  11.                                     itemNumber.ToString(), query.Query));
  12.                             }
  13.                             else
  14.                             {
  15.                                 MoveItem(listItem, web);
  16.                             }
  17.                         }
  18.                         catch (Exception ex)
  19.                         {
  20.                             LogDetails(ex.ToString());                            
  21.                         }                                                                        
  22.                     }


This function takes an item, gets the modified date and passes this to another helper function which gets the folder which the item has to be moved to. Next it builds up the URL to where the item has to be copied to. The item is then moved but because this method doesn’t return the SPFile object I had to then get it from the destination folder. Once I had the new item I then set some properties to ensure metadata is retained as otherwise the created and modified details would be incorrect. Finally the original item is deleted.

Function for each item
  1. /// <summary>
  2.         /// Method to perform the move of an individual item
  3.         /// </summary>
  4.         /// <param name="listItem">An SPListItem of the item to be moved</param>
  5.         /// <param name="web">An SpWeb object which represents the web in which the item is located</param>
  6.         private void MoveItem(SPListItem listItem, SPWeb web)
  7.         {           
  8.             DateTime modifiedDate = DateTime.Parse(listItem[SPBuiltInFieldId.Modified].ToString());            
  9.             SPFolder destinationLocation = GetDestinationLibrary(modifiedDate, web);            
  10.             string destinationLocationURL = String.Format("{0}/{1}/{2}", web.Url, destinationLocation.Url, listItem.File.Name);           
  12.             try
  13.             {
  14.                 listItem.File.CopyTo(destinationLocationURL);
  15.             }
  16.             catch (Exception ex)
  17.             {
  18.                 LogDetails(String.Concat("There was a problem copying the file to the new URL, the error was {0} ",
  19.                     ex.ToString()));
  20.                 throw;
  21.             }
  23.             SPFile file = null;
  24.             try
  25.             {
  26.                 file = destinationLocation.Files[listItem.File.Name];
  27.             }
  28.             catch (Exception ex)
  29.             {
  30.                 LogDetails(String.Format("There was a problem getting the new file {0} in location {1}, the error was {2}",
  31.                     listItem.File.Name, destinationLocation.Url, ex.ToString()));
  32.                 throw;
  33.             }
  35.             if (file == null)
  36.             {
  37.                 LogDetails(String.Format("Unable to find file {0} in new location.",
  38.                     listItem.File.Name));
  39.                 throw new NullReferenceException("Unable to find new file in destination location");
  40.             }
  42.             var fileItem = file.Item;
  44.             try
  45.             {
  46.                 /*since we are running with elevated permissions we need to set the
  47.                  * author and editor back to the details from the original item
  48.                  * otherwise it will appear as system account                                         *
  49.                  * */
  50.                 fileItem[SPBuiltInFieldId.Author] = listItem[SPBuiltInFieldId.Author];
  51.                 fileItem[SPBuiltInFieldId.Created] = listItem[SPBuiltInFieldId.Created];
  53.                 fileItem[SPBuiltInFieldId.Editor] = listItem[SPBuiltInFieldId.Editor];
  54.                 fileItem[SPBuiltInFieldId.Modified] = listItem[SPBuiltInFieldId.Modified];
  55.                 fileItem.UpdateOverwriteVersion();
  57.                 listItem.Delete();
  58.             }
  59.             catch (Exception ex)
  60.             {
  61.                 //log error and move on to next record
  62.                 LogDetails(String.Format("There was a setting the file properties for item {0} the problem was {1}",
  63.                     listItem.Title, ex.ToString()));
  64.                 throw;
  65.             }
  66.         }

As mentioned above the previous snippet calls this GetDestinationLibrary method which uses the modified date to see if a document library exists for the year in the same web and if not it creates one. Next it calls a separate function which performs the same idea but this one uses the month and creates a folder inside the document library.

Get document library
  1. /// <summary>
  2.         /// Method to get the URL of document library which the list item should be added to. If the library is not found one is created.
  3.         /// </summary>
  4.         /// <param name="modifiedDate">A DateTime of the last modified date of the item</param>
  5.         /// <param name="web">An SPWeb object representing the web which contains the items</param>
  6.         /// <returns>An SPFolder of the location the item should be copied to</returns>
  7.         private SPFolder GetDestinationLibrary(DateTime modifiedDate, SPWeb web)
  8.         {
  9.             string destinationLibraryURL = String.Empty;
  10.             string year = modifiedDate.Year.ToString();
  12.             SPList desintationDocumentLibrary = null;
  14.             try
  15.             {
  16.                 //get library
  17.                 desintationDocumentLibrary = web.Lists[year];
  18.             }
  19.             catch (ArgumentException)
  20.             {
  21.                 desintationDocumentLibrary = CreateDocumentLibrary(year, web);
  22.             }
  24.             SPFolder monthlyFolder = GetMonthlyFolder(CultureInfo.CurrentCulture.DateTimeFormat.GetMonthName(modifiedDate.Month), desintationDocumentLibrary);
  26.             return monthlyFolder;
  27.         }


Create monthly folder
  1. /// <summary>
  2.         /// Method to try and find the folder representing the month given a month name. If a month is not found one is created.
  3.         /// </summary>
  4.         /// <param name="monthlyName">A string of the month name</param>
  5.         /// <param name="list">An SPList which is the list the folder will be created in</param>
  6.         /// <returns>An SPFolder representing the folder to which we should move the item</returns>
  7.         private SPFolder GetMonthlyFolder(string monthName, SPList list)
  8.         {
  9.             SPFolder monthlyFolder = null;
  10.             //loop through to see if the folder exists
  11.             foreach (SPListItem folder in list.Folders)
  12.             {
  13.                 if (folder.Name.Equals(monthName, StringComparison.CurrentCultureIgnoreCase))
  14.                 {
  15.                     monthlyFolder = folder.Folder;
  16.                     break;
  17.                 }
  18.             }
  20.             //if we don't have it create it
  21.             if (monthlyFolder == null)
  22.             {
  23.                 monthlyFolder = list.RootFolder.SubFolders.Add(monthName);
  24.                 monthlyFolder.Update();
  25.             }
  27.             return monthlyFolder;
  28.         }



Putting all this together hopefully some people will find this an interesting article with some useful ideas. As always if anyone who reads this can think of any improvements I am always happy to discuss them.

Lastly if you do use this code as a base for your own project please ensure this is tailored to meet our own solution and has been properly tested on a development environment before being deployed to a live farm.

Add Icon to notification area in Windows tray

September 30, 2011

During a recent conversation with a client they mentioned they wanted a Windows Forms Application to pull information from their SharePoint site. We discussed the requirements and I said yes that fine we can do that. They then said they would like the application to run in the background and to have an Icon in the Notification area, see screenshot below, which pops up telling them some new content has been added to SharePoint.


My immediate reaction was it must be relatively easy but it was not something I had tried before. When I got back to the office I created a test application and I was surprised by how easy this was. I have included below some same code which hides the application when it is started, adds an item to the notification area and displays a popup every 10 seconds. If this popup is click it will open the application or if the icon in the notification area is click it will also open the application.

Hopefully you find this useful and can use this as a starting point for further developments. As always this code should always be tested before being deployed to any live environments and is used at your own risk.

Code Snippet
  1. public partial class Form1 : Form
  2.     {
  3.         NotifyIcon notify;
  4.         Timer timer;
  5.         public Form1()
  6.         {
  7.             InitializeComponent();
  8.             timer = new Timer();
  9.             timer.Interval = 10000;
  10.                 timer.Start();
  11.             timer.Tick +=new EventHandler(timer_Tick);
  13.             notify = new NotifyIcon();
  14.             notify.Icon =
  15.    new System.Drawing.Icon(System.Environment.GetFolderPath
  16.    (System.Environment.SpecialFolder.Personal)
  17.    + @"\logo.ico");
  18.             notify.Visible = true;
  19.             notify.Text = "Test notification area icon";
  21.             this.WindowState = System.Windows.Forms.FormWindowState.Minimized;
  22.             this.Visible = false;
  23.             this.ShowInTaskbar = false;
  24.         }
  26.         public void  timer_Tick(object sender, EventArgs e)
  27.         {
  28.             notify.Click += new EventHandler(notify_Click);
  29.             notify.ShowBalloonTip(3000, "Test notification area header", "Test notification area test", ToolTipIcon.Info);
  30.             notify.BalloonTipClicked += new EventHandler(notify_BalloonTipClicked);
  31.         }
  33.         void notify_Click(object sender, EventArgs e)
  34.         {
  35.             this.WindowState = System.Windows.Forms.FormWindowState.Normal;
  36.             this.Visible = true;
  37.         }
  39.         void notify_BalloonTipClicked(object sender, EventArgs e)
  40.         {
  41.             this.WindowState = System.Windows.Forms.FormWindowState.Normal;
  42.             this.Visible = true;
  43.         }
  44.     }

Access web.config from SharePoint Timer Job

June 8, 2011

While working on a public facing website using MOSS 2007 I needed to created an XML sitemap to be used by Google and other search engines. I looked around at various different options but eventually found a Blog article by Tim Dobrinski. This covered most of my requirements, however there were a few things I wanted to parameterise such as a list of sites to be excluded from the XML and also the document library the XML file was to be uploaded to.

Talking to a colleague, Gordon Duthie, he mentioned he had done something similar and blogged about it. After reading his blog, Reading from web.config’s AppSettings in a SharePoint timer job, I thought this would be nice and easy. The complication arose from the fact the client had a web application they were using to edit the content for the site and then this was extended and given the public facing site URL.

When I started to use the code it seemed to be working correctly but then I added a setting in the web application web.config and the extended web.config with different values. I found it was only ever hitting the original web.config. Thinking about it this made perfect sense as while there are two sites in IIS there is only one web application. Even though I am accessing the site via the extended application URL the web application name, which is what my colleague was using to get the path to the web.config, was always going to be the actual web application name and not the name of the extended web application.

There may be other workarounds but I decided I would use a custom property to pass information to the timer job, adding properties to a custom SharePoint Timer Job. The only down side of this is if I need to change the details being passed to the timer job I have to deactivate and then reactivate the feature.

Empty ULS Log Files

April 13, 2011

When working on a recent project I was getting the classic ‘Unexpected error’ SharePoint message, my first step was to check the ULS logs but when I went to the directory I noticed that while log files were getting created all the files were empty.

To correct this I opened the services window and re-started the ‘Windows SharePoint Services Tracing’ service and like magic its started logging.

Page layout not showing in create page list

March 22, 2011

One of my most recent projects was finishing off development of a public facing website built in MOSS 2007. The project had already been started, however the person involved had since left the company. Regardless of this I thought it would be a simple enough task as all I had to do was create some additional site columns, content types and page layouts.

The project was set up with the page layouts and master pages deployed via a feature and the page layouts were automatically associated with a content type. I created the additional site columns, content types and page layouts and deployed this to the development environment where everything looked ok. The site columns were in the site column list, the content types appeared in the content type list with all the correct columns and the page layouts and masterpages were in the page layouts and masterpage gallery and were associated with the correct content types. Next I added my new content types to a pages library but when i went to create a new page the only options were the page layouts that had been created previously, not even any of the out of the box options were showing. My first impressions were there was an issue with the feature so I redeployed this but this but it made no difference so next I checked the page layouts were all published and they were. At this point I was stumped as all the previous projects I had worked on I had followed the same steps but the pages had appeared in the list as soon as they were in the gallery.

I spent some time looking around on the web but couldn’t see anything until I was reading an article by Chris O’Brien Deploying master pages and page layouts as a feature and noticed someone else had added a comment saying they were having the same issue. Reading down I noticed Chris had suggested the page layouts might have been restricted via the site settings. When I checked the site sure enough under the ‘Look and Feel’ section in site settings there was an option for ‘Page Layouts and Site Templates’ and when I opened this I immediately noticed what the issue was. I would like to thank Chris for his very useful article but also for him to take the time to respond to all the comments and thus helping me out :).

I am sure to some people this will be obvious but this was the first time I had encountered restricting the page layouts available to end users so I wanted to document this in case others encounter the same issue.

Formula refers to a column that does not exist

February 25, 2011

While working on a recent project I was creating/updating some site columns on an intranet which was based on Windows SharePoint Services v3.0. The changes were mainly creating new site columns to capture additional data and adjusting some existing columns, including a few calculated columns. I have used calculated columns in other projects so was expect the changes to be very straight forward.

When looking at the formula site columns I copied the existing formula into notepad twice using one to change and one as a backup. I made my changes to the formula and then pasted the code back into SharePoint but when I updated the site column I got the following error:

“The formula refers to a column that does not exist.  Check the formula for spelling mistakes or change the non-existing column to an existing column.”

While I was positive none of my changes could have resulted in this error I went back and compared my new version to the original just in case I had accidentally changed one of the column names but they were all correct. Next I checked that all the columns used in my formula were in the ‘Insert Column’ list to the right hand side of the formula box in case any of the columns had been deleted since the work was originally done but again everything was as it was supposed to be. When I went back over my new formula I noticed that while one of the fields was wrapped in square brackets, [column], another wasn’t. My initial reaction was that shouldn’t make a difference as my understanding was only fields that contained spaces in the name required square brackets but I though I would add them anyway. The result was the update to the formula worked.

I have since done some additional testing and it is true that when you add a new columnn from the ‘Insert Column’ list it automatically adds square brackets around it but if you then click ok and go back into the formula it seems to remove those square brackets from any columns that don’t have spaces in the name. I have spent some time testing on a development environment but can’t seem to establish exactly what it was about this particular formula that required the square brackets around the site column name.

At the end of the day I managed to get the functionality to work but I would still be interested in others experience of anything similar.

Master page and page layout reporting

February 22, 2011

A few weeks ago while working on a publically facing website built in MOSS 2007 I had to make some changes that affected all page layouts and master pages. The problem was overtime lots of test and unused page layouts and master pages had been left in the master page gallery. Rather than add the changes to all the page layouts and master pages I decided to write a tool that iterated through all webs and all pages in a site collection outputting the master page and page layout being used.

Since I am used to writing them I created a windows forms application which asks the user to supply the URL of the site, the only downside to this is the tool must be run on one of the web front end (WFE) servers.For my purposes I decided to take a snapshot of the site at a specific point and essentially export the data to a SQL DB. I create a variety of tables to store the information but I won’t cover this here as what I needed may not be required by most people. The reason I took this approach is I wanted to do some reporting on the data and didn’t want to be constantly programmatically accessing the site in case it had an effect on the performance.

Since I am not covering the DB side of things i have adjusted the code below to instead output all the details to a text file, this should give you an idea of what the process would be and you can alter it as you see fit.

Please note that while I have tested and ran this several times all use is at your own risk and strongly recommend this is tested in a development environment prior to being used on a production site.

private void button1_Click(object sender, EventArgs e)


    if (String.IsNullOrEmpty(TbxSiteURL.Text))


        MessageBox.Show("Please enter a URL");




    //open site and get web

    using (SPSite site = new SPSite(TbxSiteURL.Text))

    using (SPWeb web = site.OpenWeb())


        //call function to loop through all sites and publishing pages





/// <summary>

/// An iterative function that takes a site and log details to a text file such as masterpage and 

/// publishing pages page layouts. It then loops through all subsites of the site

/// passed in and calls the method again.

/// </summary>

/// <param name="web">A SPWeb object of the site to log details and check subsites</param>

private void IterateWebs(SPWeb web)


    string message = String.Format("Web: {0}{1}Location: {2}{1}Masterpage URL: {3}{1}",

        web.Title, Environment.NewLine, web.Url, web.MasterUrl);




    if (PublishingWeb.IsPublishingWeb(web))


        PublishingWeb pubWeb = PublishingWeb.GetPublishingWeb(web);


        foreach (PublishingPage page in pubWeb.GetPublishingPages())


            string pageLayout = page.ListItem.Properties["PublishingPageLayout"] as string;

            message = String.Format("Page name: {0}{1}Page URL: {2}{1}Page layout: {3}{1}",

                        page.Title, Environment.NewLine, page.Url, pageLayout);






        LogWebDetails("Web is not a publishing web" + Environment.NewLine);



    foreach (SPWeb childWeb in web.Webs)








            if (childWeb != null)








/// <summary>

/// Log details to a text file

/// </summary>

/// <param name="logMessage">A string of the message to be written to the file</param>

private void LogWebDetails(string logMessage)


    File.AppendAllText("Log.txt", logMessage);




SPContext.Web access denied error on SharePoint 2007 publically available sites

February 8, 2011

Towards the end of last week and the start of this week at work I was putting live some changes to a publically available site built in SharePoint 2007 when I started to get access denied errors on some pages which had been created using a new page layout. Initially I assumed that one of the components on the page were not checked in and published so I checked all the usual items such as:

  • The content pages stored in the site Pages Library
  • Master pages and page layouts in the site collection gallery
  • Shared content used on the pages
  • Images and JavaScript used on the page

After reviewing all the items mentioned above I was no further forward so I started to remove sections from the master page and page layout until I was able to pinpoint it was a particular web part, which was used to generate the sub navigation. Using a process of elimination and colleague and I were able to identify the accessed denied error was being triggered by a call to the SPContext.Current.Web SPWeb object. Initially I was confused as the code was wrapped in an SPSecurity.RunWithElevatedPrivileges block thus I assumed all code executed was getting run under the context of the application pool and should not be getting an access denied error. After some future research I found an msdn article which detailed the fact that using the SPContext.Current.Web will not be affected by running code in the Elevated Privileges block as this is created with the context of the currently running user.

To correct this issue I created new SPSite and SPWeb objects using the URL of the page being hit and the site worked as expected. I am aware lots of people may be aware of this caveat but thought it would be useful to document, partly for myself and partly for any others that may encounter a similar issue.

%d bloggers like this: