Archiving documents timer job

February 7, 2012

Whilst working a recent project I had to create a timer job which moved some InfoPath forms from an archive document library to a location which was dynamically generated depending on the forms modified date. I am not going to cover the basics on how to create a timer job as Andrew Connell already has a very good article on how to create a custom timer job.

In this article I will focus on how to move a InfoPath form but it could equally to word documents, PDF, etc. The same could apply to list data but it will require some code changes.

As with all my projects I first mapped out what steps, see below.

  1. Get configuration data
  2. Open the site in which my forms are stored
  3. Get the document library
  4. Get the items which meet a certain criteria
  5. Iterate through the items and for each item perform the following
    1. Get the form modified date
    2. Check and see if a document library already exists for the current year and if not create one
    3. In the document library check if a folder for the month exists and if not create it
    4. Copy the form to the new location
    5. Set the metadata to be copied across. One consideration that may apply to others is version, however this was not relevant for me.
    6. Delete the original

I will assume you have read the post above by Andrew Connell so I will jump in once we have already got our site collection.

Point 1

As with all projects you want to minimise the amount of data which is hard coded as this reduces the need for additional deployments when certain environment specific variables change. There are a few options on where to store these but I decided to store mine in the web.config of the web application. This requires a few steps to read the data from the web.config so I created a helper method which returned a custom class which is used to store the variables. 

  1. ConfigurationData config = GetConfigurationData(webApplication.Name);

 

  1. /// <summary>
  2.         /// Method to get the configuration data from the web.config
  3.         /// </summary>
  4.         /// <param name="webAppName">A string of the web application name</param>
  5.         /// <returns>An ConfigurationData object with the configuration details</returns>
  6.         private ConfigurationData GetConfigurationData(string webAppName)
  7.         {
  8.             ConfigurationData configData = new ConfigurationData();
  9.             Configuration config = WebConfigurationManager.OpenWebConfiguration("/", webAppName);
  10.             if (config != null)
  11.             {
  12.                 if (config.AppSettings.Settings["Property1"] != null)
  13.                 {
  14.                     configData.Property1 = config.AppSettings.Settings["Property1"].Value;
  15.                 }
  16.  
  17.                 if (config.AppSettings.Settings["Property2"] != null)
  18.                 {
  19.                     configData.Property2 = config.AppSettings.Settings["Property2"].Value;
  20.                 }
  21.  
  22.                 if (config.AppSettings.Settings["Property3"] != null)
  23.                 {
  24.                     configData.Property3 = Int32.Parse(config.AppSettings.Settings["Property3"].Value);
  25.                 }
  26.             }
  27.  
  28.             return configData;
  29.         }

Points 2, 3 and 4

With the configuration data retrieved from the web.config the next few steps are very straightforward to anyone who has done any SharePoint development.

  1. //open site
  2.             using (var site = new SPSite(siteCollection.Url))
  3.             using (var web = site.OpenWeb(config.Property1))
  4.             {                
  5.                 web.AllowUnsafeUpdates = true;
  6.                 try
  7.                 {
  8.                     SPList list = null;
  9.                     try
  10.                     {
  11.                         //get library
  12.                         list = web.Lists[config.Property2];
  13.                     }
  14.                     catch (ArgumentException ae)
  15.                     {
  16.                         LogDetails(String.Format("There was a problem gettting the archive list, the error was {0}", ae.ToString()));
  17.                         return;
  18.                     }
  19.  
  20.                     if (list == null)
  21.                     {
  22.                         LogDetails("Unable to get the archive list");
  23.                         return;
  24.                     }                    
  25.  
  26.                     var query = new SPQuery();                    
  27.                     query.Query = "YOUR CAML QUERY";
  28.  
  29.                     LogDetails(String.Concat("CAML Query is ", query.Query));
  30.  
  31.                     //get items
  32.                     var listItems = list.GetItems(query);
  33.                     var listItemCount = listItems.Count;
  34.                     if (listItems == null || listItemCount == 0)
  35.                     {
  36.                         LogDetails(String.Concat("There were no items returned by the query {0}", query.Query));
  37.                         return;
  38.                     }
  39.  
  40.                     LogDetails(String.Concat("Got items ", listItemCount.ToString()));

Point 5

At this stage we now have a list of all items items which meet the relevant archive criteria but for obvious reason the actual logic has been removed from this blog. My next step was to iterate through all items and move them to the appropriate location. At first I used a foreach loop but this didn’t work as I was adjusting the item collection and this caused a runtime error. Next I tried a for loop using the count of the number of items. While this seemed to work I found it was only iterating through half of the list and after the half way point I was getting an error saying “Specified argument was out of the range of valid values.”. For example if the count of items was 10 it could loop through items 1-5 but as soon as it reached 6 it give the error above. To get around this I changed the code to start at the last item in the count and work backwards i.e. 10, 9, 8. To keep the code contained I separated the main functionality out into a few different methods.

Loop through items
  1. //loop through items moving them
  2.                     for (int itemNumber = listItemCount; itemNumber > 0; itemNumber–)
  3.                     {
  4.                         try
  5.                         {
  6.                             LogDetails(String.Concat("Start item ", itemNumber.ToString()));
  7.                             SPListItem listItem = listItems[itemNumber -1];
  8.                             if (listItem == null)
  9.                             {
  10.                                 LogDetails(String.Format("There was a problem getting the list item at index {0} returned by the query {1}",
  11.                                     itemNumber.ToString(), query.Query));
  12.                             }
  13.                             else
  14.                             {
  15.                                 MoveItem(listItem, web);
  16.                             }
  17.                         }
  18.                         catch (Exception ex)
  19.                         {
  20.                             LogDetails(ex.ToString());                            
  21.                         }                                                                        
  22.                     }

 

This function takes an item, gets the modified date and passes this to another helper function which gets the folder which the item has to be moved to. Next it builds up the URL to where the item has to be copied to. The item is then moved but because this method doesn’t return the SPFile object I had to then get it from the destination folder. Once I had the new item I then set some properties to ensure metadata is retained as otherwise the created and modified details would be incorrect. Finally the original item is deleted.

Function for each item
  1. /// <summary>
  2.         /// Method to perform the move of an individual item
  3.         /// </summary>
  4.         /// <param name="listItem">An SPListItem of the item to be moved</param>
  5.         /// <param name="web">An SpWeb object which represents the web in which the item is located</param>
  6.         private void MoveItem(SPListItem listItem, SPWeb web)
  7.         {           
  8.             DateTime modifiedDate = DateTime.Parse(listItem[SPBuiltInFieldId.Modified].ToString());            
  9.             SPFolder destinationLocation = GetDestinationLibrary(modifiedDate, web);            
  10.             string destinationLocationURL = String.Format("{0}/{1}/{2}", web.Url, destinationLocation.Url, listItem.File.Name);           
  11.  
  12.             try
  13.             {
  14.                 listItem.File.CopyTo(destinationLocationURL);
  15.             }
  16.             catch (Exception ex)
  17.             {
  18.                 LogDetails(String.Concat("There was a problem copying the file to the new URL, the error was {0} ",
  19.                     ex.ToString()));
  20.                 throw;
  21.             }
  22.  
  23.             SPFile file = null;
  24.             try
  25.             {
  26.                 file = destinationLocation.Files[listItem.File.Name];
  27.             }
  28.             catch (Exception ex)
  29.             {
  30.                 LogDetails(String.Format("There was a problem getting the new file {0} in location {1}, the error was {2}",
  31.                     listItem.File.Name, destinationLocation.Url, ex.ToString()));
  32.                 throw;
  33.             }
  34.  
  35.             if (file == null)
  36.             {
  37.                 LogDetails(String.Format("Unable to find file {0} in new location.",
  38.                     listItem.File.Name));
  39.                 throw new NullReferenceException("Unable to find new file in destination location");
  40.             }
  41.  
  42.             var fileItem = file.Item;
  43.  
  44.             try
  45.             {
  46.                 /*since we are running with elevated permissions we need to set the
  47.                  * author and editor back to the details from the original item
  48.                  * otherwise it will appear as system account                                         *
  49.                  * */
  50.                 fileItem[SPBuiltInFieldId.Author] = listItem[SPBuiltInFieldId.Author];
  51.                 fileItem[SPBuiltInFieldId.Created] = listItem[SPBuiltInFieldId.Created];
  52.  
  53.                 fileItem[SPBuiltInFieldId.Editor] = listItem[SPBuiltInFieldId.Editor];
  54.                 fileItem[SPBuiltInFieldId.Modified] = listItem[SPBuiltInFieldId.Modified];
  55.                 fileItem.UpdateOverwriteVersion();
  56.  
  57.                 listItem.Delete();
  58.             }
  59.             catch (Exception ex)
  60.             {
  61.                 //log error and move on to next record
  62.                 LogDetails(String.Format("There was a setting the file properties for item {0} the problem was {1}",
  63.                     listItem.Title, ex.ToString()));
  64.                 throw;
  65.             }
  66.         }

As mentioned above the previous snippet calls this GetDestinationLibrary method which uses the modified date to see if a document library exists for the year in the same web and if not it creates one. Next it calls a separate function which performs the same idea but this one uses the month and creates a folder inside the document library.

Get document library
  1. /// <summary>
  2.         /// Method to get the URL of document library which the list item should be added to. If the library is not found one is created.
  3.         /// </summary>
  4.         /// <param name="modifiedDate">A DateTime of the last modified date of the item</param>
  5.         /// <param name="web">An SPWeb object representing the web which contains the items</param>
  6.         /// <returns>An SPFolder of the location the item should be copied to</returns>
  7.         private SPFolder GetDestinationLibrary(DateTime modifiedDate, SPWeb web)
  8.         {
  9.             string destinationLibraryURL = String.Empty;
  10.             string year = modifiedDate.Year.ToString();
  11.  
  12.             SPList desintationDocumentLibrary = null;
  13.  
  14.             try
  15.             {
  16.                 //get library
  17.                 desintationDocumentLibrary = web.Lists[year];
  18.             }
  19.             catch (ArgumentException)
  20.             {
  21.                 desintationDocumentLibrary = CreateDocumentLibrary(year, web);
  22.             }
  23.  
  24.             SPFolder monthlyFolder = GetMonthlyFolder(CultureInfo.CurrentCulture.DateTimeFormat.GetMonthName(modifiedDate.Month), desintationDocumentLibrary);
  25.  
  26.             return monthlyFolder;
  27.         }

 

Create monthly folder
  1. /// <summary>
  2.         /// Method to try and find the folder representing the month given a month name. If a month is not found one is created.
  3.         /// </summary>
  4.         /// <param name="monthlyName">A string of the month name</param>
  5.         /// <param name="list">An SPList which is the list the folder will be created in</param>
  6.         /// <returns>An SPFolder representing the folder to which we should move the item</returns>
  7.         private SPFolder GetMonthlyFolder(string monthName, SPList list)
  8.         {
  9.             SPFolder monthlyFolder = null;
  10.             //loop through to see if the folder exists
  11.             foreach (SPListItem folder in list.Folders)
  12.             {
  13.                 if (folder.Name.Equals(monthName, StringComparison.CurrentCultureIgnoreCase))
  14.                 {
  15.                     monthlyFolder = folder.Folder;
  16.                     break;
  17.                 }
  18.             }
  19.  
  20.             //if we don't have it create it
  21.             if (monthlyFolder == null)
  22.             {
  23.                 monthlyFolder = list.RootFolder.SubFolders.Add(monthName);
  24.                 monthlyFolder.Update();
  25.             }
  26.  
  27.             return monthlyFolder;
  28.         }

 

Conclusion

Putting all this together hopefully some people will find this an interesting article with some useful ideas. As always if anyone who reads this can think of any improvements I am always happy to discuss them.

Lastly if you do use this code as a base for your own project please ensure this is tailored to meet our own solution and has been properly tested on a development environment before being deployed to a live farm.

Advertisements

%d bloggers like this: