Archiving documents timer job

February 7, 2012

Whilst working a recent project I had to create a timer job which moved some InfoPath forms from an archive document library to a location which was dynamically generated depending on the forms modified date. I am not going to cover the basics on how to create a timer job as Andrew Connell already has a very good article on how to create a custom timer job.

In this article I will focus on how to move a InfoPath form but it could equally to word documents, PDF, etc. The same could apply to list data but it will require some code changes.

As with all my projects I first mapped out what steps, see below.

  1. Get configuration data
  2. Open the site in which my forms are stored
  3. Get the document library
  4. Get the items which meet a certain criteria
  5. Iterate through the items and for each item perform the following
    1. Get the form modified date
    2. Check and see if a document library already exists for the current year and if not create one
    3. In the document library check if a folder for the month exists and if not create it
    4. Copy the form to the new location
    5. Set the metadata to be copied across. One consideration that may apply to others is version, however this was not relevant for me.
    6. Delete the original

I will assume you have read the post above by Andrew Connell so I will jump in once we have already got our site collection.

Point 1

As with all projects you want to minimise the amount of data which is hard coded as this reduces the need for additional deployments when certain environment specific variables change. There are a few options on where to store these but I decided to store mine in the web.config of the web application. This requires a few steps to read the data from the web.config so I created a helper method which returned a custom class which is used to store the variables. 

  1. ConfigurationData config = GetConfigurationData(webApplication.Name);

 

  1. /// <summary>
  2.         /// Method to get the configuration data from the web.config
  3.         /// </summary>
  4.         /// <param name="webAppName">A string of the web application name</param>
  5.         /// <returns>An ConfigurationData object with the configuration details</returns>
  6.         private ConfigurationData GetConfigurationData(string webAppName)
  7.         {
  8.             ConfigurationData configData = new ConfigurationData();
  9.             Configuration config = WebConfigurationManager.OpenWebConfiguration("/", webAppName);
  10.             if (config != null)
  11.             {
  12.                 if (config.AppSettings.Settings["Property1"] != null)
  13.                 {
  14.                     configData.Property1 = config.AppSettings.Settings["Property1"].Value;
  15.                 }
  16.  
  17.                 if (config.AppSettings.Settings["Property2"] != null)
  18.                 {
  19.                     configData.Property2 = config.AppSettings.Settings["Property2"].Value;
  20.                 }
  21.  
  22.                 if (config.AppSettings.Settings["Property3"] != null)
  23.                 {
  24.                     configData.Property3 = Int32.Parse(config.AppSettings.Settings["Property3"].Value);
  25.                 }
  26.             }
  27.  
  28.             return configData;
  29.         }

Points 2, 3 and 4

With the configuration data retrieved from the web.config the next few steps are very straightforward to anyone who has done any SharePoint development.

  1. //open site
  2.             using (var site = new SPSite(siteCollection.Url))
  3.             using (var web = site.OpenWeb(config.Property1))
  4.             {                
  5.                 web.AllowUnsafeUpdates = true;
  6.                 try
  7.                 {
  8.                     SPList list = null;
  9.                     try
  10.                     {
  11.                         //get library
  12.                         list = web.Lists[config.Property2];
  13.                     }
  14.                     catch (ArgumentException ae)
  15.                     {
  16.                         LogDetails(String.Format("There was a problem gettting the archive list, the error was {0}", ae.ToString()));
  17.                         return;
  18.                     }
  19.  
  20.                     if (list == null)
  21.                     {
  22.                         LogDetails("Unable to get the archive list");
  23.                         return;
  24.                     }                    
  25.  
  26.                     var query = new SPQuery();                    
  27.                     query.Query = "YOUR CAML QUERY";
  28.  
  29.                     LogDetails(String.Concat("CAML Query is ", query.Query));
  30.  
  31.                     //get items
  32.                     var listItems = list.GetItems(query);
  33.                     var listItemCount = listItems.Count;
  34.                     if (listItems == null || listItemCount == 0)
  35.                     {
  36.                         LogDetails(String.Concat("There were no items returned by the query {0}", query.Query));
  37.                         return;
  38.                     }
  39.  
  40.                     LogDetails(String.Concat("Got items ", listItemCount.ToString()));

Point 5

At this stage we now have a list of all items items which meet the relevant archive criteria but for obvious reason the actual logic has been removed from this blog. My next step was to iterate through all items and move them to the appropriate location. At first I used a foreach loop but this didn’t work as I was adjusting the item collection and this caused a runtime error. Next I tried a for loop using the count of the number of items. While this seemed to work I found it was only iterating through half of the list and after the half way point I was getting an error saying “Specified argument was out of the range of valid values.”. For example if the count of items was 10 it could loop through items 1-5 but as soon as it reached 6 it give the error above. To get around this I changed the code to start at the last item in the count and work backwards i.e. 10, 9, 8. To keep the code contained I separated the main functionality out into a few different methods.

Loop through items
  1. //loop through items moving them
  2.                     for (int itemNumber = listItemCount; itemNumber > 0; itemNumber–)
  3.                     {
  4.                         try
  5.                         {
  6.                             LogDetails(String.Concat("Start item ", itemNumber.ToString()));
  7.                             SPListItem listItem = listItems[itemNumber -1];
  8.                             if (listItem == null)
  9.                             {
  10.                                 LogDetails(String.Format("There was a problem getting the list item at index {0} returned by the query {1}",
  11.                                     itemNumber.ToString(), query.Query));
  12.                             }
  13.                             else
  14.                             {
  15.                                 MoveItem(listItem, web);
  16.                             }
  17.                         }
  18.                         catch (Exception ex)
  19.                         {
  20.                             LogDetails(ex.ToString());                            
  21.                         }                                                                        
  22.                     }

 

This function takes an item, gets the modified date and passes this to another helper function which gets the folder which the item has to be moved to. Next it builds up the URL to where the item has to be copied to. The item is then moved but because this method doesn’t return the SPFile object I had to then get it from the destination folder. Once I had the new item I then set some properties to ensure metadata is retained as otherwise the created and modified details would be incorrect. Finally the original item is deleted.

Function for each item
  1. /// <summary>
  2.         /// Method to perform the move of an individual item
  3.         /// </summary>
  4.         /// <param name="listItem">An SPListItem of the item to be moved</param>
  5.         /// <param name="web">An SpWeb object which represents the web in which the item is located</param>
  6.         private void MoveItem(SPListItem listItem, SPWeb web)
  7.         {           
  8.             DateTime modifiedDate = DateTime.Parse(listItem[SPBuiltInFieldId.Modified].ToString());            
  9.             SPFolder destinationLocation = GetDestinationLibrary(modifiedDate, web);            
  10.             string destinationLocationURL = String.Format("{0}/{1}/{2}", web.Url, destinationLocation.Url, listItem.File.Name);           
  11.  
  12.             try
  13.             {
  14.                 listItem.File.CopyTo(destinationLocationURL);
  15.             }
  16.             catch (Exception ex)
  17.             {
  18.                 LogDetails(String.Concat("There was a problem copying the file to the new URL, the error was {0} ",
  19.                     ex.ToString()));
  20.                 throw;
  21.             }
  22.  
  23.             SPFile file = null;
  24.             try
  25.             {
  26.                 file = destinationLocation.Files[listItem.File.Name];
  27.             }
  28.             catch (Exception ex)
  29.             {
  30.                 LogDetails(String.Format("There was a problem getting the new file {0} in location {1}, the error was {2}",
  31.                     listItem.File.Name, destinationLocation.Url, ex.ToString()));
  32.                 throw;
  33.             }
  34.  
  35.             if (file == null)
  36.             {
  37.                 LogDetails(String.Format("Unable to find file {0} in new location.",
  38.                     listItem.File.Name));
  39.                 throw new NullReferenceException("Unable to find new file in destination location");
  40.             }
  41.  
  42.             var fileItem = file.Item;
  43.  
  44.             try
  45.             {
  46.                 /*since we are running with elevated permissions we need to set the
  47.                  * author and editor back to the details from the original item
  48.                  * otherwise it will appear as system account                                         *
  49.                  * */
  50.                 fileItem[SPBuiltInFieldId.Author] = listItem[SPBuiltInFieldId.Author];
  51.                 fileItem[SPBuiltInFieldId.Created] = listItem[SPBuiltInFieldId.Created];
  52.  
  53.                 fileItem[SPBuiltInFieldId.Editor] = listItem[SPBuiltInFieldId.Editor];
  54.                 fileItem[SPBuiltInFieldId.Modified] = listItem[SPBuiltInFieldId.Modified];
  55.                 fileItem.UpdateOverwriteVersion();
  56.  
  57.                 listItem.Delete();
  58.             }
  59.             catch (Exception ex)
  60.             {
  61.                 //log error and move on to next record
  62.                 LogDetails(String.Format("There was a setting the file properties for item {0} the problem was {1}",
  63.                     listItem.Title, ex.ToString()));
  64.                 throw;
  65.             }
  66.         }

As mentioned above the previous snippet calls this GetDestinationLibrary method which uses the modified date to see if a document library exists for the year in the same web and if not it creates one. Next it calls a separate function which performs the same idea but this one uses the month and creates a folder inside the document library.

Get document library
  1. /// <summary>
  2.         /// Method to get the URL of document library which the list item should be added to. If the library is not found one is created.
  3.         /// </summary>
  4.         /// <param name="modifiedDate">A DateTime of the last modified date of the item</param>
  5.         /// <param name="web">An SPWeb object representing the web which contains the items</param>
  6.         /// <returns>An SPFolder of the location the item should be copied to</returns>
  7.         private SPFolder GetDestinationLibrary(DateTime modifiedDate, SPWeb web)
  8.         {
  9.             string destinationLibraryURL = String.Empty;
  10.             string year = modifiedDate.Year.ToString();
  11.  
  12.             SPList desintationDocumentLibrary = null;
  13.  
  14.             try
  15.             {
  16.                 //get library
  17.                 desintationDocumentLibrary = web.Lists[year];
  18.             }
  19.             catch (ArgumentException)
  20.             {
  21.                 desintationDocumentLibrary = CreateDocumentLibrary(year, web);
  22.             }
  23.  
  24.             SPFolder monthlyFolder = GetMonthlyFolder(CultureInfo.CurrentCulture.DateTimeFormat.GetMonthName(modifiedDate.Month), desintationDocumentLibrary);
  25.  
  26.             return monthlyFolder;
  27.         }

 

Create monthly folder
  1. /// <summary>
  2.         /// Method to try and find the folder representing the month given a month name. If a month is not found one is created.
  3.         /// </summary>
  4.         /// <param name="monthlyName">A string of the month name</param>
  5.         /// <param name="list">An SPList which is the list the folder will be created in</param>
  6.         /// <returns>An SPFolder representing the folder to which we should move the item</returns>
  7.         private SPFolder GetMonthlyFolder(string monthName, SPList list)
  8.         {
  9.             SPFolder monthlyFolder = null;
  10.             //loop through to see if the folder exists
  11.             foreach (SPListItem folder in list.Folders)
  12.             {
  13.                 if (folder.Name.Equals(monthName, StringComparison.CurrentCultureIgnoreCase))
  14.                 {
  15.                     monthlyFolder = folder.Folder;
  16.                     break;
  17.                 }
  18.             }
  19.  
  20.             //if we don't have it create it
  21.             if (monthlyFolder == null)
  22.             {
  23.                 monthlyFolder = list.RootFolder.SubFolders.Add(monthName);
  24.                 monthlyFolder.Update();
  25.             }
  26.  
  27.             return monthlyFolder;
  28.         }

 

Conclusion

Putting all this together hopefully some people will find this an interesting article with some useful ideas. As always if anyone who reads this can think of any improvements I am always happy to discuss them.

Lastly if you do use this code as a base for your own project please ensure this is tailored to meet our own solution and has been properly tested on a development environment before being deployed to a live farm.


InfoPath 2007 File name Validation

June 24, 2011

I am relatively new to InfoPath but realise it has a lot of uses so when a support call for an Issue with an InfoPath form was escalated to the development team I offered to have a look.

It had already been established that the issue was down to the fact that the file name generated in the form occasionally contained Illegal characters, see list below, thus couldn’t be saved into the SharePoint document library. My first reaction was this is easy in C# but how do I manage this in InfoPath and the answer, well the one I choose, was using C# 🙂

Looking at the existing form I could see there was a few Rules and Actions set up on the submit button one of which was to generate the file name using the formula below

Existing formula for generating the file name
concat(“PREFIX-“, FIELD_NAME, “-“, now())

My first thought was I might be able to change this formula somehow but decided against this as I wasn’t sure if it was even possible and if it was then it would make the formula, potentially, too complex. Instead I opted to create a hidden field and use the Changed Event on the existing field to get the value entered by the user, strip out any characters I didn’t want and then set the value of my hidden field. I could then change my formula on the submit button to use my new formatted field instead of the original and bingo it works.

I have included the code I used to strip out the Illegal characters below.

New formula for generating the file name
concat(“PREFIX-“, NEW_FIELD_NAME, “-“, now())

Illegal characters
[ ~ # % & * { } \ : < > ? /

public void FIELD_NAME_Changed(object sender, XmlEventArgs e)
{
      //characters not allowed
      string[] invlaidChars = new string[] { “-”, “‘”, “[“, “~”, “#”, “%”, “&”, “*”, “{“, “}”, “\\”, “:“, “<“, “>“, “?“, “/”}; 

      //check the value has changed 
      if (e.OldValue.Equals(e.NewValue))
      {
           return;
      } 

      string newFieldValue = e.NewValue; 

      //loop through invlaid characters and remove them
      foreach (string invalidChar in invlaidChars)
      {
          if (formattedValue.Contains(invalidChar))
          {
               newFieldValue = newFieldValue.Replace(invalidChar, ““);
           }
       } 

      XPathNavigator nav = MainDataSource.CreateNavigator();

      //find formatted field 
       XPathNavigator node = nav.SelectSingleNode(“/my:myFields/my:FIELD_NAME”, NamespaceManager);

      //check we have found it
      if (node != null)
      {
           //set the value
           node.SetValue(newFieldValue);
       }
}

 


%d bloggers like this: