Right SharePoint 2013 APIs for your needs – A ready reckoner

SharePoint products team amazes me every time they introduce a product version release. There are several options available when it comes to access SharePoint 2013 using APIs. Per my understanding the following will be the factors that decide which option to use

• Type of customization like web parts, Silverlight App, SharePoint App Page, timer job and more. All these customization are different in most aspects and choosing right API option would eventually decide its eventual success
• Enterprise governance rules will be the key differentiator in deciding the API option.
• Development skill level is the other factor. Expertize in a the technologies like .NET Framework, Silverlight, PowerShell, JavaScript, ASP.NET, REST/OData will ensure the best development practices are applied in design / development of custom components
• Devices we are targeting like farm server, client computer, mobile devices etc. The customization targeting different device should use appropriate API option.

Let me share a high level mapping based on the current info ...

API Option	Custom Component
Server Object Model	PowerShell script for farm / web app maintenance, tracking etc
	Timer job to perform scheduled actions on web app
	Web Parts to be displayed in sites
Silverlight client object model	Web Parts to be displayed in sites
	SharePoint  App
	Silverlight Application that integrate with SharePoint site data
	ASP.Net web application that integrate SharePoint site data
REST/Odata Endpoints	ASP.Net web application that integrate SharePoint site data
	SharePoint  App
	HTML/Javascript application that consume SharePoint site data
	App for office
	LAMP web application that integrate SharePoint site data
	iOS/Android app for mobile devices that access SharePoint site data
Javascript APIs	Web Parts to be displayed in sites
	ASP.Net web application that integrate SharePoint site data
	SharePoint  App
	App for office
	HTML/Javascript application that consume SharePoint site data
.Net Client Object Model	.Net application that consume SharePoint site data
Mobile Silverlight API	Silverlight app for windows phone
	XNA app for Windows Phone

more to follow... 

Back to Basics - 1 : Right from web apps and site collections

Basically a SharePoint Web application is a logical abstraction on top of IIS. Physically, a Web Application is a "collection of one or more IIS Web sites configured to map incoming HTTP requests to a set of SharePoint sites."  A mapping of each SharePoint site to one or more content databases that are specified within Central Administration.

Ideally we should primarily utilize separate Web apps for exposure to differing environments for example have intranet and extranet Web apps using HTTP and SSL respectively. As far as scalability is concerned, Web apps support up to 300 content databases and a recommended limit of around 2000 site collections per content DB assuming your infrastructure is up to it.

Adding additional Web apps adds additional administrative and infrastructure burden and - wherever possible - I would suggest using as few as possible. The decision to go with Web apps vs. site collections should be made as part of a wider discussion around your infrastructure.

When we need more than one web application? if there a  need of

• different authentication provider
• different service applications group association
• application pool isolation (e.g. for fault tolerance or security)
• different url (with alternate access mappings)
• different web.config
• different bin folder
• differing throttling settings
• differing security policies. (e.g. set up a "finance" Web app with a locked down user policy, whereas a collaborative Web app may be less restrictive) 

What additional overheads are associated with a Web application?

• Additional RAM required per Web app (even with 0 users) > more hardware required.
• Timer jobs (around 30 per Web app, several of which run every few minutes (e.g. alerts & workflow) and many which run daily (e.g. workflow processing and gradual site deletions)
• More Content databases - per SQL instance limit, backup concerns, storage space
• IIS sites > administrative overheads (host headers, IP addresses, SSL certificates, Web configuration files, assemblies).
• Other operational overheads: DNS / request routing; SharePoint solution management 

I would suggest having different site collection for different department, because

• site collection is root level for many settings starting from permissions,  site collection features like branding, master page, css, etc.
• Also, there are features like Move-SPSite which moves one site collection to a separate content database.
• If your SP Farm grows in time, it will be more scalable if each department having its own site collection is moved in separate content db.
• With this kind of structure, you will prevent having huge databases, have better performance, etc.

Record Management capabilities of SharePoint 2010

Last year I was part of an initiative to assess SharePoint 2010 capabilities around records / document management. Spend few weeks to compare business requirements and found the following info...

Functionality	Description	Customizations Required to address majority of enterprises' need	Rating out of 5
Declare record  in-place manually	Option to mark a sharepoint document as record manually to prevent further changes   and deletion	NA	5
Declare records by manually moving it to record center	The “send to Record Center” option to manually declare records and send them to the RecordCenter is   located in Compliance Details overlay. It would be better if its available in the ribbon / pull down menu to make it more usable.	Custom command in ribbon to trigger "send to record center" action	4.5
Declare records automatically based on rule	Process to define rules that mark a sharepoint document as record  to prevent further changes and deletion	NA	5
Retain records and their associated metadata for   their retention time period	Whenever user changes the metadata of a record in Record Center,   system is not synchronizing the metadata changes to the record link which is located in the document center.	Custom component to                         i.      Prevent metadata updation of a record link                       ii.      Propagate metadata changes to record links in document center, whenever metadata values of the linked record has been changed	4
Ensure Record integrity throughout their retention time period	Only Selected metadata fields should be editable. metadata column edit   permissions should be based on the user role   / state of record matrix	Custom feature to restrict updation of critical metadata / record content by applying field level locking based on the following parameters                         i.      Permission Level of the user                       ii.      Stage of document (record, hold)	0
Dispose records in accordance with the Records Retention Schedule	Retention and disposition process of out-of-the-box SharePoint 2010 executes on individual items. The high volume of records for disposition will cause a critical usability issue of managing disposition of individual items. It is recommended to have a batch disposition option. Out-of-the-box SharePoint terminates the expiration process whenever user cancels disposition approval request. It is recommended to have a “Cancelled Records Repository” to restart “disposition of rejected records” / put them on hold / assign a different content type. A workflow to redirect records to record library / document library on an event of cancelled item metadata updation is recommended.	Custom disposition process with the following high level functionalities                         i.      Disposition schedules should run on scheduled intervals (monthly, quarterly, yearly)                       ii.      Records/documents of sub site (project or dept) should be batched together and send for disposition approval                     iii.      Record Stewards should be able to pick and choose items from the list and approve disposition.                     iv.      Record Stewards should also able to apply hold, and change metadata to items by specifying valid remarks. These disposition cancelled records will be sent to Record Administrator for approval                       v.      The Record Administrator should be able to review the disposition of cancelled records and apply the changes suggested by the Record Steward / himself. When Record Administrator approves the changes obtained through disposition cancellation, record should move to Record Center or document center.	1.5
Discover Records for hold and apply hold in batch	The following changes in eDiscovery for Hold are recommended:                         i.      Search refinement with advanced search feature to find actual record set for hold                       ii.      Pick and choose option from the search results Every legal hold search should find the documents and links to records. On applying hold to record links, it should apply hold to the linked records. The following advanced options to streamline the hold process in batch are recommended:                         i.      Apply hold to multiple records after selecting them from a list.                       ii.      Applying hold to a library / folder	Advanced Search functionality to search records to apply hold. Custom solution to apply hold only for those records chosen from the search results. Custom solution to Select multiple records from a library and apply hold. Provision to apply hold on all records/documents of a specific library or folder The eDiscovery search in Document Center should find   documents and links to record along with in-place records.	3
Apply legal Hold to a Record manually	Option to hold a record from further processing and change	NA	5
Release legal Hold from a Record manually	Option to release hold a record to allow  further processing and change	Component to apply hold to the linked record on applying hold in record link.	1.5
Release legal Hold from set of Records	“Search and Release Hold” option should be available which would enable user to release hold by selecting records from search result	Advanced Search functionality to search records on hold. Custom solution to release hold only for those items chosen from the hold search results. Component to Select multiple records from a library and release hold. Component to release hold from the linked record on releasing hold from a record link	1.5
Delete an in place record / record in record center   / record link/ record on hold	Since SharePoint considers a link to a record as an independent entity, users with “contribute” access can delete record links. This action should be restricted to Administrators to ensure accessibility.	Custom feature to prevent record link deletion	3.5
Move an in place record / record in record center / record link/ record on hold	Since SharePoint considers a link to a record as an independent entity, users with “contribute access” can move / edit a record link. This action should be limited to Record Administrators (to make it consistent with Record Center). Also, the system should prevent metadata changes of record link in Document Center. The ability to move documents on hold should be restricted to Record Administrators.	Custom feature to restrict record link and hold records movement.	1
Create a new   version of an in place record / record in record center / record link/ record on hold	SharePoint allows users with “contribute access” to edit all metadata fields. However, only selected metadata fields should be editable based on the logged-in user’s role and status of the record. Updation of metadata (like dates) should recalculate the information management processing dates.	Custom feature to restrict updation of critical metadata / record content by applying field level locking based on the following parameters                         i.      Permission Level of the user                       ii.      Stage of document (record, hold)	0
Upload a new   version of an in place record / record in record center / record link/ record on hold	SharePoint allows users with “contribute access” to replace both in-place records and record links with a new version. This should not be allowed.	Custom feature to prevent in place, record link updation with a new version of document. Custom component to                         i.      Prevent metadata updation of a record link Propagate metadata changes to record links in document center, whenever metadata values of the linked record have been changed.	0
Modify an in place record / record in record center   / record link/ record on hold metadata	SharePoint allows users with “contribute access” to edit all metadata fields. However, only selected metadata fields should be editable based on the logged-in user’s role and status of the record. Updation of metadata (like dates) should recalculate the information management processing dates.	Custom feature to restrict updation of critical metadata / record content by applying field level locking based on the following parameters                         i.      Permission Level of the user Stage of document (record, hold)	0
Email an in place record / record in record center / record link/ record on hold	When we send link to record by attaching it with an email, it should attach the actual record, not the link file.	Custom feature that attaches record with an e-mail when user tries to send a link to record from document center.	4
Apply information management policies	Allow information management policies to process metadata and perform associated actions including record declaration	NA	5
Apply information security by setting up permissions for the library	For certain record types, System must ensure item level permission setting on upload	Custom code component to apply item level permission while uploading a document.	4.5
Generate File   Plan Report	SharePoint 2010s out-of-the-box file plan report is not sufficient to address user needs. A custom implementation to display hierarchical report of taxonomy with record retention details is recommended.	Custom Hierarchical file plan report that displays:                         i.      Site information organization (Site Collections>Sites>Libraries>Folders>Content Type)                       ii.      Record retention details (Libraries>Content Types)	1
Generate Records Audit Report	Out of the box report neither satisfy specify user requirements nor compliant with data sensitivity guideline.  Requires custom code components on top of the log data to make it usable.	Custom audit log report that can perform search SharePoint audit log data and present info related to:                         i.      User actions                       ii.      Action during a period                     iii.      Specific actions (like delete)                     iv.      Failed attempts [optional]"	1
Find a record	It is recommended to use FAST 2010 for search   integration.	Custom options to distinguish records, information on hold within search results.	3.5
Find an in place record / record in record center / record link/ record on hold	Search results do not distinguish in-place records from documents. There should be some mechanism to distinguish type of documents and records.	Option to filter search results based on the type of item (document / in-place record / hold / record link).	3.5

Bottomline:  SharePoint 2010 records management policies would be suited for a small enterprise with a limited number of record types and less repositories and have a straight forward business process to manage info.

Manage Structured and Unstructured Data - SharePoint 2010 way - Part 1

Now a day’s majority of enterprises business processes and functions are facilitated or supported by the use of electronic systems.  Many of these systems create, manage and/or store electronic data that is identified as either “structured” or “unstructured”.  

“Structured data” typically is created and stored according to a pre-defined data model and fits into relational tables, or can be stored in rows and columns. By comparison, “unstructured data” has minimal identifiable structure, and may include e-mail files, word-processing text documents, PowerPoint presentations, JPEG and GIF image files, and MPEG video files.  

Unstructured data may be stored in collaborative environments, in document management systems, on file shares, etc. The management of unstructured data poses a significant challenge to most organizations today, since it relies heavily on individual user management and falls outside the purview of the traditional IT System information management model. Most unstructured content is created in an ad hoc fashion by individual business users, with little consistency in creation, naming, storing, disposing, etc. If left unmanaged, the sheer volume of unstructured data that is generated each year within an enterprise can be costly in terms of storage. In addition to these “hard dollar” costs, significant inefficiencies result from ineffective use of knowledge worker time when searching and finding needed information, and then (hopefully) classifying that information.

Unmanaged data can also pose a liability if information cannot be located in the event of an audit, investigation or litigation.  There is further risk related to records and information that should have been disposed of in accordance with retention policy, but continues to be retained. 

Im starting a series of posts around SharePoint 2010's capabilities to manage structured /unstructured data .... more to follow....

Update created by column of a list item with a value using event handler

We tried several options to update created by column of a list item with a value using event handler
We found the way to do it .... Its below


public override void ItemAdded(SPItemEventProperties properties) 
        {
            base.ItemAdded(properties);
            ChangeCreatedBy(properties); 
        }
        private void ChangeCreatedBy(SPItemEventProperties properties)
        {  
            string strAuthor = @"domain\first.last";
            try
            {
                Guid siteID = properties.ListItem.ParentList.ParentWeb.Site.ID; 
                Guid webID = properties.ListItem.ParentList.ParentWeb.ID; 
                Guid listID = properties.ListItem.ParentList.ID; 
                Guid newItemID = properties.ListItem.UniqueId; 
                SPSecurity.RunWithElevatedPrivileges(delegate() { 
                    using (SPSite site = new SPSite(siteID)) 
                    { 
                        using (SPWeb web = site.OpenWeb(webID))
                        {
                            SPUser usrAuthor  = web.EnsureUser(strAuthor);
                            SPList questionList = web.Lists[listID]; 
                            SPListItem listItem = questionList.Items[newItemID];  
                            listItem["Author"] =  usrAuthor ;
                            listItem.SystemUpdate(false);
                        } 
                    } 
                }); 
            }
            catch (Exception ex)
            {
                //call  log error routine 
            }
        }