Tokenize your XDTs with Powershell

XDTs have gotten a bad rep over the years for being difficult to use and hard to understand. However despite that, they’re still the most reliable and consistent way to transform configurations. I’ve come up with a way to tokenize those XDTs to make them able to be used in a more flexible way.

For example say we have different cookie domains per environment that we want to patch in and out.

note: This code requires the dll Microsoft.Web.XmlTransform.dll to be in the same folder as the powershell script

param(
    [string]$Path,
    [string[]]$XDTs,
    [hashtable]$Tokens
)

function Update-XmlDocTransform($xml, $xdt, $tokens)
{
    Add-Type -LiteralPath "$PSScriptRoot\Microsoft.Web.XmlTransform.dll"

    $xmldoc = New-Object Microsoft.Web.XmlTransform.XmlTransformableDocument;
    $xmldoc.PreserveWhitespace = $true
    $xmldoc.LoadXml($xml);
    $useTokens = $false
    if ($tokens -ne $null -and $tokens.Count -gt 0){
        $useTokens = $true
        $sb = [System.Text.StringBuilder]::new((Get-Content -Path $xdt))
        $tmpPath = "$($env:TEMP)\$([guid]::NewGuid()).xdt"
        $tokens.Keys | ForEach-Object{
            $null = $sb.Replace($_, $tokens[$_])
        }
        Set-Content -Path $tmpPath -Value $sb.ToString()
        $xdt = $tmpPath
    }
    
    $transf = New-Object Microsoft.Web.XmlTransform.XmlTransformation($xdt);
    if ($transf.Apply($xmldoc) -eq $false)
    {
        throw "Transformation failed."
    }
    if ($useTokens){
        Remove-Item -Path $xdt -Force
    }
    return $xmldoc.OuterXml
}

$contents = Get-Content $Path | Out-String
$XDTs | Foreach-Object{
    $contents = Update-XmlDocTransform -xml $contents -xdt $_ -tokens $Tokens
}
Set-Content $path -Value $contents

Here is an example usage:

LocalXmlTransform.ps1 -Path "C:\inetpub\wwwroot\sc901.local" -XDTs "C:\xdt\AddBindingRedirects.xdt","C:\xdt\AddSessionCookie" -Tokens @{_ShareSessionCookie_="mysite.local";_RedirectName_="mydependency"}

In this example we’re running two XDT files against the web.config and replacing a couple of tokens in the XDT.

Here is an example of an XDT with tokens to ensure a connection string exists:

<?xml version="1.0" encoding="utf-8"?>
<connectionStrings xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform" xmlns:asmv1="urn:schemas-microsoft-com:asm.v1">
  <add name="_name_" xdt:Transform="Remove" xdt:Locator="Match(name)" />
  <add name="_name_" xdt:Transform="InsertIfMissing" xdt:Locator="Match(name)" connectionString="Encrypt=True;TrustServerCertificate=False;Data Source=_fqdn_;Initial Catalog=_databasename_;User Id=_username_;Password=_password_;" />
</connectionStrings>

To use this XDT your parameters would look something like this:

LocalXmlTransform.ps1 -Path "C:\inetpub\wwwroot\App_Config\ConnectionStrings.config" -XDTs "C:\xdt\EnsureConnectionString.xdt" -Tokens @{_name_="mySpecialDatabase";_fqdn_="myazurestuff.database.windows.net,1433";_databasename_="specialdatabase";_username_="secretuser";_password_="secretpassword"}

Hopefully this will help your devops process

PAAS Sitecore 9 with an ASE ARM template errors

ase
Sitecore 9 works great in PAAS and the arm templates are an enormous help. However if you’re like me and need to use an ASE then you find that your deployments are regularly and mysteriously failing. I poured over the arm templates searching for any reason that this might be happening. After about a month i accepted the unfortunate truth that Azure was incorrectly reporting success before it should.

I started pulling apart the templates searching for more information. I utilized a custom powershell containment system to manage the ARM template parameters

The errors originated from the application deployments. These are the parts that use web deploy to restore databases, create users, and push files to your Sitecore server.

How to stabilize the arm templates

For this i will assume that you’ve already added the hostingEnvironmentProfile parameter to the Microsoft.Web/sites ARM resources
Warning: this process is very time consuming
The first step is to pull them apart. I was able to achieve a high success rate by doing the following:

  1. Take the main azuredeploy.json and remove all of the resources, we’re going to be manually executing them
  2. In each of the nested ARM template json make sure that the required parameters are defined in the parameters and variables section, you can refer back to the azuredeploy.json for how these should be setup
  3. The application.json file is the primary culprit that’s causing our failures, we need to split this one up just like we did the azuredeploy.json except this time we’re going to be creating new ARM template json files for each of the 4 web deploy deployments that reside in application.json
  4. Now that we have the ARM templates separated out into their individual parts we need to create a new powershell wrapper for the process
  5. Note, for security reasons i’m largely omitting things of a sensitive nature here. Make sure you apply user names and passwords to your input parameters either in a parameters.json or in the parameters powershell hashtable as described below

    Powershell Magic

    powershell
    You can find the scripts Here

    Due to the lack of a central ARM template to orchestrate parameters we need to do that ourselves, this comes in a few steps

    1. populate all starting parameters in a hashtable. see Execute.ps1 for an example. Note that you will need to pass in several more parameters or you can include them in a parameters.json that’s loaded here
    2. Scan the arm templates and gather their accepted parameters as they won’t take any extra. See Get-ValidParameters in Utilities.ps1
    3. Based on each ARM template gather up the parameters needed for the deployment and generate a new hashtable of parameters and their values. See Get-Parameters in Utilities.ps1
    4. Execute ARM template using a modified version of Sitecore’s ARM template execution code. See Start-SitecoreAzureDeployment in Utilities.ps1
    5. After completion extract out populated parameters and outputs and save them using Get-ValidParameters from Utilities.ps1
    6. repeat until finished. You can see how the arm templates are ordered here.
      Note, depending on your specific case, you may need to adjust some timing between deployments if some deployments need more time to settle

Sitecore Helix Powershell Filewatch

Most likely if you’re developing with Sitecore you have your webroot and your source separated and publish to your site with webdeploy or some other kind of publishing technology.  This is a fine way to do it, but it’s far easier if it just happens automatically.  that’s what these scripts aim to do with Powershell!

Filewatch

Here is a zip file for the completed solution that is covered below: HelixFileWatcher. Or check out the source View on Github

First we need to define a few parameters:

#where your solution is
$SourceDirectory = "D:\Source\SitecoreSource\SolutionRoot"
#where your webroot is
$DeployTargetWebPath = "C:\inetpub\wwwroot\sc90.local"

Next we define how files are moved from your solution to the webroot, this is done through a hashtable matching a file extension to a script block. Note that the views are being deployed to a “Demo” MVC Area as an example.

$global:FileWatchActions = @{}
function Get-ProjectRoot{
	param(
		[string]$Path
	)
	if ($path -eq [string]::Empty){
		return [string]::Empty
	}
	if (-Not (Test-Path $Path)){
		return Get-ProjectRoot -Path (split-Path $Path)
	}
	$PathItem = Get-Item -Path $Path
	if (-Not ($PathItem -is [System.IO.DirectoryInfo])){
		return Get-ProjectRoot -Path (Split-Path $Path)
	}
	if ((resolve-path "$Path\*.csproj").Count -gt 0){
		return $Path
	}elseif($PathItem.Parent -ne $null){
		return Get-ProjectRoot -Path $PathItem.Parent.FullName
	}
	return [string]::Empty
}
function Copy-ItemToWebroot{
	param(
		$Path,
		$OldPath,
		$Delete,
		$Index,
		$IntermediatePath
	)
	if ($Index -lt 0){
		return
	}
	
	$TargetPath = $DeployTargetWebPath + $IntermediatePath + $Path.Substring($Index)
	if ($Delete -and (Test-Path $TargetPath)){
		write-host "Removing file $TargetPath" -ForegroundColor Red
		Remove-Item $TargetPath -Force -Recurse
	}elseif (-Not (Test-Path $Path) -and (Test-Path $TargetPath)){
		write-host "Removing file $TargetPath" -ForegroundColor Red
		Remove-Item $TargetPath -Force -Recurse
	}elseif(Test-Path $Path){
		if ($OldPath -ne [string]::Empty){
			$OldTargetPath = $DeployTargetWebPath + $IntermediatePath + $OldPath.Substring($Index)
			if ((Test-Path $OldTargetPath) -and ((Split-Path $Path) -eq (Split-Path $OldPath) )){
				$newName = Split-Path $Path -Leaf -Resolve
				write-host "Renaming Item" -ForegroundColor Yellow
				write-host "    $OldTargetPath" -ForegroundColor Yellow
				write-host "    =>$TargetPath" -ForegroundColor Yellow
				Rename-Item $OldTargetPath $newName -Force
				return
			}
		}
		if (-Not (Test-Path $TargetPath) -or (Compare-Object (ls $Path) (ls $TargetPath) -Property Name, Length, LastWriteTime)){
			write-host "Copying Item" -ForegroundColor Green
			write-host "    $Path" -ForegroundColor Green
			write-host "    =>$TargetPath" -ForegroundColor Green
			New-Item -Path "$(Split-Path $TargetPath)" -ItemType Directory -Force
			Copy-Item -Path $Path -Destination $TargetPath -Recurse -Force
		}
	}
}

#Add watcher action configurations
#Based on extension define how to process the files that are changed
$global:FileWatchActions.Add(".cshtml", {
	param(
		$Path,
		$OldPath,
		$Delete
	)
	$index = $Path.IndexOf("\Views", 5)
	Copy-ItemToWebroot -Path $Path -OldPath $OldPath -Delete $Delete -Index $index -IntermediatePath "\Areas\Demo"
} )

$global:FileWatchActions.Add(".config", {
	param(
		$Path,
		$OldPath,
		$Delete
	)
	$index = $Path.IndexOf("\App_Config\Include", 5)
	Copy-ItemToWebroot -Path $Path -OldPath $OldPath -Delete $Delete -Index $index
	if ($index -eq -1){
		$fileName = Split-Path $Path -Leaf
		$FileDirectory = Get-ProjectRoot -Path $Path
		if ($fileName.StartsWith("web", "CurrentCultureIgnoreCase")){
			Copy-ItemToWebroot -Path $Path -OldPath $OldPath -Delete $Delete -Index $FileDirectory.Length -IntermediatePath "\Areas\Demo"		
		}
	}
} )

$global:FileWatchActions.Add(".dll", {
	param(
		$Path,
		$OldPath,
		$Delete
	)
	$index = $Path.IndexOf("\bin", 5)
	Copy-ItemToWebroot -Path $Path -OldPath $OldPath -Delete $Delete -Index $index	
} )

$global:FileWatchActions.Add("folder", {
	param(
		$Path,
		$OldPath,
		$Delete
	)
	if (-Not( $delete -or $OldPath -ne [string]::Empty)){
		return
	}
	$index = $Path.IndexOf("\Views", 5)
	if ($index -ne -1){
		Copy-ItemToWebroot -Path $Path -OldPath $OldPath -Delete $Delete -Index $index -IntermediatePath "\Areas\Demo"
		return		
	}
	$index = $Path.IndexOf("\App_Config\Include", 5)
	if ($index -ne -1){
		Copy-ItemToWebroot -Path $Path -OldPath $OldPath -Delete $Delete -Index $index -IntermediatePath "\App_Config\Include"
		return		
	}
})

Then we set up the file watchers to watch the important parts of our code

$global:LastEvent = ((Get-Date).AddSeconds(2).ToString('HH:mm:ss.fff'))
function global:Send-ChangesToWebroot{
	param(
	[string]$Path = [string]::Empty,
	[string]$OldPath = [string]::Empty,
	[bool]$Delete = $false
	)
	$extension = [IO.Path]::GetExtension($Path)
	$IsDirectory = $false
	if (Test-Path $Path){
		$IsDirectory= (Get-Item -Path $Path) -is [System.IO.DirectoryInfo]
	}elseif ($Delete -and $extension -eq [string]::Empty){
		$IsDirectory = $true;
	}
	try{
		if (-Not $IsDirectory -and $global:FileWatchActions.ContainsKey($extension)){
			$global:LastEvent = ((Get-Date).AddSeconds(2).ToString('HH:mm:ss.fff'))
			$global:FileWatchActions.Get_Item($extension).Invoke($Path, $OldPath, $Delete)
		}elseif ($IsDirectory){
			$global:LastEvent = ((Get-Date).AddSeconds(2).ToString('HH:mm:ss.fff'))
			$global:FileWatchActions.Get_Item("folder").Invoke($Path, $OldPath, $Delete)
		}
	}catch [System.Exception]{
		Write-Host "An error has occurred while attempting to run the processor for $extension" -ForegroundColor Red
		Write-Host "Path: $Path" -ForegroundColor Red
		Write-Host "OldPath: $OldPath" -ForegroundColor Red
		Write-Host $_.Exception.ToString() -ForegroundColor Red
	}
}
function Add-Watcher{
	param(
		$Directory
	)
	$Watcher = New-Object IO.FileSystemWatcher $Directory, "*" -Property @{IncludeSubdirectories = $true;NotifyFilter = [IO.NotifyFilters]'FileName, DirectoryName, LastWrite, Size'}
	
	Register-ObjectEvent $Watcher Changed -SourceIdentifier "$Directory FileChanged" -Action {Send-ChangesToWebroot -Path $Event.SourceEventArgs.FullPath}
	
	Register-ObjectEvent $Watcher Renamed -SourceIdentifier "$Directory FileRenamed" -Action {Send-ChangesToWebroot -Path $Event.SourceEventArgs.FullPath -OldPath $Event.SourceEventArgs.OldFullPath}
	
	Register-ObjectEvent $Watcher Deleted -SourceIdentifier "$Directory FileDeleted" -Action {Send-ChangesToWebroot -Path $Event.SourceEventArgs.FullPath -Delete $true}
	
	Register-ObjectEvent $Watcher Created -SourceIdentifier "$Directory FileCreated" -Action {Send-ChangesToWebroot -Path $Event.SourceEventArgs.FullPath}
	
	$Watcher.EnableRaisingEvents = $true
}
Resolve-Path "$SourceDirectory/*/App_Config/Include" | ForEach-Object{
	Write-Host "Adding watch location: $_" -ForegroundColor Yellow
	Add-Watcher $_ | Out-Null
}

Resolve-Path "$SourceDirectory/*/Views" | ForEach-Object{
	Write-Host "Adding watch location: $_" -ForegroundColor Yellow	
	Add-Watcher $_ | Out-Null
}

Resolve-Path "$SourceDirectory/*/bin" | ForEach-Object{
	Write-Host "Adding watch location: $_" -ForegroundColor Yellow
	Add-Watcher $_ | Out-Null
}

Resolve-Path "$SourceDirectory/*/Assets" | ForEach-Object {
	Write-Host "Adding watch location: $_" -ForegroundColor Yellow
	Add-Watcher $_ | Out-Null
}

Write-Host [string]::Empty
Write-Host "Now watching for changes made in the repo." -ForegroundColor Yellow
Write-Host "Any changes made will be delivered to the Webroot automatically" -ForegroundColor Yellow
Write-Host "***************************************************************" -ForegroundColor Yellow
while($true){
	#sleep more quickly when changes are happening
	if ($global:LastEvent -gt ((Get-Date).ToString('HH:mm:ss.fff'))){
		Start-Sleep -m 5
	}else{
		Start-Sleep 1
	}
}

Unicorn Automated Operations Authentication

I was attempting to set up some powershell scripts to deploy unicorn content and i was running into a strange issue where the Unicorn powershell module’s Unicorn sync request was getting redirected to the Sitecore login page.UnicornPowershellError

Through some debugging i was able to track this down to the fact that another developer had checked in a different shared secret.  Once the shared secret was fixed i had smooth sailing.

UnicornPowershellClear.png

Really the take away is always double check your shared secrets whenever there’s a problem with an automated tool.

Migrating Sidekick App 1.(1-2) to 1.4

Upgrading your Sidekick binaries and finding that you no longer can resolve your ScsHttpHandler type?  You will need to migrate.

Why did this happen?  Previously the ScsHttpHandler was a custom built HTTP handler built from the ground up, this allowed a guaranteed isolated context.  However i decided that the security and feature set of MVC was a far better medium for these apps and easier to understand than something custom.  Unfortunately this means some changes are required in order to convert your current Apps to use the new model.

The old way

namespace ScsJobViewer
{
	public class JobViewerHandler : ScsHttpHandler
	{
		//THE BELOW STUFF IS REGISTRATION STUFF, IT NOW BELONGS IN THE REGISTRATION FILE
		public JobViewerHandler(string roles, string isAdmin, string users)
			: base(roles, isAdmin, users)
		{
		}

		public override string Directive { get; set; } = "jvdirective";
		public override NameValueCollection DirectiveAttributes { get; set; }
		public override string ResourcesPath { get; set; } = "ScsJobViewer.Resources";
		public override string Icon => "/scs/jvgearwheels.png";
		public override string Name => "Job Viewer";
		public override string CssStyle => "width:600px";
		//THE BELOW STUFF IS THE CONTROLLER STUFF, IT NOW BELONGS IN THE CONTROLLER
		public override void ProcessRequest(HttpContextBase context)
		{
			string file = this.GetFile(context);

			if (file == "jvgetjobs.json")
			{
				this.ReturnJson(context, this.GetJobs(context));
			}
			else
			{
				this.ProcessResourceRequest(context);
			}
		}
		private object GetJobs(HttpContextBase context)
		{
			var data = GetPostData(context);
			var model = JobManager.GetJobs().Where(x => data.running ? !x.IsDone : x.IsDone).OrderBy(x => x.QueueTime);
			return model.Select(x => new JobModel(x));
		}
	}
}

Step 1

Take the routing part of your ScsHttpHandler and move it into a controller implementing ScsController.
Your old code:

		public override void ProcessRequest(HttpContextBase context)
		{
			string file = this.GetFile(context);

			if (file == "jvgetjobs.json")
			{
				this.ReturnJson(context, this.GetJobs(context));
			}
			else
			{
				this.ProcessResourceRequest(context);
			}
		}
		private object GetJobs(HttpContextBase context)
		{
			var data = GetPostData(context);
			var model = JobManager.GetJobs().Where(x => data.running ? !x.IsDone : x.IsDone).OrderBy(x => x.QueueTime);
			return model.Select(x => new JobModel(x));
		}

IMPORTANT
the controller should be decorated with an ActionName attribute that matches the request name made from the angular factory.
Should be mapped like this:

	class ScsJobViewerController : ScsController
	{
		//The action name should match what the angular factory is calling, note that case sensitivity isn't an issue.
		[ActionName("jvgetjobs.json")]
		public ActionResult GetJobs(bool running)
		{
			var model = JobManager.GetJobs().Where(x => running ? !x.IsDone : x.IsDone).OrderBy(x => x.QueueTime);
			return model.Select(x => new JobModel(x));
		}
	}

Step 2

Take the registration part of your ScsHttpHandler and move it into a class that implements ScsRegistration.
your old code:

		public JobViewerHandler(string roles, string isAdmin, string users)
			: base(roles, isAdmin, users)
		{
		}

		public override string Directive { get; set; } = "jvdirective";
		public override NameValueCollection DirectiveAttributes { get; set; }
		public override string ResourcesPath { get; set; } = "ScsJobViewer.Resources";
		public override string Icon => "/scs/jvgearwheels.png";
		public override string Name => "Job Viewer";
		public override string CssStyle => "width:600px";

Should be translated like so
IMPORTANT
There is a new field for Identifier, this is a 2 letter code that is unique to your app, this is used for routing (which we will address later). Additionally you need to define your controller type.


	class ScsJobViewerRegistration : ScsRegistration
	{
		public ScsJobViewerRegistration(string roles, string isAdmin, string users) : base(roles, isAdmin, users)
		{
		}

		public override string Identifier => "jv";
		public override string Directive => "jvmasterdirective";
		public override NameValueCollection DirectiveAttributes { get; set; }
		public override string ResourcesPath => "ScsJobViewer.Resources";
		public override Type Controller => typeof(ScsJobViewerController);
		public override string Icon => "/scs/jv/resources/jvgearwheels.png";
		public override string Name => "Job Viewer";
		public override string CssStyle => "min-width:600px;";
	}

Step 3

Update your relative paths
In order to automate the routing a more specific route is defined which requires adjustment for all your paths defined in the angular factory.

/scs/jvgearwheels.png => /scs/jv/resources/jvgearwheels.png

/scs/jvgetjobs.json => /scs/jv/jvgetjobs.json

Step 4

UPdate your config file

<processor type="ScsJobViewer.JobViewerHandler, ScsJobViewer" >

should now point to the registration type

<processor type="ScsJobViewer.ScsJobViewerRegistration, ScsJobViewer" >

Sharing Header/Footer across platforms

I had a requirement that a site i was building was required to have the headers and footers sourced from a site owned by the parent company on a separate platform.  Sounds a bit insane but doable.

Make certain your site is extremely clean for JS and CSS

The first thing you’re going to want to verify is that you have nothing targeting general elements.  For example all your styling should be done by very specific class targeting.  Something like “my-secret-class” is great whereas “form” not so much.  Even worse would be to style root level elements such as assigning styling to the li element.

In short, don’t use any JS/CSS that could interfere with things coming from their other domain.

Scrape and cache

Next you’ll want to scrape the source site and parse out their header/footer and all CSS/JS using HtmlAgilityPack

		private readonly Dictionary<string, string> _referrerHeaders = new Dictionary<string, string>();
		private readonly Dictionary<string, string> _referrerFooter = new Dictionary<string, string>();
		private readonly object _refreshLocker = new object();

		public virtual string GetHeader()
		{
			lock (_refreshLocker)
			{
				ValidateUrl(GetOriginModel()?.ReturnUrl);
				string ret;
				_referrerHeaders.TryGetValue(GetOriginModel()?.ReturnUrl ?? "", out ret);
				return ret ?? "";
			}
		}
		public virtual string GetFooter()
		{
			lock (_refreshLocker)
			{
				ValidateUrl(GetOriginModel()?.ReturnUrl);
				string ret;
				_referrerFooter.TryGetValue(GetOriginModel()?.ReturnUrl ?? "", out ret);
				return ret ?? "";
			}
		}

		public virtual void ValidateUrl(string url)
		{
			if (string.IsNullOrWhiteSpace(url) || url.StartsWith("/"))
				return;
			if (!_referrerHeaders.ContainsKey(url))
			{
				HtmlDocument doc = new HtmlDocument();
				using (WebClient wc = new WebClient())
				{
					wc.Encoding = Encoding.UTF8;
					doc.LoadHtml(wc.DownloadString(url));
				}
				_referrerHeaders[url] = GenerateHeader(url, doc);
				_referrerFooter[url] = GenerateFooter(doc);
			}
		}
		public virtual string GenerateFooter(HtmlDocument doc)
		{
			return GetNodesByAttribute(doc, "class", "site-footer").FirstOrDefault()?.OuterHtml;
		}

		public virtual string GenerateHeader(string url, HtmlDocument doc)
		{
			Uri uri = new Uri(url);
			string markup =  GetNodesByAttribute(doc, "class", "site-header").FirstOrDefault()?.OuterHtml.Replace("action=\"/", $"action=\"https://{uri.Host}/");
			string svg = GetNodesByAttribute(doc, "class", "svg-legend").FirstOrDefault()?.OuterHtml;
			string stylesheets =
				GetNodesByAttribute(doc, "rel", "stylesheet")
					.Aggregate(new StringBuilder(), (tags, cur) => tags.Append(cur.OuterHtml.Replace("href=\"/bundles", $"href=\"https://{uri.Host}/bundles")))
					.ToString();
			string javascripts =
				doc.DocumentNode.SelectNodes("//script")
					.Aggregate(new StringBuilder(), (tags, cur) =>
					{
						if (cur.OuterHtml.Contains("gtm.js"))
							return tags;
					  return tags.Append(cur.OuterHtml.Replace("src=\"/bundles", $"src=\"https://{uri.Host}/bundles"));

					})
					.ToString();

			return $"{svg}{stylesheets}{markup}{javascripts}";
		}

		public virtual HtmlNodeCollection GetNodesByAttribute(HtmlDocument doc, string attribute, string value)
		{
			return doc.DocumentNode.SelectNodes($"//*[contains(@{attribute},'{value}')]");
		}

NOTE: You’ll likely need to heavily customize your GenerateHeader and GenerateFooter methods.

Lets break this down a bit as it’s a bit hard to follow.

  1. You pass in a URL that you want to source your headers and footers from
  2. Checks the cache to see if we already have that header/footer
  3. Using a WebClient it scrapes the markup off the source page
  4. Using whatever means we can we identify where the markup comes from for the header and footer, in this case it’s identifiable from a class of “site-footer” and “site-header” which makes it easier
  5. We make sure we turn any relative links into absolute links, since relative won’t work anymore since the thing is operating on a separate domain
  6. We grab their SVG sprite definition, we’ll need that or their icons will be blank
  7. Grab all stylesheets and scripts making sure to trip out the things that don’t make sense on a case by case basis like the the other domains tracking libraries
  8. Store this information in the cache

Make sure you periodically clear the caches to pick up changes from the source.  I did this simply like this

		public SiteComponentShareService()
		{
			Timer t = new Timer(600 * 1000);
			t.Elapsed += (sender, args) =>
			{
				lock (_refreshLocker)
				{
					_referrerHeaders.Clear();
					_referrerFooter.Clear();
				}
			};
			t.Start();
		}

 This clears the cache objects every 10 minutes with thread lockers to make sure it doesn’t clear the cache as something is trying to use it.

Finishing Touches

The acquired header and footer may have fancy XHR needs that need to be accounted for. Very likely for this you’ll need to proxy requests. For example i needed to catch search suggestions and pass it through to their servers endpoint for hawksearch

		[Route("hawksearch/proxyautosuggest/{target}")]
		public ActionResult RerouteAutosuggest(string target)
		{
			WebClient wc = new WebClient();
			string ret = wc.DownloadString(
				$"https://www.parentsitewherewefoundtheheaders.org/hawksearch/proxyAutoSuggest/{target}?{Request.QueryString}");
			return Content(ret);

		}

As you can see, we’re simply catching it and passing it along to their domain’s endpoint. Since we have their same javascript code and their same headers this simple pass-through allows us to seem like we have the exact same header.

Become a Sitecore PDF Ninja

I’m going to start this off by saying PDFs are evil and if you can avoid using them, i implore you to avoid at all costs.  It will inevitably lead to lots of frustration.

In our world today PDFs are incredibly prevalent.  Seems like almost every organization has a collection of PDFs for download.  Users and corporations alike seem to have embraced the PDF completely, however that doesn’t change the fact that they are incredibly annoying to programatically and dynamically manage.

In a C# world you have two main choices for managing PDFs the first is ITextSharp.  However i didn’t look into this library much because i noticed it’s pricing model.  In a nutshell it’s free as long as whatever your building is completely open source.  I suspect the vast majority of Sitecore clients are closed source.  Unfortunately it also looks like there are a sizable amount of people who missed this fact and are stealing this library for commercial gain potentially opening themselves up for lawsuit.  Scary stuff, so i looked elsewhere.

I chose to instead focus on PdfSharp which is free for any situation.  They also have a tool called MigraDoc specifically for building PDFs which i found particularly handy.

I have already outlined a solution to make PDFs searchable in the Sitecore search index.  Here i’m going to share a few more tricks I’ve discovered.

Generating PDFs

If you want to generate a PDF out of markup you’re going to be out of luck as a generality as due to the dramatic differences in the medium (HTML being for screens, PDFs being for printing) you’re never going to get perfect.  I believe ITextSharp has a method to do this but PDFSharp does not.  I did see this workaround i thought was interesting and perhaps worth a try.

I chose to use MigraDoc which ended up being quite easy.  There are a few paradigm changes that you need to understand.

  1. There are no pixels in PDFs, measurements are in actual lengths (inches, centimeters, etc.)
  2. Each Page is it’s own entity that can have different widths, headers, footers, margins etc..
  3. There are element similar to most HTML elements such as paragraphs, headers, tables, etc…
  4. Each element has default settings for sizes and spacing that can be overridden on the individual basis.

Here is a sample of setting up default elements and page settings

		private static void PdfDocumentSetup(Document doc)
		{
			//Default text
			Style style = doc.Styles["Normal"];
			style.Font.Name = "Arial Narrow";
			style.Font.Size = Unit.FromPoint(12);
			//Body Text
			style = doc.Styles.AddStyle("Paragraph2", "Normal");
			style.Font.Name = "Arial Narrow";
			style.Font.Size = Unit.FromPoint(12);
			style.ParagraphFormat.SpaceAfter = 6;
			style.ParagraphFormat.PageBreakBefore = false;
			//Title
			style = doc.Styles["Heading1"];
			style.Font.Name = "Arial Narrow";
			style.Font.Size = Unit.FromPoint(45);
			style.Font.Bold = true;
			style.ParagraphFormat.SpaceAfter = 6;
			style.ParagraphFormat.PageBreakBefore = false;
			//SubHeading
			style = doc.Styles["Heading2"];
			style.Font.Name = "Arial Narrow";
			style.Font.Size = Unit.FromPoint(16);
			style.Font.Bold = true;
			style.ParagraphFormat.SpaceAfter = 6;
			style.ParagraphFormat.PageBreakBefore = false;
			//SubHeading
			style = doc.Styles["Heading3"];
			style.Font.Name = "Arial Narrow";
			style.Font.Size = Unit.FromPoint(20);
			style.ParagraphFormat.SpaceAfter = 6;
			style.ParagraphFormat.PageBreakBefore = false;
			//SubHeading
			style = doc.Styles["Heading4"];
			style.Font.Name = "Arial";
			style.Font.Size = Unit.FromPoint(16);
			style.Font.Bold = true;
			style.ParagraphFormat.SpaceAfter = 6;
			style.ParagraphFormat.PageBreakBefore = false;
			//Column Heading
			style = doc.Styles["Heading5"];
			style.Font.Name = "Arial";
			style.Font.Size = Unit.FromPoint(20);
			style.Font.Color = Color.FromRgbColor(255, new Color(0, 128, 192));
			style.Font.Bold = true;
			style.ParagraphFormat.SpaceAfter = 6;
			style.ParagraphFormat.SpaceBefore = 12;
			style.ParagraphFormat.PageBreakBefore = false;
			//Bullets
			style = doc.AddStyle("Bullets", "Normal");
			style.ParagraphFormat.LeftIndent = Unit.FromInch(1.25);
			// Underlined section heading
			style = doc.AddStyle("Heading3Underlined", "Heading3");
			style.ParagraphFormat.Borders.Bottom = new Border() { Width = Unit.FromMillimeter(1), Color = Colors.Black };
			doc.DefaultPageSetup.PageHeight = Unit.FromInch(11);
			doc.DefaultPageSetup.PageWidth = Unit.FromInch(8.5);
			doc.DefaultPageSetup.LeftMargin = Unit.FromInch(.5);
			doc.DefaultPageSetup.RightMargin = Unit.FromInch(.5);
			doc.DefaultPageSetup.FooterDistance = Unit.FromInch(.75);
			doc.DefaultPageSetup.HeaderDistance = Unit.FromInch(.75);
			doc.DefaultPageSetup.TopMargin = Unit.FromInch(1.5);
			doc.DefaultPageSetup.BottomMargin = Unit.FromInch(2);
		}

Aggregate PDFs

You might need to combine two PDFs or take a cover letter PDF and combine it with a generated portion of the PDF.  In my case i had to take a customized cover letter and prepend it to a table output of data.

PdfSharp makes this amazingly easy.  Simply open both PDF sources in PdfSharp.  In migradoc, you can do this by saving the generated PDF to stream then opening the stream in PdfSharp.

			MemoryStream ret = new MemoryStream();
			PdfDocumentRenderer renderer = new PdfDocumentRenderer(true) { Document = doc };
			renderer.RenderDocument();
			renderer.Save(ret, false);

The above code will take the Migradoc document (doc) and render it to a memory stream which can then be opened in PdfSharp

			//_sitecore is a Sitecore Item API abstraction service to allow testability
			var coverletter = PdfReader.Open(_sitecore.GetPdfCoverletterStream(item), PdfDocumentOpenMode.Import);
			var pdf = PdfReader.Open(doc); // this is our stream from above
			for (int i = 0; i < coverletter.PageCount; i++)
			{
				var newPage = coverletter.Pages[i];
				pdf.Pages.Insert(i, newPage);
			}
			MemoryStream ret = new MemoryStream();
			pdf.Save(ret, false);
			return ret;

You simply take each page from one document and insert it into the other then save the result in whatever way you need, stream for us.

Injecting and reading PDFs from Sitecore Media

Getting PDFs from Sitecore is easy. You can use the MediaManager to get the PDF stream like so.

		public Stream GetPdfStream(Item pdf)
		{
			return MediaManager.GetMedia(pdf).GetStream().Stream;
		}

Once you’ve made your modifications you can write your PDF to a Sitecore media item like so:

					using (new SecurityDisabler())
					{
						pdfItem.Editing.BeginEdit();
						pdfItem.Fields["Blob"].SetBlobStream(pdf);//our stream that we were working with
						pdfItem.Fields["Extension"].Value = "pdf";
						pdfItem.Fields["Mime Type"].Value = "application/pdf";
						pdfItem.Editing.EndEdit();
					}

Find and replace tokens inside a PDF

			var coverletter = PdfReader.Open(_sitecore.GetPdfStream(item), PdfDocumentOpenMode.Import);
			for (int i = 0; i < coverletter.PageCount; i++)
			{
				var newPage = coverletter.Pages[i];

				for (int j = 0; j < newPage.Contents.Elements.Count; j++)
				{
					PdfDictionary.PdfStream stream = newPage.Contents.Elements.GetDictionary(j).Stream;
					var inStream = stream.Value;
					StringBuilder stringStream = new StringBuilder();
					foreach (byte b in inStream)
						stringStream.Append((char)b);

					stringStream = stringStream.Replace("__day__", DateTime.Now.Day.ToString()).Replace("__month__", DateTime.Now.ToString("MMMM")).Replace("__year__", DateTime.Now.Year.ToString());

					newPage.Contents.Elements.GetDictionary(j).Stream.Value = Encoding.UTF8.GetBytes(stringStream.ToString());
				}
			}

In this approach we can see that if we convert the stream to a byte array it will contain all the characters used in this PDF. BEFORE YOU GO THINKING THIS WILL ALWAYS WORK (it likely won’t).  There are many things that need to be the case for this to work as seen here.

as far as i can figure, a sure fire way of this working isn’t possible, however if you can work with a standard PDF creation process from your content authors with some tweaks you can get this to work.