Replacing String Tokens in Powershell

Often times in devops we find ourselves wanting to replace tokens related to environmental properties at the time of release. Token replacement is a popular method to do so and in many build platforms there are tools to provide this functionality. However if you find yourself unable to use the built in tools, or find that they aren’t sufficient then rolling your own can be fairly straightforward.

Replace tokens with environment variables.

This will replace any tokens found in the string and replace the tokens found inside with environment variables with the same name. For example if you have and environment variable named foo that had a value of bar all instances of __foo__ would be replaced with bar.

This function operates with a single iteration of the string in question, replacing strings as it goes along. This makes it as optimal as possible while also being flexible enough to handle any token format.

function Find-ReplaceToken {
    [CmdletBinding()]
	param(
        [Parameter(Mandatory = $true)]
        [string]$String, #String that you want to replace tokens in
        [string]$TokenPrefix = "__",
        [string]$TokenSuffix = "__"
    )
    $ret = [System.Text.StringBuilder]::new($String)
    $found = New-Object 'System.Collections.Stack'
    $charArr = $String.ToCharArray() 
    $start = -1
    $stop = -1
    $token = [System.Text.StringBuilder]::new()
    For ($i=0; $i -le $charArr.Length; $i++) {
        if ($start -ne -1){
            $null = $token.Append($charArr[$i])
        }
        if ($charArr[$i] -eq "`n"){
            $start = -1
            $stop = -1
            $null = $token.Clear()
        }
        elseif($start -ne -1 -and $String.Substring($i-$TokenPrefix.Length, $TokenPrefix.Length) -eq $TokenPrefix -and $charArr[$i - $TokenPrefix.Length] -eq $TokenPrefix[$TokenPrefix.Length-1]){
            $start = -1
            $stop = -1
            $null = $token.Clear()
            $i--
        }
        elseif ($start -ne -1 -and $i -lt $String.Length - $TokenPrefix.Length -and $String.Substring($i-$TokenSuffix.Length+1, $TokenSuffix.Length) -eq $TokenSuffix){
            $stop = $i+1
            $found.Push([System.Tuple]::Create($start, $stop, $token.ToString()))
            write-host "TOKEN FOUND - $($token.ToString())"
            $i--
            $start = -1
            $stop = -1
            $null = $token.Clear()
    }
        elseif ($i -ge $TokenPrefix.Length -and $String.Substring($i-$TokenPrefix.Length, $TokenPrefix.Length) -eq $TokenPrefix){
            if ($start -eq -1){
                $start = $i-$TokenPrefix.Length
                $null = $token.Append($TokenPrefix)
                $null = $token.Append($charArr[$i])
            }
        }
    }
    $replacedTokens = $false
    while ($found.Count -gt 0){
        $t = $found.Pop()
        $var = $t.Item3.TrimStart($TokenPrefix).TrimEnd($TokenSuffix)
        if ($null -ne [System.Environment]::GetEnvironmentVariable($var)){
            write-host "REPLACING $($t.Item3) with $([System.Environment]::GetEnvironmentVariable($var))"
            $replacedTokens = $true
            $null = $ret.Remove($t.Item1, $t.Item2 - $t.Item1)
            $null = $ret.Insert($t.Item1, [System.Environment]::GetEnvironmentVariable($var), 1)
        }else{
            write-host "Environment Variable $var not found."
        }
    }
    if ($replacedTokens){
        return $ret.ToString()
    }
    return [string]::Empty
}

Replacing file content

Passing in the root path of the files with tokens and an extensions filter starts the operation and recurses it over all files searching for tokens

param(
    [string]$path,
    [string[]]$extensions
)

# Reference to the string replace function above

gci $path -Include $extensions -Recurse -File | foreach{
    $contents = Find-ReplaceToken -String "$(Get-Content $_.FullName -Raw)"
    if (-not [string]::IsNullOrWhiteSpace($contents)){
        write-host "Replacing tokens in file: $($_.FullName)"
        Set-Content -Path $_.FullName -Value $contents
    }
}

Don’t want to use Environment variables?

By making a few simple changes you can make the function operate using a standard hashtable

function Find-ReplaceToken {
    [CmdletBinding()]
	param(
        [Parameter(Mandatory = $true)]
        [string]$String, #String that you want to replace tokens in
        [string]$TokenPrefix = "__", 
        [string]$TokenSuffix = "__",
        [hashtable]$Tokens # Add this parameter to pass in a hashtable instead of environment variables
    )
# ...
    while ($found.Count -gt 0){
        $t = $found.Pop()
        $var = $t.Item3.TrimStart($TokenPrefix).TrimEnd($TokenSuffix)
        if ($null -ne $Tokens[$var]){ # Replace the environment variable with the hashtable here
            write-host "REPLACING $($t.Item3) with $([System.Environment]::GetEnvironmentVariable($var))"
            $replacedTokens = $true
            $null = $ret.Remove($t.Item1, $t.Item2 - $t.Item1)
            $null = $ret.Insert($t.Item1, $Tokens[$var], 1) # Replace the environment variable with the hashtable here
        }else{
            write-host "Environment Variable $var not found."
        }
    }