Chris Warwick – The Blog Edition – Well it's probably mostly PowerShell isn't it…

Scripts on GitHub and PowerShell Gallery

Updated versions of scripts on this blog are now being posted on GitHub and on the PowerShell Gallery. You can find these here:

GitHub: https://github.com/ChrisWarwick

PS Gallery: https://www.powershellgallery.com/

PowerShell Gallery scripts can be found using the Find-Module cmdlet (this cmdlet is available in PowerShell V5 (on Windows 10) or Version 5 of the Windows Management Framework).

To list all scripts I’ve published, use:

Find-Module | Where Author -like ‘Chris*Warwick’

Follow me on Twitter for updates: @cjwarwickps

Getting SNTP Network Time with PowerShell (Improved!)

This was originally posted here, with further notes.

This is an improved version that includes the network delay as per the NTP rfc and uses the resulting offset to calculate a more accurate time.

The full code is here: https://skydrive.live.com/redir?resid=7CB58BE453F7E567!3445

Pipe to “Format-List *” to see all returned information. Usage:

PS:\> Get-NtpTime uk.pool.ntp.org

NtpTime          : 16/09/2012 14:26:46
OffsetSeconds    : -0.19
NtpVersionNumber : 3
Mode_text        : server
Stratum          : 3
PollInterval     : 00:00:01
PrecisionSeconds : 1.9073486328125E-06

PS:\> Get-NtpTime uk.pool.ntp.org | fl *

NtpTime          : 16/09/2012 14:26:51
Offset           : -190.25927734375
OffsetSeconds    : -0.19
Delay            : 18.0205078125
t1ms             : 3556790811477.35
t2ms             : 3556790811296.1
t3ms             : 3556790811296.1
t4ms             : 3556790811495.37
t1               : 16/09/2012 14:26:51
t2               : 16/09/2012 14:26:51
t3               : 16/09/2012 14:26:51
t4               : 16/09/2012 14:26:51
LI               : 0
LI_text          : no warning
NtpVersionNumber : 3
Mode             : 4
Mode_text        : server
Stratum          : 3
Stratum_text     : secondary reference (via NTP or SNTP)
PollIntervalRaw : 0
PollInterval     : 00:00:01
Precision        : -19
PrecisionSeconds : 1.9073486328125E-06
Raw              : {28, 3, 0, 237, 0, 0, 2, 183, 0, 0, 0, 0, 80, 93, 163, 202, 212, 0, 80, 97…}

If you don’t like comments in your code (and just want the resulting timestamp), here’s the bare bones version:

<#
Chris Warwick, @cjwarwickps, August 2012
chrisjwarwick.wordpress.com 
#>

$Server = 'pool.ntp.org'
$StartOfEpoch=New-Object DateTime(1900,1,1,0,0,0,[DateTimeKind]::Utc)   

[Byte[]]$NtpData = ,0 * 48
$NtpData[0] = 0x1B    # NTP Request header in first byte

$Socket = New-Object Net.Sockets.Socket([Net.Sockets.AddressFamily]::InterNetwork,
                                        [Net.Sockets.SocketType]::Dgram,
                                        [Net.Sockets.ProtocolType]::Udp)
$Socket.Connect($Server,123)
$t1 = Get-Date    # Start of transaction... the clock is ticking...
[Void]$Socket.Send($NtpData)
[Void]$Socket.Receive($NtpData)  
$t4 = Get-Date    # End of transaction time
$Socket.Close()

$IntPart = [BitConverter]::ToUInt32($NtpData[43..40],0)   # t3
$FracPart = [BitConverter]::ToUInt32($NtpData[47..44],0)
$t3ms = $IntPart * 1000 + ($FracPart * 1000 / 0x100000000)

$IntPart = [BitConverter]::ToUInt32($NtpData[35..32],0)   # t2
$FracPart = [BitConverter]::ToUInt32($NtpData[39..36],0)
$t2ms = $IntPart * 1000 + ($FracPart * 1000 / 0x100000000)

$t1ms = ([TimeZoneInfo]::ConvertTimeToUtc($t1) - $StartOfEpoch).TotalMilliseconds
$t4ms = ([TimeZoneInfo]::ConvertTimeToUtc($t4) - $StartOfEpoch).TotalMilliseconds
 
$Offset = (($t2ms - $t1ms) + ($t3ms-$t4ms))/2

$StartOfEpoch.AddMilliseconds($t4ms + $Offset).ToLocalTime()

More Regular Expressions: Regex for IP v4 Addresses

I’ve seen some posts suggesting this as a useful IPv4 Regex:

\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}

This matches four groups of between 1-3 digits, with each group separated by a dot.

An obvious enhancement here would be to remove some redundancy:

(\d{1,3}\.){3}\.\d{1,3}

\d{1,3}(\.\d{1,3}){3}

These are both shorter and say the same thing. However, there’s unfortunately an issue with correctness. These Regexs match IPv4 addresses, but they also match strings that aren’t valid IPv4 addresses. The following will match, for example:

999.999.999.999

What to do? Well, as mentioned previously, start with Jeffrey Friedl’s book “Mastering Regular Expressions”, 3rd Edition, O’Reilly, ISBN: 978-0-596-52812-6 from Amazon or directly from O’Reilly. Read the customer reviews on these sites to see what I mean.

Otherwise, there are several good Regex generation/authoring programs out there. I have used Jan Goyvaerts’ RegexBuddy for a long time and unconditionally recommend it. It’s not free but I would say it’s a very worthwhile investment of 30 Euros if you intend to use regex to any degree. Even without the program itself just the help files are great. You can get your money back up to 3 months after purchase if you’re not happy, so try it!

If these two options don’t appeal to you then it’s off to the web. Sadly, there’s lots and lots of dross and rubbish out there, some of it wildly misleading.

Valid Formats

To cut a long story short, match IPv4 address with this:

\b((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3} (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b

Or this:

\b((25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b

Or this:

\b(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}\b

These all match valid IPv4 addresses without inadvertently matching other things.

Breakdown

There are two things that can help understanding of complex regexs: using intermediate patterns and using extended mode.

PowerShell can save intermediate patterns (or parts of a regex) in variables that can later be uses in a more complex string, using standard variable expansion.

Using regex ‘Extended Mode’ allows a regex pattern to be split across lines (and to use white space and comments) and is indicated by using ‘(?x)’ at the beginning of the regex pattern. This can be combined with a here-string to allow the regex to be more easily visualised…

Using these facilities we can break down those IPv4 regexs to something a little more comprehensible (note, I’ve used the ${xxxx} format for variable names here to save spelling out $TwoHundredToTwoFiveFive).

An IPv4 address is made up of 4 Octets – each with a value between 0 and 255. To specify this in a regex we break the range down into 3 groups (divide and conquer), specifically 0-199, 200-249 and 250-255. Remember that ‘[0-4]’ matches a single character (one of 0,1,2,3,4). We then say that an Octet is either one of these by using an ( x | y | z ) alternation. We build up the final address by inserting dots between 4 of these Octets.

Here’s the result. Tidy and (hopefully!) comprehensible :-)

${250-255}=’25[0-5]’   # Matches 3 digit numbers between 250 and 255
${200-249}=‘2[0-4]\d’  # Matches 3 digit numbers between 200 and 249
${0-199}=‘[01]?\d\d?’  # Matches 1, 2 or 3 digit numbers between 0 and 199
$Octet=“( ${250-255} | ${200-249} | ${0-199} )”
$IPv4=@”
(?x) ^
$Octet (\.$Octet){3}
$
“@

IPv4 Function

Here’s a quick PowerShell function that will return the first valid IPv4 address from a given string (or $Null if the string doesn’t contain a valid address):

Extract IP Address from String

# Extract an IP address from a string and return it.  Return $Null if no valid address found.
# (If the string contains more than one valid address only the first one will be returned)
Function ExtractValidIPAddress($String){
    $IPregex=‘(?<Address>((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))’
    If ($String -Match $IPregex) {$Matches.Address}
} 

Thus:

C:\PS> ExtractValidIPAddress ‘This is a string with an IPv4 address 192.168.0.1 in it’
192.168.0.1

Finding Prime Factors

A bit off the wall… Here’s a script that started life in 1980 in a magazine article and which I’ve revisited over the years, converting to new languages as they’ve come along. Here’s the latest PowerShell version, updated since I first created it in 2006.

Sample:

PS:\> get-factors 21
21 = 3 x 7

PS:\> get-factors 21 -Verbose
VERBOSE: Finding factors of 21 (with root ~4.58)
VERBOSE: Trying 2
VERBOSE: 2 is not a factor
VERBOSE: Adding 1; divisor is now 3
VERBOSE: Trying 3
VERBOSE: Found factor 3
VERBOSE: Remainder is now 7 (root ~2.65), finding factors of this…
VERBOSE: Trying 3
VERBOSE: 3 is not a factor
VERBOSE: Adding 2; divisor is now 5
VERBOSE: Trying 5
VERBOSE: 5 is not a factor
VERBOSE: Adding 2; divisor is now 7
VERBOSE: Trying 7
VERBOSE: Found factor 7
VERBOSE: Remainder is 1, all factors found
21 = 3 x 7

See comments in the code for further info…

# Calculates Prime Factors of Given Integer
#
# Algorithm based on an article from "Practical Computing", February 1980 (when computer magazines were really about computing:-)
#
# This is a standard sieve, but rather than trying all increasing integers as divisors the next possible divisor is
# calculated by adding an increment from a (repeating) sequence - thereby missing any even numbers or any multiples of 3 or 5.
#
# The initial divisor is 2.  The increment list repeats from the forth entry, so the possible divisors will start with:
#
# Divisor:   2, 3, 5, 7,   11, 13, 17, 19, 23, 29, 31, 37,           41, 43, 47, 49, 53, 59, 61, 67,           71, 73, 77, 79, 83, 89, 91, 97
# Increment:  +1 +2 +2   +4  +2  +4  +2  +4  +6  +2  +6   [repeat] +4  +2  +4  +2  +4  +6  +2  +6   [repeat] +4  +2  +4  +2  +4  +6  +2  +6 
#
# This approach skips a large number of trial division operations at the expense of some array (list) management.
#
# PowerShell version, Chris Warwick, 2006
# Last modified September 2012
# .

[CmdletBinding()]
Param ($n=$(Throw "Specify a number to factorise"))

$DivisorIncrements=1,2,2,4,2,4,2,4,6,2,6
$IncrementLength=$DivisorIncrements.Length

$Root=[Math]::Pow($n,0.5)    # All factors found once divisor is greater than number's square-root

$Divisor=2        # Initial divisor - try this as first factor
$Number=$n        # Save original number
$Answer=""        # Resultant list of factors appended to this string
$i=0            # Index into increment list

Write-Verbose ("Finding factors of $n (with root ~{0:0.##})" -f $Root)

For (;;) {
   Write-Verbose "Trying $Divisor"
   $Remainder=$n/$Divisor
   If ($Remainder -eq [Math]::Truncate($Remainder)) {
      # Remainder is a whole number, found a factor...
      Write-Verbose "Found factor $Divisor"
      If ($Remainder -eq 1) {
         # All factors have been found
         Write-Verbose "Remainder is 1, all factors found"
         Break
      }
      $n=$Remainder        # ...remove this factor and calculate factors of remainder
      $Root=[Math]::Pow($n,0.5)        # ...new end-point
      Write-Verbose ("Remainder is now $Remainder (root ~{0:0.##}), finding factors of this..." -f $Root)
      $Answer+=" $Divisor x"    # ... save list of factors for display
   }
   Else {
      # Current divisor was not a factor
      Write-Verbose "$Divisor is not a factor"
      $Divisor+=$DivisorIncrements[$i]        # Add to the divisor, skipping multiples of 2, 3, 4, 5
      Write-Verbose "Adding $($DivisorIncrements[$i]); divisor is now $Divisor"
      $i++        # Next time add the next increment from the list
      If ($i -ge $IncrementLength) {
         # Got to end of increments list...
         $i=3        # List repeats from the 4th element
         If ($Divisor -gt $Root) {
            # Check at this point that the divisor isn't too large
            Write-Verbose "Divisor now greater than Sqrt of remainder... done"
            Break
         }
      }
   }
}

If ($n -eq $Number) {"$n is Prime"}
Else {
   "$Number =$Answer $n"   # Append last remainder and display list of factors
}

Getting NTP/SNTP Network Time with PowerShell

(Update: There’s now an improved version of this script here that calculates network offsets as per the NTP rfc)

I wrote this function because I needed to check the NTP responses from a Windows Active Directory Domain Controller. I found some NTP procedures on the web but actually most of them were using the Daytime service on Port 13 (not NTP).

Here’s a very quick version of the active part of the script:

$Server = ‘uk.pool.ntp.org’

# Construct client NTP time packet to send to specified server
# (Request Header: [00=No Leap Warning; 011=Version 3; 011=Client Mode]; 00011011 = 0x1B)

[Byte[]]$NtpData = ,0 * 48
$NtpData[0] = 0x1B

$Socket = New-Object Net.Sockets.Socket([Net.Sockets.AddressFamily]::InterNetwork,
[Net.Sockets.SocketType]::Dgram,
[Net.Sockets.ProtocolType]::Udp)

$Socket.Connect($Server,123)
[Void]$Socket.Send($NtpData)
[Void]$Socket.Receive($NtpData) # Returns length – should be 48…

$Socket.Close()

# Decode the received NTP time packet

# We now have the 64-bit NTP time in the last 8 bytes of the received data.
# The NTP time is the number of seconds since 1/1/1900 and is split into an
# integer part (top 32 bits) and a fractional part, multipled by 2^32, in the
# bottom 32 bits.

# Convert Integer and Fractional parts of 64-bit NTP time from byte array
$IntPart=0; Foreach ($Byte in $NtpData[40..43]) {$IntPart = $IntPart * 256 + $Byte}
$FracPart=0; Foreach ($Byte in $NtpData[44..47]) {$FracPart = $FracPart * 256 + $Byte}

# Convert to Millseconds (convert fractional part by dividing value by 2^32)
[UInt64]$Milliseconds = $IntPart * 1000 + ($FracPart * 1000 / 0x100000000)

# Create UTC date of 1 Jan 1900, add the NTP offset and convert result to local time
(New-Object DateTime(1900,1,1,0,0,0,[DateTimeKind]::Utc)).AddMilliseconds($Milliseconds).ToLocalTime()

This just returns a [DateTime] object derived from the NTP Server. There’s more information available though; the following full function decodes the NTP Time data and returns it in a custom PowerShell object. Note that the returned object does not display all of its attributes by default – pipe to “Format-List *” to see all the values.

Usage: Get-NtpTime <Server>

Sample output:

PS:\> Get-NtpTime
NtpTime          : 26/08/2012 19:54:52
NtpVersionNumber : 3
Mode_text        : server
Stratum          : 2
PollInterval     : 00:00:08
PrecisionSeconds : 4.76837158203125E-07
LocalDifference : -00:00:00.2891348

Full function:

<#
Chris Warwick, @cjwarwickps, August 2012
chrisjwarwick.wordpress.com

Get Datetime from NTP server.

This sends an NTP time packet to the specified NTP server and reads back the response.
The NTP time packet from the server is decoded and returned.

Note: this uses NTP (rfc-1305: http://www.faqs.org/rfcs/rfc1305.html) on UDP 123. Because the
function makes a single call to a single server this is strictly a SNTP client (rfc-2030).
Although the SNTP protocol data is similar (and can be identical) and the clients and servers
are often unable to distinguish the difference. Where SNTP differs is that is does not
accumulate historical data (to enable statistical averaging) and does not retain a session
between client and server.

An alternative to NTP or SNTP is to use Daytime (rfc-867) on TCP port 13 – although this is an
old protocol and is not supported by all NTP servers.

See comments at the end of the script for an extract of the SNTP rfc.

#Requires -Version 3

Function Get-NtpTime {

<#
.Synopsis
Gets (Simple) Network Time Protocol time (SNTP/NTP, rfc-1305, rfc-2030) from a specified server
.DESCRIPTION
This function connects to an NTP server on UDP port 123 and retrieves the current NTP time.
Selected components of the returned time information are decoded and returned in a PSObject.
.PARAMETER Server
The NTP Server to contact. Uses pool.ntp.org by default.
.EXAMPLE
Get-NtpTime uk.pool.ntp.org
Gets time from the specified server.
.EXAMPLE
Get-NtpTime | fl *
Get time from default server (pool.ntp.org) and displays all output object attributes.
.OUTPUTS
A PSObject containing decoded values from the NTP server. Pipe to fl * to see all attributes.
.FUNCTIONALITY
Gets NTP time from a specified server.
#>

[CmdletBinding()]
Param (
[String]$Server = ‘pool.ntp.org’
)
# Construct a 48-byte client NTP time packet to send to the specified server
# (Request Header: [00=No Leap Warning; 011=Version 3; 011=Client Mode]; 00011011 = 0x1B)

    [Byte[]]$NtpData = ,0 * 48
$NtpData[0] = 0x1B
    $Socket = New-Object Net.Sockets.Socket([Net.Sockets.AddressFamily]::InterNetwork,
[Net.Sockets.SocketType]::Dgram,
[Net.Sockets.ProtocolType]::Udp)

$LocalTime = Get-Date

Try {
$Socket.Connect($Server,123)
[Void]$Socket.Send($NtpData)
[Void]$Socket.Receive($NtpData)
}
Catch {
Write-Error “Failed to communicate with server $Server”
Throw $_
}

$Socket.Close()

# Decode the received NTP time packet

# Extract the flags from the first byte by masking and shifting (dividing)

    $LI = ($NtpData[0] -band 0xC0)/64    # Leap Second indicator
$LI_text = Switch ($LI) {
0    {‘no warning’}
1    {‘last minute has 61 seconds’}
2    {‘last minute has 59 seconds’}
3    {‘alarm condition (clock not synchronized)’}
}
$VN = ($NtpData[0] -band 0x38)/8     # Server versions number
$Mode = ($NtpData[0] -band 0x07)     # Server mode (probably ‘server’!)
$Mode_text = Switch ($Mode) {
0    {‘reserved’}
1    {‘symmetric active’}
2    {‘symmetric passive’}
3    {‘client’}
4    {‘server’}
5    {‘broadcast’}
6    {‘reserved for NTP control message’}
7    {‘reserved for private use’}
}

    $Stratum = [UInt16]$NtpData[1]   # Actually [UInt8] but we don’t have one of those…
$Stratum_text = Switch ($Stratum) {
0                            {‘unspecified or unavailable’}
1                            {‘primary reference (e.g., radio clock)’}
{$_ -ge 2 -and $_ -le 15}    {‘secondary reference (via NTP or SNTP)’}
{$_ -ge 16}                  {‘reserved’}
}

$PollInterval = $NtpData[2] # Poll interval – to neareast power of 2
$PollIntervalSeconds = [Math]::Pow(2, $PollInterval)

    $PrecisionBits = $NtpData[3]      # Precision in seconds to nearest power of 2
# …this is a signed 8-bit int
If ($PrecisionBits -band 0x80) {    # ? negative (top bit set)
[Int]$Precision = $PrecisionBits -bor 0xFFFFFFE0    # Sign extend
} else {
# ..this is unlikely – indicates a precision of less than 1 second
[Int]$Precision = $PrecisionBits   # top bit clear – just use positive value
}
$PrecisionSeconds = [Math]::Pow(2, $Precision)

# Convert to Millseconds (convert fractional part by dividing value by 2^32)
[UInt64]$Milliseconds = $IntPart * 1000 + ($FracPart * 1000 / 0x100000000)

# Create UTC date of 1 Jan 1900, add the NTP offset and convert the result from utc to local time
$NtpTime=(New-Object DateTime(1900,1,1,0,0,0,[DateTimeKind]::Utc)).AddMilliseconds($Milliseconds).ToLocalTime()
# Create Output object and return

$NtpTimeObj = [PSCustomObject]@{
NtpTime = $NtpTime
LI = $LI
LI_text = $LI_text
NtpVersionNumber = $VN
Mode = $Mode
Mode_text = $Mode_text
Stratum = $Stratum
Stratum_text = $Stratum_text
PollIntervalRaw = $PollInterval
PollInterval = New-Object TimeSpan(0,0,$PollIntervalSeconds)
Precision = $Precision
PrecisionSeconds = $PrecisionSeconds
LocalDifference = [TimeSpan]($NtpTime – $LocalTime)
Raw = $NtpData # The undecoded bytes returned from the NTP server
}

# Set default display properties for object

[String[]]$DefaultProperties = ‘NtpTime’, ‘NtpVersionNumber’,’Mode_text’,’Stratum’, ‘PollInterval’,
‘PrecisionSeconds’, ‘LocalDifference’

# Add the PSStandardMembers.DefaultDisplayPropertySet member
$ddps = New-Object Management.Automation.PSPropertySet(‘DefaultDisplayPropertySet’, $DefaultProperties)
$PSStandardMembers = [Management.Automation.PSMemberInfo[]]$ddps

# Attach default display property set and output object
$NtpTimeObj | Add-Member -MemberType MemberSet -Name PSStandardMembers -Value $PSStandardMembers -PassThru
}

From rfc-2030
~~~~~~~~~~~~~

NTP time packet format

                           1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|LI | VN |Mode |    Stratum    |     Poll      |   Precision   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          Root Delay                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       Root Dispersion                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Reference Identifier                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                   Reference Timestamp (64)                    |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                   Originate Timestamp (64)                    |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                    Receive Timestamp (64)                     |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                    Transmit Timestamp (64)                    |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   Leap Indicator (LI): This is a two-bit code warning of an impending
leap second to be inserted/deleted in the last minute of the current
day, with bit 0 and bit 1, respectively, coded as follows:

      LI       Value     Meaning
——————————————————-
00       0         no warning
01       1         last minute has 61 seconds
10       2         last minute has 59 seconds)
11       3         alarm condition (clock not synchronized)

Version Number (VN): This is a three-bit integer indicating the
NTP/SNTP version number. The version number is 3 for Version 3 (IPv4
only) and 4 for Version 4 (IPv4, IPv6 and OSI). If necessary to
distinguish between IPv4, IPv6 and OSI, the encapsulating context
must be inspected.

Mode: This is a three-bit integer indicating the mode, with values
defined as follows:

      Mode     Meaning
————————————
0        reserved
1        symmetric active
2        symmetric passive
3        client
4        server
5        broadcast
6        reserved for NTP control message
7        reserved for private use

In unicast and anycast modes, the client sets this field to 3
(client) in the request and the server sets it to 4 (server) in the
reply. In multicast mode, the server sets this field to 5
(broadcast).

Stratum: This is a eight-bit unsigned integer indicating the stratum
level of the local clock, with values defined as follows:

      Stratum Meaning
———————————————-
0        unspecified or unavailable
1        primary reference (e.g., radio clock)
2-15     secondary reference (via NTP or SNTP)
16-255   reserved

Poll Interval: This is an eight-bit signed integer indicating the
maximum interval between successive messages, in seconds to the
nearest power of two. The values that can appear in this field
presently range from 4 (16 s) to 14 (16284 s); however, most
applications use only the sub-range 6 (64 s) to 10 (1024 s).

Precision: This is an eight-bit signed integer indicating the
precision of the local clock, in seconds to the nearest power of two.
The values that normally appear in this field range from -6 for
mains-frequency clocks to -20 for microsecond clocks found in some
workstations.

Root Delay: This is a 32-bit signed fixed-point number indicating the
total roundtrip delay to the primary reference source, in seconds
with fraction point between bits 15 and 16. Note that this variable
can take on both positive and negative values, depending on the
relative time and frequency offsets. The values that normally appear
in this field range from negative values of a few milliseconds to
positive values of several hundred milliseconds.

Root Dispersion: This is a 32-bit unsigned fixed-point number
indicating the nominal error relative to the primary reference
source, in seconds with fraction point between bits 15 and 16. The
values that normally appear in this field range from 0 to several
hundred milliseconds.

Reference Identifier: This is a 32-bit bitstring identifying the
particular reference source. In the case of NTP Version 3 or Version
4 stratum-0 (unspecified) or stratum-1 (primary) servers, this is a
four-character ASCII string, left justified and zero padded to 32
bits. In NTP Version 3 secondary servers, this is the 32-bit IPv4
address of the reference source. In NTP Version 4 secondary servers,
this is the low order 32 bits of the latest transmit timestamp of the
reference source. NTP primary (stratum 1) servers should set this
field to a code identifying the external reference source according
to the following list. If the external reference is one of those
listed, the associated code should be used. Codes for sources not
listed can be contrived as appropriate.

      Code     External Reference Source
—————————————————————-
LOCL     uncalibrated local clock used as a primary reference for
a subnet without external means of synchronization
PPS      atomic clock or other pulse-per-second source
individually calibrated to national standards
ACTS     NIST dialup modem service
USNO     USNO modem service
PTB      PTB (Germany) modem service
TDF      Allouis (France) Radio 164 kHz
DCF      Mainflingen (Germany) Radio 77.5 kHz
MSF      Rugby (UK) Radio 60 kHz
WWV      Ft. Collins (US) Radio 2.5, 5, 10, 15, 20 MHz
WWVB     Boulder (US) Radio 60 kHz
WWVH     Kaui Hawaii (US) Radio 2.5, 5, 10, 15 MHz
CHU      Ottawa (Canada) Radio 3330, 7335, 14670 kHz
LORC     LORAN-C radionavigation system
OMEG     OMEGA radionavigation system
GPS      Global Positioning Service
GOES     Geostationary Orbit Environment Satellite

Reference Timestamp: This is the time at which the local clock was
last set or corrected, in 64-bit timestamp format.

Originate Timestamp: This is the time at which the request departed
the client for the server, in 64-bit timestamp format.

Receive Timestamp: This is the time at which the request arrived at
the server, in 64-bit timestamp format.

Transmit Timestamp: This is the time at which the reply departed the
server for the client, in 64-bit timestamp format.

Wake On Lan Magic Packet script

There are a bunch of these to be found on the Web (see the additional notes on Origins below) although sadly some of them are either incorrect or of very dubious style. Here’s my take on the theme in PowerShell (naturally:-)

I use this a lot – especially these days with spiralling electricity costs – to wake suspended machines and servers.

Wake on Lan uses a “Magic Packet” that consists of six bytes of 0xFF (the physical layer broadcast address), followed by 16 copies of the 6-byte (48-bit) MAC address (see http://en.wikipedia.org/wiki/Wake-on-LAN). This packet is sent via UDP to the LAN Broadcast address (255.255.255.255) on arbitrary Port 4000 – although the layer 3 baggage is actually largely irrelevant. Construction of this packet in PowerShell is a breeze thanks to the array semantics in the language (“$Packet = [Byte[]](,0xFF*6)+($Mac*16)” – neat).

This script has a table of saved MAC addresses to allow aliases to be specified on the command line (the real addresses have been obfuscated here) and uses a regex to validate the resulting MAC address string. It would be possible to use DNS and the ARP Cache to resolve MAC addresses but the ARP cache will only be populated with a valid entry for any given target adapter for a relative short period of time after the last use of the address (10 minutes or less depending on usage); ARP cannot be used to dynamically resolve the address of a suspended adapter.

Here’s the script:

WakeOnLan.ps1

# Send Wake-on-Lan Magic Packet to specified Mac address
[CmdletBinding()]
Param ($MacString=$(Throw 'Mac address is required'))
 
$Table=@{
    Hyperion  ='00-00-00-00-00-44';
    Nova      ='00-00-00-00-00-C8';
    Desktop   ='00-00-00-00-00-1B';
    Laptop    ='00-00-00-00-00-18';
    Playroom  ='00-00-00-00-00-5C';
    Betty     ='00-00-00-00-00-32';
    gr8       ='00-00-00-00-00-D7'
}
If ($Table.ContainsKey($MacString)) {$MacString=$Table[$MacString]}
 
Write-Verbose "Using MAC string $MacString"
 
If ($MacString -NotMatch '^([0-9A-F]{2}[:-]){5}([0-9A-F]{2})$') {
    Throw 'Mac address must be 6 hex bytes separated by : or -'
}
 
# Split and convert to array of bytes
$Mac=$MacString.Split('-:')|%{[Byte]"0x$_"}
 
# Packet is byte array; first six bytes are 0xFF, followed by 16 copies of the MAC address
$Packet = [Byte[]](,0xFF*6)+($Mac*16)
Write-Verbose "Broadcast packet: $([BitConverter]::ToString($Packet))"
 
$UdpClient=New-Object System.Net.Sockets.UdpClient
$UdpClient.Connect(([System.Net.IPAddress]::Broadcast),4000)
[Void]$UdpClient.Send($Packet,$Packet.Length)
$UdpClient.Close()
Write-Verbose "Wake-on-Lan Packet sent to $MacString"

Origins

I’ve been using this script for a long time and while writing this post I wondered where the idea had come from. As is often the case it would appear that, at least for the correct entries (including those from the like of Brandon http://social.technet.microsoft.com/Forums/en-US/ITCG/thread/51ef86e5-f3d4-459c-a08a-615c669aff2e and John Savill http://www.windowsitpro.com/article/networking/Q-How-can-I-easily-send-a-magic-packet-to-wake-a-machine-on-my-subnet-), MOW is involved(http://thepowershellguy.com/blogs/posh/archive/2007/04/01/powershell-wake-on-lan-script.aspx) – although in this particular case, his blog entry is not correct but the answer is provided in the blog comments by James Manning who used to have an MSDN blog and I imagine (used to?) work for Microsoft – although the blog hasn’t been updated since 2009.

Modify Date Taken values on Photos with PowerShell, the Update-ExifDateTaken script cmdlet

(Part 3 of 3) Download Script Here

The last two posts (Part 1, Part 2) have described the rationale behind creating these Exif cmdlets and looked at how to read Exif information from image files. This final part looks at the Update-ExifDateTaken script cmdlet which will modify Exif meta-data contained within photo (.jpg) files.

See the examples in Part 1 for some ways you can use these script cmdlets.

Scott-Hanselman-Style DISCLAIMER: You Do Backup Your Photos Don’t You? I am not a .Net programmer. I have not be paid to program anything but PowerShell since around 1987 (seriously, I’m old) and .Net didn’t exist then. The last non-PowerShell program I was actually paid to write was written in IBM System/370 Assembler. No one taught me about .Net programming, I only know about it through PowerShell and MSDN and Bruce Payette. I’ve used this script without any problems on literally thousands of photos. I’m not saying don’t use it – just use it on a COPY of your precious images.

So, first up, because the script modifies files it must support the “-WhatIf” parameter. There are two new parameters Update-ExifDateTaken version of the script: “-Offset”, which allows us to change the Exif Date Taken value by a specified amount; and “-PassThru”, which will pass the PathInfo objects along the pipeline.

(Note that Get-ExifDateTaken always passes on the PathInfo objects with the additional Exif DateTaken [datetime] attribute; there would be no point in running the script if the output was not passed on because it otherwise has no side-effects. In this Update version the output is optional because just updating the Exif data might be all that is needed. If you want the output in the pipeline, specify –PassThru and you’ll get PathInfo objects decorated with additional ExifDateTaken and ExifOriginalDateTaken attributes).

The Process{} block structure is similar to the Get-ExifDateTaken script, but now there’s an extra step; once the script has the DateTaken value it updates it by adding the specified offset and replaces the value in the image file.

Pertinent points here are that to avoid file locking problems the script creates a MemoryStream object and saves the modified image data to the MemoryStream; it can then close the original input file and overwrite it (‘in-place’) by re-opening the file with a filemode of ‘Create’. Note that this part of the code is protected by the ‘If ($PSCmdlet.ShouldProcess(…))’ test that supports the –WhatIf and –Confirm common parameters.

If the –PassThru switch parameter is the script outputs the current file’s PathInfo object along with two added attributes: the ExifDateTaken [datetime] object which reflects the modified DateTaken value; and the ExifOriginalDateTaken [datetime] object, in case that might be needed later in the pipeline.

That wraps it up. The full download is available on SkyDrive here. Finally, here’s the full script listing, including both the Get- and Update- variants of the script cmdlet:

ExifDateTime.ps1
<#Chris Warwick, @cjwarwickps, August 2013
chrisjwarwick.wordpress.comRevision: Now support PowerShell version 2.0 and above.This version published on SkyDrive here:
https://skydrive.live.com/redir?resid=7CB58BE453F7E567!1289&authkey=!ANY88H3ABytkigkThe script file contains two functions:
Get-ExifDateTaken -Path [filepaths]
Takes a file (fileinfo or string) or an array of these

Gets the ExifDT value (EXIF Property 36867)
Update-ExifDateTaken -Path [filepaths] -Offset [TimeSpan]
Takes a file (fileinfo or string) or an array of these

Modifies the ExifDT value (EXIF Property 36867) as specified
# Further samples:
# Just Update
gci *.jpg|Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru|ft Path, ExifDateTaken
# Update & Rename
gci *.jpg|

 Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru|

 Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg” -f $_.ExifDateTaken, (Split-Path (Split-Path $_) -Leaf)}
# Just Rename
gci *.jpg|

 Get-ExifDateTaken |

 Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg” -f $_.ExifDateTaken, (Split-Path (Split-Path $_) -Leaf)}
#>
#Requires -Version 2.0
Function Get-ExifDateTaken {

<#

.Synopsis

Gets the DateTaken EXIF property in an image file.

.DESCRIPTION

This script cmdlet reads the EXIF DateTaken property in an image and passes is down the pipeline

attached to the PathInfo item of the image file.

.PARAMETER Path

The image file or files to process.

.EXAMPLE

Get-ExifDateTaken img3.jpg

(Reads the img3.jpg file and returns the im3.jpg PathInfo item with the EXIF DateTaken attached)

.EXAMPLE

Get-ExifDateTaken *3.jpg |ft path, exifdatetaken

(Output the EXIF DateTaken values for all matching files in the current folder)

.EXAMPLE

gci *.jpeg,*.jpg|Get-ExifDateTaken 

(Read multiple files from the pipeline)

.EXAMPLE

gci *.jpg|Get-ExifDateTaken|Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss}.jpg” -f $_.ExifDateTaken}

(Gets the EXIF DateTaken on multiple files and renames the files based on the time)

.OUTPUTS

The scripcmdlet outputs PathInfo objects with an additional ExifDateTaken

property that can be used for later processing.

.FUNCTIONALITY

Gets the EXIF DateTaken image property on a specified image file.

#>
[CmdletBinding()]
Param (

[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]

[Alias(‘FullName’, ‘FileName’)]

$Path

)
Begin

{

Set-StrictMode -Version Latest

If ($PSVersionTable.PSVersion.Major -lt 3) {

Add-Type -AssemblyName “System.Drawing”

}

}
Process

{

# Cater for arrays of filenames and wild-cards by using Resolve-Path

Write-Verbose “Processing input item ‘$Path‘”
$PathItems=Resolve-Path $Path -ErrorAction SilentlyContinue -ErrorVariable ResolveError

If ($ResolveError) {

Write-Warning “Bad path ‘$Path‘ ($($ResolveError[0].CategoryInfo.Category))”

}
Foreach ($PathItem in $PathItems) {

# Read the current file and extract the Exif DateTaken property
$ImageFile=(Get-ChildItem $PathItem.Path).FullName
Try {

$FileStream=New-Object System.IO.FileStream($ImageFile,

[System.IO.FileMode]::Open,

[System.IO.FileAccess]::Read,

[System.IO.FileShare]::Read,

1024,     # Buffer size

[System.IO.FileOptions]::SequentialScan

)

$Img=[System.Drawing.Imaging.Metafile]::FromStream($FileStream)

$ExifDT=$Img.GetPropertyItem(‘36867’)

}

Catch{

Write-Warning “Check $ImageFile is a valid image file ($_)”

If ($Img) {$Img.Dispose()}

If ($FileStream) {$FileStream.Close()}

Break

}

# Convert the raw Exif data
Try {

$ExifDtString=[System.Text.Encoding]::ASCII.GetString($ExifDT.Value)
# Convert the result to a [DateTime]

# Note: This looks like a string, but it has a trailing zero (0x00) character that 

# confuses ParseExact unless we include the zero in the ParseExact pattern….
$OldTime=[datetime]::ParseExact($ExifDtString,“yyyy:MM:dd HH:mm:ss`0”,$Null)

}

Catch {

Write-Warning “Problem reading Exif DateTaken string in $ImageFile ($_)”

Break

}

Finally {

If ($Img) {$Img.Dispose()}

If ($FileStream) {$FileStream.Close()}

}
Write-Verbose “Extracted EXIF infomation from $ImageFile“

Write-Verbose “Original Time is $($OldTime.ToString(‘F’))“
# Decorate the path object with the EXIF dates and pass it on…
$PathItem | Add-Member -MemberType NoteProperty -Name ExifDateTaken -Value $OldTime -PassThru
} # End Foreach Path
} # End Process Block
End

{

# There is no end processing…

}
} # End Function
# ——————————————————————————————————
Function Update-ExifDateTaken {

<#

.Synopsis

Changes the DateTaken EXIF property in an image file.

.DESCRIPTION

This script cmdlet updates the EXIF DateTaken property in an image by adding an offset to the 

existing DateTime value.  The offset (which must be able to be interpreted as a [TimeSpan] type)

can be positive or negative – moving the DateTaken value to a later or earlier time, respectively.

This can be useful (for example) to correct times where the camera clock was wrong for some reason – 

perhaps because of timezones; or to synchronise photo times from different cameras.

.PARAMETER Path

The image file or files to process.

.PARAMETER Offset

The time offset by which the EXIF DateTaken value should be adjusted.

Offset can be positive or negative and must be convertible to a [TimeSpan] type.

.PARAMETER PassThru

Switch parameter, if specified the paths of the image files processed are written to the pipeline.

The PathInfo objects are additionally decorated with the Old and New EXIF DateTaken values.

.EXAMPLE

Update-ExifDateTaken img3.jpg -Offset 0:10:0  -WhatIf

(Update the img3.jpg file, adding 10 minutes to the DateTaken property)

.EXAMPLE

Update-ExifDateTaken *3.jpg -Offset -0:01:30 -Passthru|ft path, exifdatetaken

(Subtract 1 Minute 30 Seconds from the DateTaken value on all matching files in the current folder)

.EXAMPLE

gci *.jpeg,*.jpg|Update-ExifDateTaken -Offset 0:05:00

(Update multiple files from the pipeline)

.EXAMPLE

gci *.jpg|Update-ExifDateTaken -Offset 0:5:0 -PassThru|Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss}.jpg” -f $_.ExifDateTaken}

(Updates the EXIF DateTaken on multiple files and renames the files based on the new time)

.OUTPUTS

If -PassThru is specified, the scripcmdlet outputs PathInfo objects with additional ExifDateTaken

and ExifOriginalDateTaken properties that can be used for later processing.

.NOTES

This scriptcmdlet will overwrite files without warning – take backups first…

.FUNCTIONALITY

Modifies the EXIF DateTaken image property on a specified image file.

#>
[CmdletBinding(SupportsShouldProcess=$True)]
Param (

[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]

[Alias(‘FullName’, ‘FileName’)]

$Path,
[Parameter(Mandatory=$True)]

[TimeSpan]$Offset,
[Switch]$PassThru

)
Begin

{

Set-StrictMode -Version Latest

If ($PSVersionTable.PSVersion.Major -lt 3) {

Add-Type -AssemblyName “System.Drawing”

}
}
Process

{

# Cater for arrays of filenames and wild-cards by using Resolve-Path

Write-Verbose “Processing input item ‘$Path‘”
$PathItems=Resolve-Path $Path -ErrorAction SilentlyContinue -ErrorVariable ResolveError

If ($ResolveError) {

Write-Warning “Bad path ‘$Path‘ ($($ResolveError[0].CategoryInfo.Category))”

}
Foreach ($PathItem in $PathItems) {

# Read the current file and extract the Exif DateTaken property
$ImageFile=(Get-ChildItem $PathItem.Path).FullName
Try {

$FileStream=New-Object System.IO.FileStream($ImageFile,

[System.IO.FileMode]::Open,

[System.IO.FileAccess]::Read,

[System.IO.FileShare]::Read,

1024,     # Buffer size

[System.IO.FileOptions]::SequentialScan

)

$Img=[System.Drawing.Imaging.Metafile]::FromStream($FileStream)

$ExifDT=$Img.GetPropertyItem(‘36867’)

}

Catch{

Write-Warning “Check $ImageFile is a valid image file ($_)”

If ($Img) {$Img.Dispose()}

If ($FileStream) {$FileStream.Close()}

Break

}

#region Convert the raw Exif data and modify the time
Try {

$ExifDtString=[System.Text.Encoding]::ASCII.GetString($ExifDT.Value)
# Convert the result to a [DateTime]

# Note: This looks like a string, but it has a trailing zero (0x00) character that 

# confuses ParseExact unless we include the zero in the ParseExact pattern….
$OldTime=[datetime]::ParseExact($ExifDtString,“yyyy:MM:dd HH:mm:ss`0”,$Null)

}

Catch {

Write-Warning “Problem reading Exif DateTaken string in $ImageFile ($_)”

# Only continue if an absolute time was specified…

#Todo: Add an absolute parameter and a parameter-set

# If ($Absolute) {Continue} Else {Break}

$Img.Dispose();

$FileStream.Close()

Break

}
Write-Verbose “Extracted EXIF infomation from $ImageFile“

Write-Verbose “Original Time is $($OldTime.ToString(‘F’))“
Try {

# Convert the time by adding the offset

$NewTime=$OldTime.Add($Offset)

}

Catch {

Write-Warning “Problem with time offset $Offset ($_)”

$Img.Dispose()

$FileStream.Close()

Break

}
# Convert to a string, changing slashes back to colons in the date.  Include trailing 0x00…

$ExifTime=$NewTime.ToString(“yyyy:MM:dd HH:mm:ss`0”)
Write-Verbose “New Time is $($NewTime.ToString(‘F’)) (Exif: $ExifTime)”
#endregion
# Overwrite the EXIF DateTime property in the image and set

$ExifDT.Value=[Byte[]][System.Text.Encoding]::ASCII.GetBytes($ExifTime)

$Img.SetPropertyItem($ExifDT)
# Create a memory stream to save the modified image…

$MemoryStream=New-Object System.IO.MemoryStream
Try {

# Save to the memory stream then close the original objects

# Save as type $Img.RawFormat  (Usually [System.Drawing.Imaging.ImageFormat]::JPEG)

$Img.Save($MemoryStream, $Img.RawFormat)

}

Catch {

Write-Warning “Problem modifying image $ImageFile ($_)”

$MemoryStream.Close(); $MemoryStream.Dispose()

Break

}

Finally {

$Img.Dispose()

$FileStream.Close()

}
# Update the file (Open with Create mode will truncate the file)
If ($PSCmdlet.ShouldProcess($ImageFile,‘Update EXIF DateTaken’)) {

Try {

$Writer = New-Object System.IO.FileStream($ImageFile, [System.IO.FileMode]::Create)

$MemoryStream.WriteTo($Writer)

}

Catch {

Write-Warning “Problem saving to $OutFile ($_)”

Break

}

Finally {

If ($Writer) {$Writer.Flush(); $Writer.Close()}

$MemoryStream.Close(); $MemoryStream.Dispose()

}

}

# Finally, if requested, decorate the path object with the EXIF dates and pass it on…
If ($PassThru) {

$PathItem |

Add-Member -MemberType NoteProperty -Name ExifDateTaken -Value $NewTime -PassThru |

Add-Member -MemberType NoteProperty -Name ExifOriginalDateTaken -Value $OldTime -PassThru

}
} # End Foreach Path
} # End Process Block
End

{

# There is no end processing…

}
} # End Function

The Get-ExifDateTaken PowerShell Script Cmdlet

(Part 2 of 3) (Part 1) (Part 3)

{Updated: 08/2013}

The last blog post described how to update a whole bunch of photos using two script cmdlets called Get-ExifDateTaken and Update-ExifDateTaken. This post describes the first of these script cmdlets in more detail.

There are several Windows APIs (Wia, GDI etc) that can read Exif values from image files. I tend to avoid using COM if there’s a .Net alternative, so I used the [System.Drawing.Imaging.Metafile] class and its associated GetPropertyItem() method to read the Exif data I needed.

The script needs to handle pipeline input, so the first thing to do is to get the script Param() statement coded correctly:

[CmdletBinding()]Param (

[Parameter(Mandatory=$True,

ValueFromPipeline=$True,

ValueFromPipelineByPropertyName=$True)

]

[Alias(‘FullName’, ‘$FileName’)]

$Path

)

This allows image file names to be passed in as normal parameters or to be read from the pipeline using various aliases if needed.

The script cmdlet’s Process{} block needs to accept multiple file names, arrays of names, wildcards and so on. This is all handled with a call to the Resolve-Path cmdlet and we iterate over the results in a foreach:

Process

{
# Cater for arrays of filenames and wild-cards by using Resolve-Path
Write-Verbose“Processing input item ‘$Path'”$PathItems=Resolve-Path$Path-ErrorActionSilentlyContinue-ErrorVariableResolveError
If ($ResolveError) {
Write-Warning“Bad path ‘$Path’ ($($ResolveError[0].CategoryInfo.Category))”

}    Foreach ($PathItemin$PathItems) {
# Read the current file and extract the Exif DateTaken property# (……SNIP…)} # End Foreach Path} # End Process Block

Within the loop we open each image file into a FileStream (this is quicker than using the $Img.FromFile(…) method) and get the Exif DateTaken value by using GetPropertyItem(‘36867’) – Exif property 36867 is DateTaken; just love those magic numbers…

The value is converted to a [DateTime] and passed down the pipeline attached to the PathInfo object that we got from Resolve-Path. Passing rich objects in this way (rather than just outputting the DateTaken value on its own, for example) ensures that stages further down the pipeline have access to all the information they might need.

Note that in the code snippet here the error checking code has been removed for clarity – see the full listing in the last post for the complete source:

$ImageFile=$PathItem.Path$FileStream=New-ObjectSystem.IO.FileStream($ImageFile,
[System.IO.FileMode]::Open,
[System.IO.FileAccess]::Read,
[System.IO.FileShare]::Read,
1024,# Buffer size
[System.IO.FileOptions]::SequentialScan

)
$Img=[System.Drawing.Imaging.Metafile]::FromStream($FileStream)
$ExifDT=$Img.GetPropertyItem(‘36867’)# Convert the raw Exif data$ExifDtString=[System.Text.Encoding]::ASCII.GetString($ExifDT.Value)# Convert the result to a [DateTime]
# Note: This looks like a string, but it has a trailing zero (0x00) 
# character that confuses ParseExact unless we include the zero 
# in the ParseExact pattern….$OldTime=[datetime]::ParseExact($ExifDtString,“yyyy:MM:dd HH:mm:ss`0”,$Null)
$FileStream.Close(); $Img.Dispose()
Write-Verbose “Extracted EXIF infomation from $ImageFile”

Write-Verbose “Original Time is $($OldTime.ToString(‘F’))”
# Decorate the path object with the EXIF dates and pass it on…
$PathItem | Add-Member -MemberType NoteProperty -Name ExifDateTaken -Value $OldTime

Write-Output $PathItem

Note how easily PowerShell enable calls to .Net methods – a great feature of PowerShell.

The final part will look at updating Exif dates and included the full code for the Get-ExifDateTaken and Update-ExifDateTaken cmdlets.

Reading and Changing Exif Photo Times with PowerShell

(Part 1 of 3) (Part 2) (Part 3)

{Updated: 08/2013}

Back in 2011 I went off with a bunch of like-minded folk on a long-distance cycle (see, for example, here). We all had a great time, but with six separate cameras in use we came back with a whole load of photos that needed sorting out.

We wanted to combine all the photos from the trip into a single consolidated collection, but apart from renaming all the image files it was also going to be necessary to change the times (the Date Taken values) so the photos all appeared in the correct order when viewed.

Rather than trying to accurately synchronise the times on all of cameras, we used a simple trick of simultaneously pointing them all at a scene and taking a common image; this would allow us to later correct the times of each image to at least the nearest second. (See below for the actual photos!)

The obvious question is then how to actually change the times and rename the files? Well, there are countless photo programs (e.g. Windows Live Photo Gallery, Picassa, etc, etc) that would do this, but for batch updates it’s a lot easier to use PowerShell.

Here’s how to rename the files:

$FormStr=”LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg”
gci *.jpg | Get-ExifDateTaken | Ren -New {$FormStr -f $_.ExifDateTaken,(Split-Path (Split-Path $_) -Leaf)}

There are a couple of things to notice here…. Firstly, the new file name is specified in a scriptblock, this returns a dynamically calculated file name based on the original PathInfo item that was passed to the Rename cmdlet (in the $_ variable) and the time the photo was taken (passed in $_.ExifDateTaken).

The PowerShell format operator (-f) combines the template string, in the $FormStr variable, with the values necessary to calculate the new name. In this case, the photos from each camera were held in a folder named for the camera’s owner (so my photos were all in ..\Chris\img001.jpg … etc); so ‘Split-Path (Split-Path $_) –Leaf’ just returns the parent folder names – in this case, ‘Chris’, ‘David’, ‘Mick’, ‘Mike’ or ‘Andy’ [name plugs].

The date-time part of the name just uses standard DateTime format strings, so the resulting renamed image might be something like:

“LeJog 2011 07-10 14.27.21 Sunday (Chris).jpg”

…the important point here is that you can create any name you want; you might not like my choice of filename (although it is at least sortable!) but flexibility is unlimited here.

The other thing to notice is the Get-ExifDateTaken cmdlet. The source for this, along with the companion Update-ExifDateTaken cmdlet will be published in a the next two blog entries (Part 2) (Part 3).

The example above shows how to rename a bunch of image files. To actually change the Date Taken value needs a different approach. Photos store meta-data such as the Date Taken value as Exif information which is actually combined with the image data within the image file, so the image file must be opened and saved to modify the Date Taken Exif value. The Update-ExifDateTaken script cmdlet does this:

gci *.jpg | Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru | ft Path, ExifDateTaken

Here, we specify the amount of time the Date Taken value on each image file should be changed by. In the example the offset is negative, so any Date Taken values will be moved forwards to earlier times (presumably this particular camera’s clock was too fast).

Note that this technique can also be useful if you go across time zones and forget to change your camera’s clock…

Finally, here’s an example that shows how to modify the Exif Date Taken meta-data and rename the image file in the same command:

$FormStr=”LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg”
gci *.jpg | Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru |

Ren -New {$FormStr -f $_.ExifDateTaken,(Split-Path (Split-Path $_) -Leaf)}

The Get-ExifDateTaken and the Update-ExifDateTaken script cmdlets were written with input from James O’Neill’s session at the 2011 European PowerShell Deep Dive event in Frankfurt; James has written-up the session in a blog here: “Maximize the reuse of your PowerShell”.

The script cmdlets will be published in the follow-up blog entries: (Part 2) (Part 3)

Oh, and the image we used to sync the camera times? Here’s one of our tireless support drivers, Andy, holding up the lunchtime sausage:

Regex Toolkit, Prayer-Based Parsing, Bad Examples

There was a recent entry on the Scripting Guy blog showing how to use PowerShell to parse email message headers. While it’s true to say that I have a number of problems with the script itself in this article it was the regex that really caught my eye. You need to read the Scripting Guy article to understand the context, but here’s the regex:

‘Received: from([\s\S]*?)by([\s\S]*?)with([\s\S]*?);([(\s\S)*]{32,36})(?:\s\S*?)’

Unfortunately there are serious issues with the Regex. I explain why and present an alternative later in this post.

What looked immediately strange to me in the regex was the character set ‘[\s\S]’. This matches a single character that is either a space (‘\s’) or is not a space (‘\S’) – in other words it matches *any* single character (which is [almost] the same as the ‘.’ matching character)

It’s clear, too, that the regex will likely fail whenever the server names in the email headers contain the substrings ‘by’ or ‘with’ as there are no delimiters around these characters (it would be better/safer/more correct to test for white space around the delimiters using ‘\s+’ – which means ‘match one or more white space characters’; so ‘\s+by\s+’ and ‘\s+with\s+’)

Looking further on I was struggling to see what this part of the regex was supposed to do: ‘([(\s\S)*]{32,36})’, so I broke it down … The surrounding parens in this case mean it captures something – taking that away leaves ‘[(\s\S)*]{32,36}’.

The {32,36} part says ‘match the preceding pattern between 32 and 36 times’; the pattern actually being matched in this case is then ‘[(\s\S)*]’.

Because this pattern is enclosed in square brackets it means that ‘[(\s\S)*]’ is actually any single one of a set of characters – matching any of the single characters in the set. The characters it will match in this case are therefore: an open paren, ‘any space character’, ‘any non-space character’, a close paren or an asterisk. By inspection you can see that this matches any character (repeated 32-36 times).

Huh?

At this point I was thoroughly confused. This is a good time to say that (a) I have no affiliation with the RegexBuddy company or (b) that I’m not getting any payment for a plug! I can say that I’m a big fan of the RegexBuddy program; if you need to write a regex that is more complex than the average then RegexBuddy is a real help (and the documentation and regex library are great too). So, I started up RegexBuddy. Its decoding of the regular expression is included at the end of this post but it confirmed what I thought, seriously broken…

So, in the Scripting Guy article, the author is right when he says ‘If you are good at Windows PowerShell and still haven’t used regular expressions, you are missing an important weapon in your Windows PowerShell arsenal’. But, unfortunately, his solution is misleading and dangerous as an example.

Where’s the QA on the article?? This is posted on a Microsoft site; a site aimed at beginners – unfortunately not good.

A Fixed Regex

Here’s a better way. Initially this is a simple version that defines a domain name (e.g. outgoing.red.com) as any sequence of any non-blank characters; we’ll refine that in a moment:

Received: from\s+([^\s]+)\s+by\s+([^\s]+)\s+with\s+([^;]+);\s(.+)

I’m not claiming this is perfect; Regex gurus will undoubtedly have refinements. However, I will say that in comparison to the original this is shorter, more robust and, importantly, correct.

Improving Further

We can refine this further by using RegexBuddy’s library (or looking on the web) for a domain name pattern. RegexBuddy suggests this:

\b([a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,}\b

This matches ‘a-z’ or ‘0-9’ characters one of more times and allows embedded hyphens, all this followed by a dot. This pattern is then repeated (it must occur at least once). It then matches ‘a-z’ characters (at least 2 of them) in order to match the top-level domain name.

If you haven’t yet got the regex way of things, this looks incomprehensible. Breaking it down into chunks makes it manageable. If you’re trying to learn regex then buy yourself a copy of “Mastering Regular Expressions” by Jeffrey Friedl. This is the regex reference, no question. If you can, get a copy of the second edition which covers .Net regex. (Or get a copy of RegexBuddy and read the extensive help).

Non-Capturing Parens

We need to revise this domain name pattern slightly because some of the parenthesised parts of the regex here are used for grouping. Because it isn’t explicitly stated otherwise these grouping parens will, by default, also capture whatever they happen to match. This isn’t a big deal but at the very least it means that referring to the captured groups will need to use different indexes. To avoid this we can modify the grouping parens so that they don’t capture by changing them from ‘(…)’ to ‘(?:…)’, so:

\b(?:[a-z0-9]+(?:-[a-z0-9]+)*\.)+[a-z]{2,}\b

Even more gobbledygook!

Outstanding Issues

This is pretty robust now, but I can spot at least one outstanding risk. If the name of SMTP system includes a semicolon followed by white space (unlikely I agree) then this will be taken to be the delimiter between the SMTP system name and the date. This could be fixed by parsing explicitly along the date, but because the date is in the rather unfortunate RFC822 format [(Use RFC 3339/ISO 8601 format people!)] it’s not so easy to tie down. Instead, we can make sure that the semicolon we match as a delimiter is the last semicolon before the end of the string (of course, this fix assumes the date will never contain a semicolon!)

To modify the regex to do this we can change the delimiter and the final capture to:

‘;\s([^;]+)’

This gives us the following as the final regex:

Received: from\s+(\b(?:[a-z0-9]+(?:-[a-z0-9]+)*\.)+[a-z]{2,}\b)\s+by\s+(\b(?:[a-z0-9]+(?:-[a-z0-9]+)*\.)+[a-z]{2,}\b)\s+with\s+([^;]+);\s([^;]+)

Oh dear. We’ve fixed a bunch of things, but this is not very user friendly (even if you have got the regex way of thinking…)

Layout

Things can be made better by splitting the regex over multiple lines. In order to be able to do this we first have to include the ‘(?x)’ flag at the start of the regex. This turns on ‘Extended mode’ and allows white space (including newlines) in the regex pattern, as well as allowing comments. Here’s the final formatted pattern, enclosed in a Here-string. This is more complex than the original working solution but it’s also likely to work more of the time…

$regex= @’

(?x)Received:\sfrom\s+ # Starting delimiter

(

\b(?:[a-z0-9]+(?:-[a-z0-9]+)*\.)+[a-z]{2,}\b # Match domain name and capture

)

\s+by\s+    # Delimiter between domain names

(

\b(?:[a-z0-9]+(?:-[a-z0-9]+)*\.)+[a-z]{2,}\b # Match domain name and capture

)

\s+with\s+   # Delimiter between domain names and SMTP system name

(

[^;]+   # Capture SMTP system name

)

;\s     # Delimiter between SMTP system name and date

(

[^;]+   # Capture the date

)

‘@

Aren’t objects great?! If the SMTP headers were emitted as objects we wouldn’t need to do this ‘Prayer-based Parsing’ James O’Neill has recently posted on.

Finally

Finally, note that this parsing is still potentially broken. The RFC2822 header format (RFC2822 supersedes RFC822) defines a number of optional components in the header. Here’s an example extracted from the RFC:

Received: from x.y.test

by example.net

via TCP

with ESMTP

id ABC12345

Oops! Where did that ‘via TCP’ part come from??! Obviously Prayer-Based parsing is – errr, pretty shaky.

—————

Here’s what RegexBuddy says when decoding the faulty regex :

Received: from([\s\S]*?)by([\s\S]*?)with([\s\S]*?);([(\s\S)*]{32,36})(?:\s\S*?)

Match the characters “Received: from” literally «Received: from»

Match the regular expression below and capture its match into backreference number 1 «([\s\S]*?)»

Match a single character present in the list below «[\s\S]*?»

Between zero and unlimited times, as few times as possible, expanding as needed (lazy) «*?»

A whitespace character (spaces, tabs, line breaks, etc.) «\s»

Any character that is NOT a whitespace character «\S»

Match the characters “by” literally «by»

Match the regular expression below and capture its match into backreference number 2 «([\s\S]*?)»

Match a single character present in the list below «[\s\S]*?»

Between zero and unlimited times, as few times as possible, expanding as needed (lazy) «*?»

A whitespace character (spaces, tabs, line breaks, etc.) «\s»

Any character that is NOT a whitespace character «\S»

Match the characters “with” literally «with»

Match the regular expression below and capture its match into backreference number 3 «([\s\S]*?)»

Match a single character present in the list below «[\s\S]*?»

Between zero and unlimited times, as few times as possible, expanding as needed (lazy) «*?»

A whitespace character (spaces, tabs, line breaks, etc.) «\s»

Any character that is NOT a whitespace character «\S»

Match the character “;” literally «;»

Match the regular expression below and capture its match into backreference number 4 «([(\s\S)*]{32,36})»

Match a single character present in the list below «[(\s\S)*]{32,36}»

Between 32 and 36 times, as many times as possible, giving back as needed (greedy) «{32,36}»

The character “(” «(»

A whitespace character (spaces, tabs, line breaks, etc.) «\s»

Any character that is NOT a whitespace character «\S»

One of the characters “)*” «)*»

Match the regular expression below «(?:\s\S*?)»

Match a single character that is a “whitespace character” (spaces, tabs, line breaks, etc.) «\s»

Match a single character that is a “non-whitespace character” «\S*?»

Between zero and unlimited times, as few times as possible, expanding as needed (lazy) «*?»

Created with RegexBuddy