More Regular Expressions: Regex for IP v4 Addresses

I’ve seen some posts suggesting this as a useful IPv4 Regex:

\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}

This matches four groups of between 1-3 digits, with each group separated by a dot.

An obvious enhancement here would be to remove some redundancy:

(\d{1,3}\.){3}\.\d{1,3}  

or  

\d{1,3}(\.\d{1,3}){3}

These are both shorter and say the same thing.  However, there’s unfortunately an issue  with correctness.  These Regexs match IPv4 addresses, but they also match strings that aren’t valid IPv4 addresses.  The following will match, for example:

999.999.999.999

What to do?  Well, as mentioned previously, start with Jeffrey Friedl’s book “Mastering Regular Expressions”, 3rd Edition, O’Reilly, ISBN: 978-0-596-52812-6 from Amazon or directly from O’Reilly.  Read the customer reviews on these sites to see what I mean.

Otherwise, there are several good Regex generation/authoring programs out there.  I have used Jan Goyvaerts’ RegexBuddy for a long time and unconditionally recommend it.  It’s not free but I would say it’s a very worthwhile investment of 30 Euros if you intend to use regex to any degree.  Even without the program itself just the help files are great.  You can get your money back up to 3 months after purchase if you’re not happy, so try it!

If these two options don’t appeal to you then it’s off to the web.  Sadly, there’s lots and lots of dross and rubbish out there, some of it wildly misleading.

Valid Formats

To cut a long story short, match IPv4 address with this:

\b((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3} (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b

Or this:

\b((25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\b

Or this:

\b(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}\b

These all match valid IPv4 addresses without inadvertently matching other things.

Breakdown

There are two things that can help understanding of complex regexs: using intermediate patterns and using extended mode.

PowerShell can save intermediate patterns (or parts of a regex) in variables that can later be uses in a more complex string, using standard variable expansion. 

Using regex ‘Extended Mode’ allows a regex pattern to be split across lines (and to use white space and comments) and is indicated by using ‘(?x)’ at the beginning of the regex pattern.  This can be combined with a here-string to allow the regex to be more easily visualised…

Using these facilities we can break down those IPv4 regexs to something a little more comprehensible (note, I’ve used the ${xxxx} format for variable names here to save spelling out $TwoHundredToTwoFiveFive).

An IPv4 address is made up of 4 Octets – each with a value between 0 and 255.  To specify this in a regex we break the range down into 3 groups (divide and conquer), specifically 0-199, 200-249 and 250-255.  Remember that ‘[0-4]’ matches a single character (one of 0,1,2,3,4).   We then say that an Octet is either one of these by using an ( x | y | z ) alternation.  We build up the final address by inserting dots between 4 of these Octets.

Here’s the result.  Tidy and (hopefully!) comprehensible :-)

${250-255}=’25[0-5]’   # Matches 3 digit numbers between 250 and 255
${200-249}=‘2[0-4]\d’  # Matches 3 digit numbers between 200 and 249
${0-199}=‘[01]?\d\d?’  # Matches 1, 2 or 3 digit numbers between 0 and 199

$Octet=“( ${250-255} | ${200-249} | ${0-199} )”

$IPv4=@”
(?x) ^

$Octet (\.$Octet){3}

$

“@

 

IPv4 Function

Here’s a quick PowerShell function that will return the first valid IPv4 address from a given string (or $Null if the string doesn’t contain a valid address):

Extract IP Address from String
  1. # Extract an IP address from a string and return it.  Return $Null if no valid address found.
  2. # (If the string contains more than one valid address only the first one will be returned)
  3. Function ExtractValidIPAddress($String){
  4.     $IPregex=‘(?<Address>((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))’
  5.     If ($String -Match $IPregex) {$Matches.Address}
  6. }

Thus:

C:\PS> ExtractValidIPAddress ‘This is a string with an IPv4 address 192.168.0.1 in it’
192.168.0.1

Getting NTP/SNTP Network Time with PowerShell

(Update: There’s now an improved version of this script here that calculates network offsets as per the NTP rfc)

I wrote this function because I needed to check the NTP responses from a Windows Active Directory Domain Controller.  I found some NTP procedures on the web but actually most of them were using the Daytime service on Port 13 (not NTP).

Here’s a very quick version of the active part of the script:

$Server = ‘uk.pool.ntp.org’

# Construct client NTP time packet to send to specified server
# (Request Header: [00=No Leap Warning; 011=Version 3; 011=Client Mode]; 00011011 = 0x1B)

[Byte[]]$NtpData = ,0 * 48
$NtpData[0] = 0x1B  

$Socket = New-Object Net.Sockets.Socket([Net.Sockets.AddressFamily]::InterNetwork,
[Net.Sockets.SocketType]::Dgram,
[Net.Sockets.ProtocolType]::Udp)

$Socket.Connect($Server,123)
[Void]$Socket.Send($NtpData)
[Void]$Socket.Receive($NtpData)    # Returns length – should be 48…

$Socket.Close()

# Decode the received NTP time packet

# We now have the 64-bit NTP time in the last 8 bytes of the received data.
# The NTP time is the number of seconds since 1/1/1900 and is split into an
# integer part (top 32 bits) and a fractional part, multipled by 2^32, in the
# bottom 32 bits.

# Convert Integer and Fractional parts of 64-bit NTP time from byte array
$IntPart=0;  Foreach ($Byte in $NtpData[40..43]) {$IntPart  = $IntPart  * 256 + $Byte}
$FracPart=0; Foreach ($Byte in $NtpData[44..47]) {$FracPart = $FracPart * 256 + $Byte}

# Convert to Millseconds (convert fractional part by dividing value by 2^32)
[UInt64]$Milliseconds = $IntPart * 1000 + ($FracPart * 1000 / 0x100000000)

# Create UTC date of 1 Jan 1900, add the NTP offset and convert result to local time
(New-Object DateTime(1900,1,1,0,0,0,[DateTimeKind]::Utc)).AddMilliseconds($Milliseconds).ToLocalTime()

 

This just returns a [DateTime] object derived from the NTP Server.  There’s more information available though; the following full function decodes the NTP Time data and returns it in a custom PowerShell object.  Note that the returned object does not display all of its attributes by default – pipe to “Format-List *” to see all the values.

Usage: Get-NtpTime <Server>

Sample output:

PS:\>  Get-NtpTime
NtpTime          : 26/08/2012 19:54:52
NtpVersionNumber : 3
Mode_text        : server
Stratum          : 2
PollInterval     : 00:00:08
PrecisionSeconds : 4.76837158203125E-07
LocalDifference  : -00:00:00.2891348

Full function:

<#
Chris Warwick, @cjwarwickps, August 2012
chrisjwarwick.wordpress.com

Get Datetime from NTP server.

This sends an NTP time packet to the specified NTP server and reads back the response.
The NTP time packet from the server is decoded and returned.

Note: this uses NTP (rfc-1305: http://www.faqs.org/rfcs/rfc1305.html) on UDP 123.  Because the
function makes a single call to a single server this is strictly a SNTP client (rfc-2030).
Although the SNTP protocol data is similar (and can be identical) and the clients and servers
are often unable to distinguish the difference.  Where SNTP differs is that is does not
accumulate historical data (to enable statistical averaging) and does not retain a session
between client and server.

An alternative to NTP or SNTP is to use Daytime (rfc-867) on TCP port 13 – although this is an
old protocol and is not supported by all NTP servers.

See comments at the end of the script for an extract of the SNTP rfc.

#>

#Requires -Version 3

Function Get-NtpTime {

<#
.Synopsis
Gets (Simple) Network Time Protocol time (SNTP/NTP, rfc-1305, rfc-2030) from a specified server
.DESCRIPTION
This function connects to an NTP server on UDP port 123 and retrieves the current NTP time.
Selected components of the returned time information are decoded and returned in a PSObject.
.PARAMETER Server
The NTP Server to contact.  Uses pool.ntp.org by default.
.EXAMPLE
Get-NtpTime uk.pool.ntp.org
Gets time from the specified server.
.EXAMPLE
Get-NtpTime | fl *
Get time from default server (pool.ntp.org) and displays all output object attributes.
.OUTPUTS
A PSObject containing decoded values from the NTP server.  Pipe to fl * to see all attributes.
.FUNCTIONALITY
Gets NTP time from a specified server.
#>

    [CmdletBinding()]
Param (
[String]$Server = ‘pool.ntp.org’
)

    # Construct a 48-byte client NTP time packet to send to the specified server
# (Request Header: [00=No Leap Warning; 011=Version 3; 011=Client Mode]; 00011011 = 0x1B)

    [Byte[]]$NtpData = ,0 * 48
$NtpData[0] = 0x1B  

    $Socket = New-Object Net.Sockets.Socket([Net.Sockets.AddressFamily]::InterNetwork,
[Net.Sockets.SocketType]::Dgram,
[Net.Sockets.ProtocolType]::Udp)

    $LocalTime = Get-Date

    Try {
$Socket.Connect($Server,123)
[Void]$Socket.Send($NtpData)
[Void]$Socket.Receive($NtpData)
}
Catch {
Write-Error “Failed to communicate with server $Server”
Throw $_
}

    $Socket.Close()

    # Decode the received NTP time packet

    # Extract the flags from the first byte by masking and shifting (dividing)

    $LI = ($NtpData[0] -band 0xC0)/64    # Leap Second indicator
$LI_text = Switch ($LI) {
0    {‘no warning’}
1    {‘last minute has 61 seconds’}
2    {‘last minute has 59 seconds’}
3    {‘alarm condition (clock not synchronized)’}
}
$VN = ($NtpData[0] -band 0x38)/8     # Server versions number
$Mode = ($NtpData[0] -band 0x07)     # Server mode (probably ‘server’!)
$Mode_text = Switch ($Mode) {
0    {‘reserved’}
1    {‘symmetric active’}
2    {‘symmetric passive’}
3    {‘client’}
4    {‘server’}
5    {‘broadcast’}
6    {‘reserved for NTP control message’}
7    {‘reserved for private use’}
}

    $Stratum = [UInt16]$NtpData[1]   # Actually [UInt8] but we don’t have one of those…
$Stratum_text = Switch ($Stratum) {
0                            {‘unspecified or unavailable’}
1                            {‘primary reference (e.g., radio clock)’}
{$_ -ge 2 -and $_ -le 15}    {‘secondary reference (via NTP or SNTP)’}
{$_ -ge 16}                  {‘reserved’}
}

    $PollInterval = $NtpData[2]              # Poll interval – to neareast power of 2
$PollIntervalSeconds = [Math]::Pow(2, $PollInterval)

    $PrecisionBits = $NtpData[3]      # Precision in seconds to nearest power of 2
# …this is a signed 8-bit int
If ($PrecisionBits -band 0x80) {    # ? negative (top bit set)
[Int]$Precision = $PrecisionBits -bor 0xFFFFFFE0    # Sign extend
} else {
# ..this is unlikely – indicates a precision of less than 1 second
[Int]$Precision = $PrecisionBits   # top bit clear – just use positive value
}
$PrecisionSeconds = [Math]::Pow(2, $Precision)

    # We now have the 64-bit NTP time in the last 8 bytes of the received data.
# The NTP time is the number of seconds since 1/1/1900 and is split into an
# integer part (top 32 bits) and a fractional part, multipled by 2^32, in the
# bottom 32 bits.

    # Convert Integer and Fractional parts of 64-bit NTP time from byte array
$IntPart=0;  Foreach ($Byte in $NtpData[40..43]) {$IntPart  = $IntPart  * 256 + $Byte}
$FracPart=0; Foreach ($Byte in $NtpData[44..47]) {$FracPart = $FracPart * 256 + $Byte}

    # Convert to Millseconds (convert fractional part by dividing value by 2^32)
[UInt64]$Milliseconds = $IntPart * 1000 + ($FracPart * 1000 / 0x100000000)

    # Create UTC date of 1 Jan 1900, add the NTP offset and convert the result from utc to local time
$NtpTime=(New-Object DateTime(1900,1,1,0,0,0,[DateTimeKind]::Utc)).AddMilliseconds($Milliseconds).ToLocalTime()

    # Create Output object and return

    $NtpTimeObj = [PSCustomObject]@{
NtpTime = $NtpTime
LI = $LI
LI_text = $LI_text
NtpVersionNumber = $VN
Mode = $Mode
Mode_text = $Mode_text
Stratum = $Stratum
Stratum_text = $Stratum_text
PollIntervalRaw = $PollInterval
PollInterval = New-Object TimeSpan(0,0,$PollIntervalSeconds)
Precision = $Precision
PrecisionSeconds = $PrecisionSeconds
LocalDifference = [TimeSpan]($NtpTime – $LocalTime)
Raw = $NtpData   # The undecoded bytes returned from the NTP server
}

    # Set default display properties for object

    [String[]]$DefaultProperties =  ‘NtpTime’, ‘NtpVersionNumber’,’Mode_text’,’Stratum’, ‘PollInterval’,
‘PrecisionSeconds’, ‘LocalDifference’

    # Add the PSStandardMembers.DefaultDisplayPropertySet member
$ddps = New-Object Management.Automation.PSPropertySet(‘DefaultDisplayPropertySet’, $DefaultProperties)
$PSStandardMembers = [Management.Automation.PSMemberInfo[]]$ddps

    # Attach default display property set and output object
$NtpTimeObj | Add-Member -MemberType MemberSet -Name PSStandardMembers -Value $PSStandardMembers -PassThru
}

 

<#

From rfc-2030
~~~~~~~~~~~~~

NTP time packet format

                           1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|LI | VN  |Mode |    Stratum    |     Poll      |   Precision   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          Root Delay                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       Root Dispersion                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Reference Identifier                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                   Reference Timestamp (64)                    |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                   Originate Timestamp (64)                    |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                    Receive Timestamp (64)                     |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
|                    Transmit Timestamp (64)                    |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Leap Indicator (LI): This is a two-bit code warning of an impending
leap second to be inserted/deleted in the last minute of the current
day, with bit 0 and bit 1, respectively, coded as follows:

      LI       Value     Meaning
——————————————————-
00       0         no warning
01       1         last minute has 61 seconds
10       2         last minute has 59 seconds)
11       3         alarm condition (clock not synchronized)

   Version Number (VN): This is a three-bit integer indicating the
NTP/SNTP version number. The version number is 3 for Version 3 (IPv4
only) and 4 for Version 4 (IPv4, IPv6 and OSI). If necessary to
distinguish between IPv4, IPv6 and OSI, the encapsulating context
must be inspected.

   Mode: This is a three-bit integer indicating the mode, with values
defined as follows:

      Mode     Meaning
————————————
0        reserved
1        symmetric active
2        symmetric passive
3        client
4        server
5        broadcast
6        reserved for NTP control message
7        reserved for private use

   In unicast and anycast modes, the client sets this field to 3
(client) in the request and the server sets it to 4 (server) in the
reply. In multicast mode, the server sets this field to 5
(broadcast).

   Stratum: This is a eight-bit unsigned integer indicating the stratum
level of the local clock, with values defined as follows:

      Stratum  Meaning
———————————————-
0        unspecified or unavailable
1        primary reference (e.g., radio clock)
2-15     secondary reference (via NTP or SNTP)
16-255   reserved

   Poll Interval: This is an eight-bit signed integer indicating the
maximum interval between successive messages, in seconds to the
nearest power of two. The values that can appear in this field
presently range from 4 (16 s) to 14 (16284 s); however, most
applications use only the sub-range 6 (64 s) to 10 (1024 s).

   Precision: This is an eight-bit signed integer indicating the
precision of the local clock, in seconds to the nearest power of two.
The values that normally appear in this field range from -6 for
mains-frequency clocks to -20 for microsecond clocks found in some
workstations.

   Root Delay: This is a 32-bit signed fixed-point number indicating the
total roundtrip delay to the primary reference source, in seconds
with fraction point between bits 15 and 16. Note that this variable
can take on both positive and negative values, depending on the
relative time and frequency offsets. The values that normally appear
in this field range from negative values of a few milliseconds to
positive values of several hundred milliseconds.

   Root Dispersion: This is a 32-bit unsigned fixed-point number
indicating the nominal error relative to the primary reference
source, in seconds with fraction point between bits 15 and 16. The
values that normally appear in this field range from 0 to several
hundred milliseconds.

   Reference Identifier: This is a 32-bit bitstring identifying the
particular reference source. In the case of NTP Version 3 or Version
4 stratum-0 (unspecified) or stratum-1 (primary) servers, this is a
four-character ASCII string, left justified and zero padded to 32
bits. In NTP Version 3 secondary servers, this is the 32-bit IPv4
address of the reference source. In NTP Version 4 secondary servers,
this is the low order 32 bits of the latest transmit timestamp of the
reference source. NTP primary (stratum 1) servers should set this
field to a code identifying the external reference source according
to the following list. If the external reference is one of those
listed, the associated code should be used. Codes for sources not
listed can be contrived as appropriate.

      Code     External Reference Source
—————————————————————-
LOCL     uncalibrated local clock used as a primary reference for
a subnet without external means of synchronization
PPS      atomic clock or other pulse-per-second source
individually calibrated to national standards
ACTS     NIST dialup modem service
USNO     USNO modem service
PTB      PTB (Germany) modem service
TDF      Allouis (France) Radio 164 kHz
DCF      Mainflingen (Germany) Radio 77.5 kHz
MSF      Rugby (UK) Radio 60 kHz
WWV      Ft. Collins (US) Radio 2.5, 5, 10, 15, 20 MHz
WWVB     Boulder (US) Radio 60 kHz
WWVH     Kaui Hawaii (US) Radio 2.5, 5, 10, 15 MHz
CHU      Ottawa (Canada) Radio 3330, 7335, 14670 kHz
LORC     LORAN-C radionavigation system
OMEG     OMEGA radionavigation system
GPS      Global Positioning Service
GOES     Geostationary Orbit Environment Satellite

   Reference Timestamp: This is the time at which the local clock was
last set or corrected, in 64-bit timestamp format.

   Originate Timestamp: This is the time at which the request departed
the client for the server, in 64-bit timestamp format.

   Receive Timestamp: This is the time at which the request arrived at
the server, in 64-bit timestamp format.

   Transmit Timestamp: This is the time at which the reply departed the
server for the client, in 64-bit timestamp format.

#>

 

Wake On Lan Magic Packet script

There are a bunch of these to be found on the Web (see the additional notes on Origins below) although sadly some of them are either incorrect or of very dubious style.  Here’s my take on the theme in PowerShell (naturally:-)

I use this a lot – especially these days with spiralling electricity costs – to wake suspended machines and servers.

Wake on Lan uses a “Magic Packet” that consists of six bytes of 0xFF (the physical layer broadcast address), followed by 16 copies of the 6-byte (48-bit) MAC address (see http://en.wikipedia.org/wiki/Wake-on-LAN).   This packet is sent via UDP to the LAN Broadcast address (255.255.255.255) on arbitrary Port 4000 – although the layer 3 baggage is actually largely irrelevant.  Construction of this packet in PowerShell is a breeze thanks to the array semantics in the language (“$Packet = [Byte[]](,0xFF*6)+($Mac*16)” – neat).

This script has a table of saved MAC addresses to allow aliases to be specified on the command line (the real addresses have been obfuscated here) and uses a regex to validate the resulting MAC address string.  It would be possible to use DNS and the ARP Cache to resolve MAC addresses but the ARP cache will only be populated with a valid entry for any given target adapter for a relative short period of time after the last use of the address (10 minutes or less depending on usage); ARP cannot be used to dynamically resolve the address of a suspended adapter.

Here’s the script:

WakeOnLan.ps1
  1. # Send Wake-on-Lan Magic Packet to specified Mac address
  2. [CmdletBinding()]
  3. Param ($MacString=$(Throw 'Mac address is required'))
  4.  
  5. $Table=@{
  6.     Hyperion  ='00-00-00-00-00-44';
  7.     Nova      ='00-00-00-00-00-C8';
  8.     Desktop   ='00-00-00-00-00-1B';
  9.     Laptop    ='00-00-00-00-00-18';
  10.     Playroom  ='00-00-00-00-00-5C';
  11.     Betty     ='00-00-00-00-00-32';
  12.     gr8       ='00-00-00-00-00-D7'
  13. }
  14. If ($Table.ContainsKey($MacString)) {$MacString=$Table[$MacString]}
  15.  
  16. Write-Verbose "Using MAC string $MacString"
  17.  
  18. If ($MacString -NotMatch '^([0-9A-F]{2}[:-]){5}([0-9A-F]{2})$') {
  19.     Throw 'Mac address must be 6 hex bytes separated by : or -'
  20. }
  21.  
  22. # Split and convert to array of bytes
  23. $Mac=$MacString.Split('-:')|%{[Byte]"0x$_"}
  24.  
  25. # Packet is byte array; first six bytes are 0xFF, followed by 16 copies of the MAC address
  26. $Packet = [Byte[]](,0xFF*6)+($Mac*16)
  27. Write-Verbose "Broadcast packet: $([BitConverter]::ToString($Packet))"
  28.  
  29. $UdpClient=New-Object System.Net.Sockets.UdpClient
  30. $UdpClient.Connect(([System.Net.IPAddress]::Broadcast),4000)
  31. [Void]$UdpClient.Send($Packet,$Packet.Length)
  32. $UdpClient.Close()
  33. Write-Verbose "Wake-on-Lan Packet sent to $MacString"

 

Origins

I’ve been using this script for a long time and while writing this post I wondered where the idea had come from.  As is often the case it would appear that, at least for the correct entries (including those from the like of Brandon http://social.technet.microsoft.com/Forums/en-US/ITCG/thread/51ef86e5-f3d4-459c-a08a-615c669aff2e and John Savill http://www.windowsitpro.com/article/networking/Q-How-can-I-easily-send-a-magic-packet-to-wake-a-machine-on-my-subnet-), MOW is involved(http://thepowershellguy.com/blogs/posh/archive/2007/04/01/powershell-wake-on-lan-script.aspx) – although in this particular case, his blog entry is not correct but the answer is provided in the blog comments by James Manning who used to have an MSDN blog and I imagine (used to?) work for Microsoft – although the blog hasn’t been updated since 2009.

Modify Date Taken values on Photos with PowerShell, the Update-ExifDateTaken script cmdlet

(Part 3 of 3)    Download Script Here

The last two posts (Part 1, Part 2) have described the rationale behind creating these Exif cmdlets and looked at how to read Exif information from image files.  This final part looks at the Update-ExifDateTaken script cmdlet which will modify Exif meta-data contained within photo (.jpg) files.

See the examples in Part 1 for some ways you can use these script cmdlets.

Scott-Hanselman-Style DISCLAIMER: You Do Backup Your Photos Don’t You?  I am not a .Net programmer.  I have not be paid to program anything but PowerShell since around 1987 (seriously, I’m old) and .Net didn’t exist then.  The last non-PowerShell program I was actually paid to write was written in IBM System/370 Assembler.  No one taught me about .Net programming, I only know about it through PowerShell and MSDN and Bruce Payette.  I’ve used this script without any problems on literally thousands of photos.  I’m not saying don’t use it – just use it on a COPY of your precious images.

So, first up, because the script modifies files it must support the “-WhatIf” parameter.  There are two new parameters Update-ExifDateTaken version of the script: “-Offset”, which allows us to change the Exif Date Taken value by a specified amount; and “-PassThru”, which will pass the PathInfo objects along the pipeline.

(Note that Get-ExifDateTaken always passes on the PathInfo objects with the additional  Exif DateTaken [datetime] attribute; there would be no point in running the script if the output was not passed on because it otherwise has no side-effects.  In this Update version the output is optional because just updating the Exif data might be all that is needed.   If you want the output in the pipeline, specify –PassThru and you’ll get PathInfo objects decorated with additional ExifDateTaken and ExifOriginalDateTaken attributes).

The Process{} block structure is similar to the Get-ExifDateTaken script, but now there’s an extra step; once the script has the DateTaken value it updates it by adding the specified offset and replaces the value in the image file.

Pertinent points here are that to avoid file locking problems the script creates a MemoryStream object and saves the modified image data to the MemoryStream; it can then close the original input file and overwrite it (‘in-place’) by re-opening the file with a filemode of ‘Create’.   Note that this part of the code is protected by the ‘If ($PSCmdlet.ShouldProcess(…))’ test that supports the –WhatIf and –Confirm common parameters.

If the –PassThru switch parameter is the script outputs the current file’s PathInfo object along with two added attributes: the ExifDateTaken [datetime] object which reflects the modified DateTaken value; and the ExifOriginalDateTaken [datetime] object, in case that might be needed later in the pipeline.

That wraps it up.  The full download is available on SkyDrive here.  Finally, here’s the full script listing, including both the Get- and Update- variants of the script cmdlet:

ExifDateTime.ps1
<#Chris Warwick, @cjwarwickps, August 2013
chrisjwarwick.wordpress.comRevision: Now support PowerShell version 2.0 and above.This version published on SkyDrive here:
https://skydrive.live.com/redir?resid=7CB58BE453F7E567!1289&authkey=!ANY88H3ABytkigkThe script file contains two functions:

Get-ExifDateTaken -Path [filepaths]

Takes a file (fileinfo or string) or an array of these
Gets the ExifDT value (EXIF Property 36867)

Update-ExifDateTaken -Path [filepaths] -Offset [TimeSpan]

Takes a file (fileinfo or string) or an array of these
Modifies the ExifDT value (EXIF Property 36867) as specified

# Further samples:

# Just Update

gci *.jpg|Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru|ft Path, ExifDateTaken

# Update & Rename

gci *.jpg|
Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru|
Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg” -f $_.ExifDateTaken, (Split-Path (Split-Path $_) -Leaf)}

# Just Rename

gci *.jpg|
Get-ExifDateTaken |
Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg” -f $_.ExifDateTaken, (Split-Path (Split-Path $_) -Leaf)}

#>

#Requires -Version 2.0

Function Get-ExifDateTaken {
<#
.Synopsis
Gets the DateTaken EXIF property in an image file.
.DESCRIPTION
This script cmdlet reads the EXIF DateTaken property in an image and passes is down the pipeline
attached to the PathInfo item of the image file.
.PARAMETER Path
The image file or files to process.
.EXAMPLE
Get-ExifDateTaken img3.jpg
(Reads the img3.jpg file and returns the im3.jpg PathInfo item with the EXIF DateTaken attached)
.EXAMPLE
Get-ExifDateTaken *3.jpg |ft path, exifdatetaken
(Output the EXIF DateTaken values for all matching files in the current folder)
.EXAMPLE
gci *.jpeg,*.jpg|Get-ExifDateTaken
(Read multiple files from the pipeline)
.EXAMPLE
gci *.jpg|Get-ExifDateTaken|Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss}.jpg” -f $_.ExifDateTaken}
(Gets the EXIF DateTaken on multiple files and renames the files based on the time)
.OUTPUTS
The scripcmdlet outputs PathInfo objects with an additional ExifDateTaken
property that can be used for later processing.
.FUNCTIONALITY
Gets the EXIF DateTaken image property on a specified image file.
#>

[CmdletBinding()]

Param (
[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]
[Alias(‘FullName’, ‘FileName’)]
$Path
)

Begin
{
Set-StrictMode -Version Latest
If ($PSVersionTable.PSVersion.Major -lt 3) {
Add-Type -AssemblyName “System.Drawing”
}
}

Process
{
# Cater for arrays of filenames and wild-cards by using Resolve-Path
Write-Verbose “Processing input item ‘$Path‘”

$PathItems=Resolve-Path $Path -ErrorAction SilentlyContinue -ErrorVariable ResolveError
If ($ResolveError) {
Write-Warning “Bad path ‘$Path‘ ($($ResolveError[0].CategoryInfo.Category))”
}

Foreach ($PathItem in $PathItems) {
# Read the current file and extract the Exif DateTaken property

$ImageFile=(Get-ChildItem $PathItem.Path).FullName

Try {
$FileStream=New-Object System.IO.FileStream($ImageFile,
[System.IO.FileMode]::Open,
[System.IO.FileAccess]::Read,
[System.IO.FileShare]::Read,
1024,     # Buffer size
[System.IO.FileOptions]::SequentialScan
)
$Img=[System.Drawing.Imaging.Metafile]::FromStream($FileStream)
$ExifDT=$Img.GetPropertyItem(‘36867’)
}
Catch{
Write-Warning “Check $ImageFile is a valid image file ($_)”
If ($Img) {$Img.Dispose()}
If ($FileStream) {$FileStream.Close()}
Break
}
# Convert the raw Exif data

Try {
$ExifDtString=[System.Text.Encoding]::ASCII.GetString($ExifDT.Value)

# Convert the result to a [DateTime]
# Note: This looks like a string, but it has a trailing zero (0x00) character that
# confuses ParseExact unless we include the zero in the ParseExact pattern….

$OldTime=[datetime]::ParseExact($ExifDtString,“yyyy:MM:dd HH:mm:ss`0”,$Null)
}
Catch {
Write-Warning “Problem reading Exif DateTaken string in $ImageFile ($_)”
Break
}
Finally {
If ($Img) {$Img.Dispose()}
If ($FileStream) {$FileStream.Close()}
}

Write-Verbose “Extracted EXIF infomation from $ImageFile
Write-Verbose “Original Time is $($OldTime.ToString(‘F’))

# Decorate the path object with the EXIF dates and pass it on…

$PathItem | Add-Member -MemberType NoteProperty -Name ExifDateTaken -Value $OldTime -PassThru

} # End Foreach Path

} # End Process Block

End
{
# There is no end processing…
}

} # End Function

# ——————————————————————————————————

Function Update-ExifDateTaken {
<#
.Synopsis
Changes the DateTaken EXIF property in an image file.
.DESCRIPTION
This script cmdlet updates the EXIF DateTaken property in an image by adding an offset to the
existing DateTime value.  The offset (which must be able to be interpreted as a [TimeSpan] type)
can be positive or negative – moving the DateTaken value to a later or earlier time, respectively.
This can be useful (for example) to correct times where the camera clock was wrong for some reason –
perhaps because of timezones; or to synchronise photo times from different cameras.
.PARAMETER Path
The image file or files to process.
.PARAMETER Offset
The time offset by which the EXIF DateTaken value should be adjusted.
Offset can be positive or negative and must be convertible to a [TimeSpan] type.
.PARAMETER PassThru
Switch parameter, if specified the paths of the image files processed are written to the pipeline.
The PathInfo objects are additionally decorated with the Old and New EXIF DateTaken values.
.EXAMPLE
Update-ExifDateTaken img3.jpg -Offset 0:10:0  -WhatIf
(Update the img3.jpg file, adding 10 minutes to the DateTaken property)
.EXAMPLE
Update-ExifDateTaken *3.jpg -Offset -0:01:30 -Passthru|ft path, exifdatetaken
(Subtract 1 Minute 30 Seconds from the DateTaken value on all matching files in the current folder)
.EXAMPLE
gci *.jpeg,*.jpg|Update-ExifDateTaken -Offset 0:05:00
(Update multiple files from the pipeline)
.EXAMPLE
gci *.jpg|Update-ExifDateTaken -Offset 0:5:0 -PassThru|Rename-Item -NewName {“LeJog 2011 {0:MM-dd HH.mm.ss}.jpg” -f $_.ExifDateTaken}
(Updates the EXIF DateTaken on multiple files and renames the files based on the new time)
.OUTPUTS
If -PassThru is specified, the scripcmdlet outputs PathInfo objects with additional ExifDateTaken
and ExifOriginalDateTaken properties that can be used for later processing.
.NOTES
This scriptcmdlet will overwrite files without warning – take backups first…
.FUNCTIONALITY
Modifies the EXIF DateTaken image property on a specified image file.
#>

[CmdletBinding(SupportsShouldProcess=$True)]

Param (
[Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]
[Alias(‘FullName’, ‘FileName’)]
$Path,

[Parameter(Mandatory=$True)]
[TimeSpan]$Offset,

[Switch]$PassThru
)

Begin
{
Set-StrictMode -Version Latest
If ($PSVersionTable.PSVersion.Major -lt 3) {
Add-Type -AssemblyName “System.Drawing”
}

}

Process
{
# Cater for arrays of filenames and wild-cards by using Resolve-Path
Write-Verbose “Processing input item ‘$Path‘”

$PathItems=Resolve-Path $Path -ErrorAction SilentlyContinue -ErrorVariable ResolveError
If ($ResolveError) {
Write-Warning “Bad path ‘$Path‘ ($($ResolveError[0].CategoryInfo.Category))”
}

Foreach ($PathItem in $PathItems) {
# Read the current file and extract the Exif DateTaken property

$ImageFile=(Get-ChildItem $PathItem.Path).FullName

Try {
$FileStream=New-Object System.IO.FileStream($ImageFile,
[System.IO.FileMode]::Open,
[System.IO.FileAccess]::Read,
[System.IO.FileShare]::Read,
1024,     # Buffer size
[System.IO.FileOptions]::SequentialScan
)
$Img=[System.Drawing.Imaging.Metafile]::FromStream($FileStream)
$ExifDT=$Img.GetPropertyItem(‘36867’)
}
Catch{
Write-Warning “Check $ImageFile is a valid image file ($_)”
If ($Img) {$Img.Dispose()}
If ($FileStream) {$FileStream.Close()}
Break
}
#region Convert the raw Exif data and modify the time

Try {
$ExifDtString=[System.Text.Encoding]::ASCII.GetString($ExifDT.Value)

# Convert the result to a [DateTime]
# Note: This looks like a string, but it has a trailing zero (0x00) character that
# confuses ParseExact unless we include the zero in the ParseExact pattern….

$OldTime=[datetime]::ParseExact($ExifDtString,“yyyy:MM:dd HH:mm:ss`0”,$Null)
}
Catch {
Write-Warning “Problem reading Exif DateTaken string in $ImageFile ($_)”
# Only continue if an absolute time was specified…
#Todo: Add an absolute parameter and a parameter-set
# If ($Absolute) {Continue} Else {Break}
$Img.Dispose();
$FileStream.Close()
Break
}

Write-Verbose “Extracted EXIF infomation from $ImageFile
Write-Verbose “Original Time is $($OldTime.ToString(‘F’))

Try {
# Convert the time by adding the offset
$NewTime=$OldTime.Add($Offset)
}
Catch {
Write-Warning “Problem with time offset $Offset ($_)”
$Img.Dispose()
$FileStream.Close()
Break
}

# Convert to a string, changing slashes back to colons in the date.  Include trailing 0x00…
$ExifTime=$NewTime.ToString(“yyyy:MM:dd HH:mm:ss`0”)

Write-Verbose “New Time is $($NewTime.ToString(‘F’)) (Exif: $ExifTime)”

#endregion

# Overwrite the EXIF DateTime property in the image and set
$ExifDT.Value=[Byte[]][System.Text.Encoding]::ASCII.GetBytes($ExifTime)
$Img.SetPropertyItem($ExifDT)

# Create a memory stream to save the modified image…
$MemoryStream=New-Object System.IO.MemoryStream

Try {
# Save to the memory stream then close the original objects
# Save as type $Img.RawFormat  (Usually [System.Drawing.Imaging.ImageFormat]::JPEG)
$Img.Save($MemoryStream, $Img.RawFormat)
}
Catch {
Write-Warning “Problem modifying image $ImageFile ($_)”
$MemoryStream.Close(); $MemoryStream.Dispose()
Break
}
Finally {
$Img.Dispose()
$FileStream.Close()
}

# Update the file (Open with Create mode will truncate the file)

If ($PSCmdlet.ShouldProcess($ImageFile,‘Update EXIF DateTaken’)) {
Try {
$Writer = New-Object System.IO.FileStream($ImageFile, [System.IO.FileMode]::Create)
$MemoryStream.WriteTo($Writer)
}
Catch {
Write-Warning “Problem saving to $OutFile ($_)”
Break
}
Finally {
If ($Writer) {$Writer.Flush(); $Writer.Close()}
$MemoryStream.Close(); $MemoryStream.Dispose()
}
}
# Finally, if requested, decorate the path object with the EXIF dates and pass it on…

If ($PassThru) {
$PathItem |
Add-Member -MemberType NoteProperty -Name ExifDateTaken -Value $NewTime -PassThru |
Add-Member -MemberType NoteProperty -Name ExifOriginalDateTaken -Value $OldTime -PassThru
}

} # End Foreach Path

} # End Process Block

End
{
# There is no end processing…
}

} # End Function

The Get-ExifDateTaken PowerShell Script Cmdlet

(Part 2 of 3) (Part 1) (Part 3)

{Updated: 08/2013}

The last blog post described how to update a whole bunch of photos using two script cmdlets called Get-ExifDateTaken and Update-ExifDateTaken.   This post describes the first of these script cmdlets in more detail.

There are several Windows APIs (Wia, GDI etc) that can read Exif values from image files.  I tend to avoid using COM if there’s a .Net alternative, so I used the [System.Drawing.Imaging.Metafile] class and its associated GetPropertyItem() method to read the Exif data I needed.

The script needs to handle pipeline input, so the first thing to do is to get the script Param() statement coded correctly:

[CmdletBinding()]Param (
[Parameter(Mandatory=$True,
ValueFromPipeline=$True,
ValueFromPipelineByPropertyName=$True)
]
[Alias(‘FullName’, ‘$FileName’)]
$Path
)

This allows image file names to be passed in as normal parameters or to be read from the pipeline using various aliases if needed.

The script cmdlet’s Process{} block needs to accept multiple file names, arrays of names, wildcards and so on.  This is all handled with a call to the Resolve-Path cmdlet and we iterate over the results in a foreach:

Process
{
# Cater for arrays of filenames and wild-cards by using Resolve-Path
Write-Verbose “Processing input item ‘$Path'”$PathItems=Resolve-Path $Path -ErrorAction SilentlyContinue -ErrorVariable ResolveError
If ($ResolveError) {
Write-Warning “Bad path ‘$Path’ ($($ResolveError[0].CategoryInfo.Category))”
}    Foreach ($PathItem in $PathItems) {
# Read the current file and extract the Exif DateTaken property# (……SNIP…)} # End Foreach Path} # End Process Block

Within the loop we open each image file into a FileStream (this is quicker than using the $Img.FromFile(…) method) and get the Exif DateTaken value by using GetPropertyItem(‘36867’) – Exif property 36867 is DateTaken; just love those magic numbers…

The value is converted to a [DateTime] and passed down the pipeline attached to the PathInfo object that we got from Resolve-Path.  Passing rich objects in this way (rather than just outputting the DateTaken value on its own, for example) ensures that stages further down the pipeline have access to all the information they might need.

Note that in the code snippet here the error checking code has been removed for clarity – see the full listing in the last post for the complete source:

$ImageFile=$PathItem.Path$FileStream=New-Object System.IO.FileStream($ImageFile,
[System.IO.FileMode]::Open,
[System.IO.FileAccess]::Read,
[System.IO.FileShare]::Read,
1024,     # Buffer size
[System.IO.FileOptions]::SequentialScan
)
$Img=[System.Drawing.Imaging.Metafile]::FromStream($FileStream)
$ExifDT=$Img.GetPropertyItem(‘36867’)# Convert the raw Exif data$ExifDtString=[System.Text.Encoding]::ASCII.GetString($ExifDT.Value)# Convert the result to a [DateTime]
# Note: This looks like a string, but it has a trailing zero (0x00)
# character that confuses ParseExact unless we include the zero
# in the ParseExact pattern….$OldTime=[datetime]::ParseExact($ExifDtString,“yyyy:MM:dd HH:mm:ss`0”,$Null)

$FileStream.Close(); $Img.Dispose()

Write-Verbose “Extracted EXIF infomation from $ImageFile”
Write-Verbose “Original Time is $($OldTime.ToString(‘F’))”

# Decorate the path object with the EXIF dates and pass it on…

$PathItem | Add-Member -MemberType NoteProperty -Name ExifDateTaken -Value $OldTime
Write-Output $PathItem

Note how easily PowerShell enable calls to .Net methods – a great feature of  PowerShell.

The final part will look at updating Exif dates and included the full code for the Get-ExifDateTaken and Update-ExifDateTaken cmdlets.

Reading and Changing Exif Photo Times with PowerShell

(Part 1 of 3) (Part 2) (Part 3)

{Updated: 08/2013}

Back in 2011 I went off with a bunch of like-minded folk on a long-distance cycle (see, for example, here).  We all had a great time, but with six separate cameras in use we came back with a whole load of photos that needed sorting out.

We wanted to combine all the photos from the trip into a single consolidated collection, but apart from renaming all the image files it was also going to be necessary to change the times (the Date Taken values) so the photos all appeared in the correct order when viewed.

Rather than trying to accurately synchronise the times on all of cameras, we used a simple trick of simultaneously pointing them all at a scene and taking a common image; this would allow us to later correct the times of each image to at least the nearest second.  (See below for the actual photos!)

The obvious question is then how to actually change the times and rename the files?  Well, there are countless photo programs (e.g. Windows Live Photo Gallery, Picassa, etc, etc) that would do this, but for batch updates it’s a lot easier to use PowerShell.

Here’s how to rename the files:

$FormStr=”LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg”
gci *.jpg | Get-ExifDateTaken | Ren -New {$FormStr -f $_.ExifDateTaken,(Split-Path (Split-Path $_) -Leaf)}

There are a couple of things to notice here…. Firstly, the new file name is specified in a scriptblock, this returns a dynamically calculated file name based on the original PathInfo item that was passed to the Rename cmdlet (in the $_ variable) and the time the photo was taken (passed in $_.ExifDateTaken).

The PowerShell format operator (-f) combines the template string, in the $FormStr variable, with the values necessary to calculate the new name. In this case, the photos from each camera were held in a folder named for the camera’s owner (so my photos were all in ..\Chris\img001.jpg … etc); so ‘Split-Path (Split-Path $_) –Leaf’ just returns the parent folder names – in this case, ‘Chris’, ‘David’, ‘Mick’, ‘Mike’ or ‘Andy’ [name plugs].

The date-time part of the name just uses standard DateTime format strings, so the resulting renamed image might be something like:

“LeJog 2011 07-10 14.27.21 Sunday (Chris).jpg”

…the important point here is that you can create any name you want; you might not like my choice of filename (although it is at least sortable!) but flexibility is unlimited here.

The other thing to notice is the Get-ExifDateTaken cmdlet.  The source for this, along with the  companion Update-ExifDateTaken cmdlet will be published in a the next two blog entries (Part 2) (Part 3).

The example above shows how to rename a bunch of image files.  To actually change the Date Taken value needs a different approach.  Photos store meta-data such as the Date Taken value as Exif information which is actually combined with the image data within the image file, so the image file must be opened and saved to modify the Date Taken Exif value.  The Update-ExifDateTaken script cmdlet does this:

gci *.jpg | Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru | ft Path, ExifDateTaken

Here, we specify the amount of time the Date Taken value on each image file should be changed by.  In the example the offset is negative, so any Date Taken values will be moved forwards to earlier times (presumably this particular camera’s clock was too fast).

Note that this technique can also be useful if you go across time zones and forget to change your camera’s clock…

Finally, here’s an example that shows how to modify the Exif Date Taken meta-data and rename the image file in the same command:

$FormStr=”LeJog 2011 {0:MM-dd HH.mm.ss dddd} ({1}).jpg”
gci *.jpg | Update-ExifDateTaken -Offset ‘-0:07:10’ -PassThru |
Ren -New {$FormStr -f $_.ExifDateTaken,(Split-Path (Split-Path $_) -Leaf)}

The Get-ExifDateTaken and the Update-ExifDateTaken script cmdlets were written with input from James O’Neill’s session at the 2011 European PowerShell Deep Dive event in Frankfurt; James has written-up the session in a blog here: “Maximize the reuse of your PowerShell”.

The script cmdlets will be published in the follow-up blog entries: (Part 2) (Part 3)

Oh, and the image we used to sync the camera times?  Here’s one of our tireless support drivers, Andy, holding up the lunchtime sausage:

LeJog 2011 07-06 14.13.54 Wednesday (Andy) LeJog 2011 07-06 14.13.54 Wednesday (Chris) LeJog 2011 07-06 14.13.54 Wednesday (David)
LeJog 2011 07-06 14.13.54 Wednesday (Mick) (2) LeJog 2011 07-06 14.13.54 Wednesday (Mick) LeJog 2011 07-06 14.13.54 Wednesday (Mike)

Using Parameter [ValidateSet()]

I recently wanted to check a function parameter against a small set of valid values.  The Advanced Function parameter option [ValidateSet()] is exactly what’s needed, but I’d like to suggest an enhancement.

Suppose I have a function that lets a user pick from a pre-defined range of colours.  I could use this:

Function Get-Colour {

    Param (
        [Parameter(Position=0)]
        [ValidateSet('Red','Blue','Green')]
        [String]
        $Colour='Blue'
    )

#   ... rest of code

}

This is great; but consider the way that PowerShell handles parameter names by default;  when I call a function or cmdlet then as long as I make my parameter names long enough to be unique that’s good enough.  So I can say “Get-Colour –Colour Blue”  or “Get-Colour –c Blue” (in this case I can also say “Get-Colour Blue” as well) – all work as expected.

I’d like to suggest that [ValidateSet()] optionally enables this ability to shorten parameters (in this case it’s actually the arguments).  I could then say something like:

Function Get-Colour {

    Param (
        [Parameter(Position=0)]
        [ValidateSetAllowAbbreviation('Red','Blue','Green')]
        [String]
        $Colour='Blue'
    )

#   ... Rest of code

}

Allowing me to call the function with “Get-Colour r” or “Get-Colour g” and so on.

This allows me to still tightly constrain the possible argument values but without requiring that whoever calls the function has to type the fully qualified argument name.

If you think this might be a useful enhancement please vote for it on connect Here (note: you need a Live id to vote on the connect site).