Archive for August 2006

Background “jobs” and PowerShell   10 comments

One of the things that I’m used to on my unix systems is the ability to run applications in the background and that functionality is not available in PowerShell.  However, our RunSpace architecture can be used to create a pseudo-job environment.  A "simple" script, a few functions and a custom formatting file leaves me with a pretty good experience.  I’m not able to move jobs between foreground and background, but I never did that very much anyway.
 
So, the experience looks like this:
 
PS> new-job { get-date; start-sleep 5; get-date }
Job 0 Started
#<insert 5 second wait here>
PS>
Job 0 Completed
PS> jobs 0
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
5        Completed    get-date; start-sleep 5; … 8/30/2006 4:47:32 PM
PS> (jobs 0).results
Wednesday, August 30, 2006 4:47:32 PM
Wednesday, August 30, 2006 4:47:37 PM
 
This gives me what I need ok – I can run the command and get the results.  Let’s do something a little more interesting (at least it is to me).  I want to find all the files on my system that are larger than 100mb and haven’t been written for more than 3months.  This would probably take some time, since I’m going to be going over the entire filesystem.  Here’s what I got:
 
PS> new-job { ls -rec c:\ | ?{ $_.length -gt 100mb -and $_.lastwritetime -lt [datetime]::now.adddays(-90) } }
Job 1 Started
I would like to see what’s going on – so let’s check:
 
PS> jobs
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
0        Completed    get-date; start-sleep 5; … 8/30/2006 4:56:25 PM
1        Running      ls -rec c:\ | ?{ $_.lengt…
 
after using the shell for a while, I get the following message (when I get a prompt):
 
Job 1 Completed
 
yea!  now I can check my results:
 
PS> jobs
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
0        Completed    get-date; start-sleep 5; … 8/30/2006 4:56:25 PM
1        Completed    ls -rec c:\ | ?{ $_.lengt… MSO060408_0001.wmv
PS> (jobs 1).results
    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Documents and Settings\jimtru\Desktop\mso
Mode                LastWriteTime     Length Name
—-                ————-     —— —-
-a—          4/9/2006   2:15 PM  368465546 MSO060408_0001.wmv
    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Documents and Settings\jimtru\Desktop\mso\audio51204 msoWav
Mode                LastWriteTime     Length Name
—-                ————-     —— —-
-a—         12/5/2005  10:53 AM  121192364 2005-12-05 10_26.wav
-a—         12/5/2005  10:54 AM  213057644 2005-12-05 10_36.wav
 
So, I’ve got a bunch of huge wav files that I haven’t touched for a while (they’re recordings of the Microsoft Orchestra for those of you that are curious).   I can even stop the jobs if I want:
 
PS> new-job { ls -rec c:\ | ?{ $_.length -gt 100mb -and $_.lastwritetime -lt [da
tetime]::now.adddays(-90) } }
Job 2 Started
PS> (jobs 2)
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
2        Running      ls -rec c:\ | ?{ $_.lengt…
PS> (jobs 2).stop()
Job 2 Stopped
PS> jobs
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
0        Completed    get-date; start-sleep 5; … 8/30/2006 4:56:25 PM
1        Completed    ls -rec c:\ | ?{ $_.lengt… MSO060408_0001.wmv
2        Stopped      ls -rec c:\ | ?{ $_.lengt… MSO060408_0001.wmv
 
The new-job script has all the magic in it, so here it is:
 
# New-Job.ps1
# This script creates an object that can be used to invoke a
# scriptblock asynchronously.
#
param ( [scriptblock]$scriptToRun )
##
## Object Created – Custom Object
##
## METHODS
##
## void InvokeAsync([string] $script, [array] $input = @()) 
## Invokes a script asynchronously.
## void Stop([bool] $async = $false) # Stop the pipeline.
##
## PROPERTIES
##
## [System.Management.Automation.Runspaces.LocalPipeline] LastPipeline     
##      The last pipeline that executed.
## [bool] IsRunning                                                        
##      Whether the last pipeline is still running.
## [System.Management.Automation.Runspaces.PipelineState] LastPipelineState
##      The state of the last pipeline to be created.
## [array] Results                                                         
##      The output of the last pipeline that was run.
## [array] LastError                                                       
##      The errors produced by the last pipeline run.
## [object] LastException                                                  
##      If the pipeline failed, the exception that caused it to fail.
##
## Private Fields
##
## [array] _lastOutput    # The objects output from the last pipeline run.
## [array] _lastError     # The errors output from the last pipeline run.
#region Message
$MultiplePipeline = "A pipeline was already running.   " +
    "Cannot invoke two pipelines concurrently."
##
## MAIN
##
# First check to be sure that there is a Job array
if ( test-path variable:jobs )
{
    if ( $global:jobs -isnot [array] )
    {
        throw ‘$jobs exists and is not an array’
    }
}
else
{
    $global:jobs = @()
}
# Create a runspace and open it
$config = [Management.Automation.Runspaces.RunspaceConfiguration]::Create()
$runspace = [System.Management.Automation.Runspaces.RunspaceFactory]::CreateRunspace($config)
$runspace.Open()
# Create the object – we’ll use this as the collector for the entire job.
$object = new-object System.Management.Automation.PsObject
# Add the object as a note on the runspace
$object | add-member Noteproperty Runspace $runspace
# Add a field for storing the last pipeline that was run.
$object | add-member Noteproperty LastPipeline $null
# Add an invoke method to the object that takes a script to invoke asynchronously.
$invokeAsyncBody = {
  if ($args.Count -lt 1)
  {
    throw ‘Usage: $obj.InvokeAsync([string] $script, [Optional][params][array]$inputObjects)’
  }
  & {
    [string]$script, [array] $inputArray =  @($args[0])
    $PipelineRunning = [System.Management.Automation.Runspaces.PipelineState]::Running
    # Check that there isn’t a currently executing pipeline.
    # Only one pipeline may run at a time.
    if ($this.LastPipeline -eq $null -or
        $this.LastPipeline.PipelineStateInfo.State -ne $PipelineRunning )
    {
      $this.LastPipeline = $this.Runspace.CreatePipeline($script)
      # if there’s input, write it into the input pipeline.
      if ($inputArray.Count -gt 0)
      {
        $this.LastPipeline.Input.Write($inputArray, $true)
      }
      $this.LastPipeline.Input.Close()
      # Set the Results and LastError to null.
      $this.Results   = $null
      $this.LastError = $null
      # GO!
      $this.LastPipeline.InvokeAsync()
    }
    else
    {
      # A pipeline was running.  Report an error.
      throw
    }
  } $args
}
$object | add-member ScriptMethod InvokeAsync $invokeAsyncBody
# Adds a getter script property that lets you determine whether the runspace is still running.
$get_isRunning = {
  $PipelineRunning = [System.Management.Automation.Runspaces.PipelineState]::Running 
  return -not ($this.LastPipeline -eq $null -or
               $this.LastPipeline.PipelineStateInfo.State -ne $PipelineRunning  )
}
$object | add-member ScriptProperty IsRunning $get_isRunning
# Add a getter for finding out the state of the last pipeline.
$get_PipelineState = { return $this.LastPipeline.PipelineStateInfo.State }
$object | add-member ScriptProperty LastPipelineState $get_PipelineState
# Add a getter script property that lets you get the last output.
$get_lastOutput = {
  if ($this._lastOutput -eq $null -and -not $this.IsRunning)
  {
    $this._lastOutput = @($this.LastPipeline.Output.ReadToEnd())
  }
  return $this._lastOutput
}
$set_lastOutput = { $this._lastOutput = $_ }
$object | add-member ScriptProperty Results $get_lastOutput $set_lastOutput
$object | add-member Noteproperty _lastOutput $null
# Add a getter for finding out the last exception thrown if any.
$get_lastException = {
  if ($this.LastPipelineState -eq "Failed" -and -not $this.IsRunning)
  {
    return $this.LastPipeline.PipelineStateInfo.Reason
  }
}
$object | add-member ScriptProperty LastException $get_lastException
# Add a getter script property that lets you get the last errors.
$get_lastError = {
  if ($this._lastError -eq $null -and -not $this.IsRunning)
  {
    $this._lastError = @($this.LastPipeline.Error.ReadToEnd())
  }
  return $this._lastError
}
$set_lastError = { $this._lastError = $args[0] }
$object | add-member ScriptProperty LastError $get_lastError $set_lastError
$object | add-member Noteproperty _lastError $null
# Add a script method for stopping the execution of the pipeline.
$stopScript = {
  if ($args.Count -gt 1)
  {
    throw ‘Too many arguments.  Usage: $object.Stop([optional] [bool] $async’
  }
  if ($args.Count -eq 1 -and [bool] $args[0])
  {
    $this.LastPipeline.StopAsync()
  }
  else
  {
    $this.LastPipeline.Stop()
  }
}
$object | add-member ScriptMethod Stop $stopScript
# finally, attach the script to run to the object
$object | add-member Noteproperty Command $scriptToRun 
# Ensure that the object has a "type" for which we can build a
# formatting file.
$object.Psobject.typenames[0] = "PowerShellJobObject"
$object.InvokeAsync($scriptToRun)
#$object
$object | add-member NoteProperty JobId $jobs.count
"Job " + $jobs.count + " Started"
# Since we add this job to the we need to be sure that
# we can remove jobs.  The clear-job function will allow for that
$global:jobs += $object
 
I’ve created 2 functions to help me with getting the job information (and an alias to make it a bit more UNIX like) add these to your profile.
 
# get my job
function get-job
([int[]]$range = 0..($jobs.count-1))
{
    $jobs[$range]
}
# make a UNIX like alias
alias jobs get-job
function clear-job
{
    # remove all the variables that hold my job info
    rm variable:jobs
    rm variable:jobshash
    # call the garbage collector, just because I can
    [system.gc]::Collect()
}
 
Then to be sure that I know when a job is finished, I added this to my prompt function:
 
    ### Job info code – only useful for new-job script
    ### paranoia – make a jobhash hashtable so I can track what jobs are done
    if ( $jobshash -isnot [hashtable] ) { $global:jobshash = @{} }
    $global:jobs | where { $_.lastpipelinestate -ne "Running" } | foreach-object {
        if ( ! $global:jobshash[([string]$_.jobid)] )
        {         
            $global:jobshash[([string]$_.jobid)] = 1
            if ( $_ ) { write-host Job $_.jobid   $_.lastpipelinestate }
        }
    }
    ### End job info code
 
Finally, I created a format table view so I can see the output the way I want (when I get a PowerShellJobObject – see the new-job script).  My profile runs
    update-formatdata c:\powershell\format\job.format.ps1xml
to ensure that the format file get loaded.  Here’s the content of the format file
 
PS> get-content c:\powershell\format\job.format.ps1xml
<?xml version="1.0" encoding="utf-8" ?>
<Configuration>
    <ViewDefinitions>
        <View>
            <Name>PowerShellJobObject</Name>
            <ViewSelectedBy>
                <TypeName>PowerShellJobObject</TypeName>
            </ViewSelectedBy>
            <TableControl>
                <TableHeaders>
                    <TableColumnHeader>
                        <Label>JobId</Label>
                        <Width>8</Width>
                    </TableColumnHeader>
                    <TableColumnHeader>
                        <Label>Status</Label>
                        <Width>12</Width>
                    </TableColumnHeader>
                    <TableColumnHeader>
                        <Label>Command</Label>
                    </TableColumnHeader>
                    <TableColumnHeader>
                        <Label>Results</Label>
                    </TableColumnHeader>
                </TableHeaders>
                <TableRowEntries>
                    <TableRowEntry>
                        <TableColumnItems>
                            <TableColumnItem>
                                <PropertyName>JobId</PropertyName>
                            </TableColumnItem>
                            <TableColumnItem>
                                <PropertyName>LastPipelineState</PropertyName>
                            </TableColumnItem>
                            <TableColumnItem>
                                <PropertyName>Command</PropertyName>
                            </TableColumnItem>
                            <TableColumnItem>
                                <ScriptBlock>
                                    if ( $_.results -is [array])
                                    {
                                        $_.results[0]
                                    }
                                    else
                                    {
                                        $_.results
                                    }
                                </ScriptBlock>
                            </TableColumnItem>
                        </TableColumnItems>
                    </TableRowEntry>
                </TableRowEntries>
            </TableControl>
        </View>
    </ViewDefinitions>
</Configuration>
 
The extra functions and addition to my prompt and the formatting file are all extra’s and not really necessary to the operation of the runspace.  It just makes it easier for me to deal with.
 
There you have it – you can run background "jobs" too!  Right now, I use a simple array to hold all my jobs – in the future I’ll use an array list, so that way I can remove the old jobs if I want (rather than just blowing away the entire array as clear-job does.
 
jim
 
 
Advertisements

Posted August 30, 2006 by jtruher3 in PowerShell