Background “jobs” and PowerShell   10 comments

One of the things that I’m used to on my unix systems is the ability to run applications in the background and that functionality is not available in PowerShell.  However, our RunSpace architecture can be used to create a pseudo-job environment.  A "simple" script, a few functions and a custom formatting file leaves me with a pretty good experience.  I’m not able to move jobs between foreground and background, but I never did that very much anyway.
So, the experience looks like this:
PS> new-job { get-date; start-sleep 5; get-date }
Job 0 Started
#<insert 5 second wait here>
Job 0 Completed
PS> jobs 0
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
5        Completed    get-date; start-sleep 5; … 8/30/2006 4:47:32 PM
PS> (jobs 0).results
Wednesday, August 30, 2006 4:47:32 PM
Wednesday, August 30, 2006 4:47:37 PM
This gives me what I need ok – I can run the command and get the results.  Let’s do something a little more interesting (at least it is to me).  I want to find all the files on my system that are larger than 100mb and haven’t been written for more than 3months.  This would probably take some time, since I’m going to be going over the entire filesystem.  Here’s what I got:
PS> new-job { ls -rec c:\ | ?{ $_.length -gt 100mb -and $_.lastwritetime -lt [datetime]::now.adddays(-90) } }
Job 1 Started
I would like to see what’s going on – so let’s check:
PS> jobs
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
0        Completed    get-date; start-sleep 5; … 8/30/2006 4:56:25 PM
1        Running      ls -rec c:\ | ?{ $_.lengt…
after using the shell for a while, I get the following message (when I get a prompt):
Job 1 Completed
yea!  now I can check my results:
PS> jobs
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
0        Completed    get-date; start-sleep 5; … 8/30/2006 4:56:25 PM
1        Completed    ls -rec c:\ | ?{ $_.lengt… MSO060408_0001.wmv
PS> (jobs 1).results
    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Documents and Settings\jimtru\Desktop\mso
Mode                LastWriteTime     Length Name
—-                ————-     —— —-
-a—          4/9/2006   2:15 PM  368465546 MSO060408_0001.wmv
    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Documents and Settings\jimtru\Desktop\mso\audio51204 msoWav
Mode                LastWriteTime     Length Name
—-                ————-     —— —-
-a—         12/5/2005  10:53 AM  121192364 2005-12-05 10_26.wav
-a—         12/5/2005  10:54 AM  213057644 2005-12-05 10_36.wav
So, I’ve got a bunch of huge wav files that I haven’t touched for a while (they’re recordings of the Microsoft Orchestra for those of you that are curious).   I can even stop the jobs if I want:
PS> new-job { ls -rec c:\ | ?{ $_.length -gt 100mb -and $_.lastwritetime -lt [da
tetime]::now.adddays(-90) } }
Job 2 Started
PS> (jobs 2)
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
2        Running      ls -rec c:\ | ?{ $_.lengt…
PS> (jobs 2).stop()
Job 2 Stopped
PS> jobs
JobId    Status       Command                      Results
—–    ——       ——-                      ——-
0        Completed    get-date; start-sleep 5; … 8/30/2006 4:56:25 PM
1        Completed    ls -rec c:\ | ?{ $_.lengt… MSO060408_0001.wmv
2        Stopped      ls -rec c:\ | ?{ $_.lengt… MSO060408_0001.wmv
The new-job script has all the magic in it, so here it is:
# New-Job.ps1
# This script creates an object that can be used to invoke a
# scriptblock asynchronously.
param ( [scriptblock]$scriptToRun )
## Object Created – Custom Object
## void InvokeAsync([string] $script, [array] $input = @()) 
## Invokes a script asynchronously.
## void Stop([bool] $async = $false) # Stop the pipeline.
## [System.Management.Automation.Runspaces.LocalPipeline] LastPipeline     
##      The last pipeline that executed.
## [bool] IsRunning                                                        
##      Whether the last pipeline is still running.
## [System.Management.Automation.Runspaces.PipelineState] LastPipelineState
##      The state of the last pipeline to be created.
## [array] Results                                                         
##      The output of the last pipeline that was run.
## [array] LastError                                                       
##      The errors produced by the last pipeline run.
## [object] LastException                                                  
##      If the pipeline failed, the exception that caused it to fail.
## Private Fields
## [array] _lastOutput    # The objects output from the last pipeline run.
## [array] _lastError     # The errors output from the last pipeline run.
#region Message
$MultiplePipeline = "A pipeline was already running.   " +
    "Cannot invoke two pipelines concurrently."
# First check to be sure that there is a Job array
if ( test-path variable:jobs )
    if ( $global:jobs -isnot [array] )
        throw ‘$jobs exists and is not an array’
    $global:jobs = @()
# Create a runspace and open it
$config = [Management.Automation.Runspaces.RunspaceConfiguration]::Create()
$runspace = [System.Management.Automation.Runspaces.RunspaceFactory]::CreateRunspace($config)
# Create the object – we’ll use this as the collector for the entire job.
$object = new-object System.Management.Automation.PsObject
# Add the object as a note on the runspace
$object | add-member Noteproperty Runspace $runspace
# Add a field for storing the last pipeline that was run.
$object | add-member Noteproperty LastPipeline $null
# Add an invoke method to the object that takes a script to invoke asynchronously.
$invokeAsyncBody = {
  if ($args.Count -lt 1)
    throw ‘Usage: $obj.InvokeAsync([string] $script, [Optional][params][array]$inputObjects)’
  & {
    [string]$script, [array] $inputArray =  @($args[0])
    $PipelineRunning = [System.Management.Automation.Runspaces.PipelineState]::Running
    # Check that there isn’t a currently executing pipeline.
    # Only one pipeline may run at a time.
    if ($this.LastPipeline -eq $null -or
        $this.LastPipeline.PipelineStateInfo.State -ne $PipelineRunning )
      $this.LastPipeline = $this.Runspace.CreatePipeline($script)
      # if there’s input, write it into the input pipeline.
      if ($inputArray.Count -gt 0)
        $this.LastPipeline.Input.Write($inputArray, $true)
      # Set the Results and LastError to null.
      $this.Results   = $null
      $this.LastError = $null
      # GO!
      # A pipeline was running.  Report an error.
  } $args
$object | add-member ScriptMethod InvokeAsync $invokeAsyncBody
# Adds a getter script property that lets you determine whether the runspace is still running.
$get_isRunning = {
  $PipelineRunning = [System.Management.Automation.Runspaces.PipelineState]::Running 
  return -not ($this.LastPipeline -eq $null -or
               $this.LastPipeline.PipelineStateInfo.State -ne $PipelineRunning  )
$object | add-member ScriptProperty IsRunning $get_isRunning
# Add a getter for finding out the state of the last pipeline.
$get_PipelineState = { return $this.LastPipeline.PipelineStateInfo.State }
$object | add-member ScriptProperty LastPipelineState $get_PipelineState
# Add a getter script property that lets you get the last output.
$get_lastOutput = {
  if ($this._lastOutput -eq $null -and -not $this.IsRunning)
    $this._lastOutput = @($this.LastPipeline.Output.ReadToEnd())
  return $this._lastOutput
$set_lastOutput = { $this._lastOutput = $_ }
$object | add-member ScriptProperty Results $get_lastOutput $set_lastOutput
$object | add-member Noteproperty _lastOutput $null
# Add a getter for finding out the last exception thrown if any.
$get_lastException = {
  if ($this.LastPipelineState -eq "Failed" -and -not $this.IsRunning)
    return $this.LastPipeline.PipelineStateInfo.Reason
$object | add-member ScriptProperty LastException $get_lastException
# Add a getter script property that lets you get the last errors.
$get_lastError = {
  if ($this._lastError -eq $null -and -not $this.IsRunning)
    $this._lastError = @($this.LastPipeline.Error.ReadToEnd())
  return $this._lastError
$set_lastError = { $this._lastError = $args[0] }
$object | add-member ScriptProperty LastError $get_lastError $set_lastError
$object | add-member Noteproperty _lastError $null
# Add a script method for stopping the execution of the pipeline.
$stopScript = {
  if ($args.Count -gt 1)
    throw ‘Too many arguments.  Usage: $object.Stop([optional] [bool] $async’
  if ($args.Count -eq 1 -and [bool] $args[0])
$object | add-member ScriptMethod Stop $stopScript
# finally, attach the script to run to the object
$object | add-member Noteproperty Command $scriptToRun 
# Ensure that the object has a "type" for which we can build a
# formatting file.
$object.Psobject.typenames[0] = "PowerShellJobObject"
$object | add-member NoteProperty JobId $jobs.count
"Job " + $jobs.count + " Started"
# Since we add this job to the we need to be sure that
# we can remove jobs.  The clear-job function will allow for that
$global:jobs += $object
I’ve created 2 functions to help me with getting the job information (and an alias to make it a bit more UNIX like) add these to your profile.
# get my job
function get-job
([int[]]$range = 0..($jobs.count-1))
# make a UNIX like alias
alias jobs get-job
function clear-job
    # remove all the variables that hold my job info
    rm variable:jobs
    rm variable:jobshash
    # call the garbage collector, just because I can
Then to be sure that I know when a job is finished, I added this to my prompt function:
    ### Job info code – only useful for new-job script
    ### paranoia – make a jobhash hashtable so I can track what jobs are done
    if ( $jobshash -isnot [hashtable] ) { $global:jobshash = @{} }
    $global:jobs | where { $_.lastpipelinestate -ne "Running" } | foreach-object {
        if ( ! $global:jobshash[([string]$_.jobid)] )
            $global:jobshash[([string]$_.jobid)] = 1
            if ( $_ ) { write-host Job $_.jobid   $_.lastpipelinestate }
    ### End job info code
Finally, I created a format table view so I can see the output the way I want (when I get a PowerShellJobObject – see the new-job script).  My profile runs
    update-formatdata c:\powershell\format\job.format.ps1xml
to ensure that the format file get loaded.  Here’s the content of the format file
PS> get-content c:\powershell\format\job.format.ps1xml
<?xml version="1.0" encoding="utf-8" ?>
                                    if ( $_.results -is [array])
The extra functions and addition to my prompt and the formatting file are all extra’s and not really necessary to the operation of the runspace.  It just makes it easier for me to deal with.
There you have it – you can run background "jobs" too!  Right now, I use a simple array to hold all my jobs – in the future I’ll use an array list, so that way I can remove the old jobs if I want (rather than just blowing away the entire array as clear-job does.

Posted August 30, 2006 by jtruher3 in PowerShell

10 responses to “Background “jobs” and PowerShell

Subscribe to comments with RSS.

  1. Wow, this is way cool! But you are on a newer build than RC1, right? Because the mb for mega byte doesn\’t work for the public RC1 release, I had to change that to just m. Also, the "alias jobs get-job" didn\’t work, is creating aliases for scripts a feature of RC2 as well? I just put that line into my profile.

  2. If you want to speed up that file iteration example you may want to take a look at the Windows Desktop Search cmdlet that I knocked up a while ago 😉


  3. This is cool, but the new runspace doesn\’t contain any of the functions or anything from the old runspace, except those added by default. Is there a way to import the objects from the current runspace into the new runspace? Or even better, to share the objects between them, so that a change in one affects the other?
    I have many useful utility functions, but I can\’t use any of them inside the jobs. 😦

  4. With regard to loading your aliases and functions, you can do that to, you just have to add that as the first things that you do after you open the runspace.  Here\’s an example (after $
    $pdir = "C:\\Documents and Settings\\jimtru\\My Documents\\WindowsPowerShell"$localprofile =  "$pdir\\Microsoft.PowerShell_profile.ps1"$pipeline = $Runspace.CreatePipeline(". `"$localprofile`"")$pipeline.invoke()
    before I added this code, here\’s the output:
    PS> new-job { get-alias windiff }Job 0 StartedJob 0 CompletedPS>PS> jobs
    JobId    Status       Command                      Results—–    ——       ——-                      ——-0        Completed    get-alias windiff
    boo – no alias for windiff – but after I added those 4 lines, I re-ran the script and get!
    PS> new-job { get-alias windiff }Job 1 StartedJob 1 CompletedPS>PS> jobs
    JobId    Status       Command                      Results—–    ——       ——-                      ——-0        Completed    get-alias windiff1        Completed    get-alias windiff            windiff
    woo hoo!
    There\’s another way to do this,  I could have added the dot-source of the profile to the script that I handed new job.
    new-job { . "path to profile"; get-alias windiff }

  5. davidacoder –
    yep, you\’re right, this was built on the next build where "mb" is supported rather than "m"
    and I have
        set-alias alias set-alias
    as the first line in my profile – sorry about that.

  6. This is going to be quite useful – way cool!  I sure hope similar functionality makes it into the next release of PowerShell.

  7. Simply perfect !

    Krunoslav Mihalic
  8. Thank you for your code . I finally finished my project.

  9. Fix :[void] $this.LastPipeline.Input.Write($inputArray, $true)

  10. Pingback: PowerShell Runspace Debugging: Part 1 - Windows PowerShell Blog - Site Home - MSDN Blogs

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: