Automate task using script is always helpful, it reduced human error and efforts, but sometimes it still takes a lots of time to execute same command on multiple remote servers one by one. For example, my disk utilization script, it works perfectly fine, but taking lots of time to generate report for thousands of servers. To get the result faster I started looking for using Background Jobs, and the result was really promising.
Earlier approx. for 500 servers, it took almost 10 to 15 seconds each and total more than 2 hours, and after using PS Background Jobs, it takes less than 5 minute to generate the report.
This is what I did with my script,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | Clear-Host $ExportFileLocation = "$env:USERPROFILE\Desktop\DiskReport.csv" $Servers = Get-Content PATH:\ServerList.txt Get-Job "DISK*" | Stop-Job Get-Job "DISK*" | Remove-Job $MAXJOB = "20" foreach ($Server in $Servers) { $i++ Write-Progress -Activity "Creating job for Disk Utilization Report.." -Status $Server -PercentComplete (100*$i/($Servers.count)) Start-Job -Name "DISK.$Server" -ScriptBlock { #SB-Start $LDisks = Get-WmiObject -ComputerName $args[0] Win32_LogicalDisk $LocalDisks = $LDisks | ? {$_.DriveType -eq '3'} $LocalDiskReport = @() if($LocalDisks) { foreach ($LocalDisk in $LocalDisks) { $Drive = $LocalDisk.DeviceID $VolumeName = $LocalDisk.VolumeName $TotalSpace = [math]::Round(($LocalDisk.Size)/1mb) $PartitionType = $LocalDisk.FileSystem $FreeSpace = [math]::Round(($LocalDisk.FreeSpace)/1mb) $MyObject = New-Object -TypeName PSObject $MyObject | Add-Member @{ServerName=$args[0]} $MyObject | Add-Member @{Drive="$Drive"} $MyObject | Add-Member @{Name="$VolumeName"} $MyObject | Add-Member @{TotalSpace="$TotalSpace"} $MyObject | Add-Member @{FreeSpace="$FreeSpace"} $MyObject | Add-Member @{Comments="$null"} $LocalDiskReport += $MyObject } return $LocalDiskReport } else { $MyObject = New-Object -TypeName PSObject $MyObject | Add-Member @{ServerName=$args[0]} $MyObject | Add-Member @{Drive="$null"} $MyObject | Add-Member @{Name="$null"} $MyObject | Add-Member @{TotalSpace="$null"} $MyObject | Add-Member @{FreeSpace="$null"} $MyObject | Add-Member @{Comments="Failed to get WMI data"} $LocalDiskReport = $MyObject return $LocalDiskReport } #SB-END } -ArgumentList $Server | Out-Null $getRunningJobsCount = (Get-Job "DISK*" | ? {$_.State -eq "Running"}).count while ($getRunningJobsCount -gt $MAXJOB) { Write-Progress -Activity "Reached maximum number of threads ($($MAXJOB))..." -Status "Wait till it gets reduced.." -PercentComplete (100*$i/($Servers.count)) Start-Sleep 10 $getRunningJobsCount = (Get-Job "DISK*" | ? {$_.State -eq "Running"}).count } } $JobStatus = "Please wait" While (Get-Job "DISK*" | ? {$_.State -eq "Running"}) { $CurrentRunningJobs = (Get-Job "DISK*" | ? {$_.State -eq "Running"}).count Write-Progress -Activity "Jobs are running, please wait." -Status "$($CurrentRunningJobs) jobs running" -PercentComplete (100*($i-$CurrentRunningJobs)/$i) #Get-Job | Get-Member #(Get-Job).ChildJobs Clear-Host $c ="." $JobStatus = "$JobStatus $c" $JobStatus Start-Sleep 1 } $Result=@() foreach ($Job in (Get-Job | ? { $_.Name -like "DISK.*"})) { $JobResult = $null $JobResult = Receive-Job $Job $Result +=$JobResult Remove-Job $Job } $Result = $Result | select ServerName, Drive, Name, TotalSpace, FreeSpace, Comments $Result | Export-Csv $ExportFileLocation -NoTypeInformation |
Now the script will check for input file containing list of the servers and it will create jobs for every server till it reach the predefined max thread count (20 in my case), and will wait till the running job count reduced, and add jobs again. This loop will continue till jobs created for all the servers. Once all jobs created, it will wait till the status of the jobs changed from ‘running’ state to something else. Once all jobs completed it will collect the job out-put and merge them all and generate the report.
Note: While creating Jobs, make sure you use max thread, in this script each jobs took 30-35 MB memory, so you have to make sure that you have sufficient resource your jobs.
Once you execute the script, it will start adding the jobs with a progress report.
As soon as running job count reached to max thread count, it will wait till it get reduced again
Once reduced, it will start adding jobs again.
One all jobs created, it will wait till all jobs finished, and then it will generate the report.
You can get the report on console if you running the script from ISE by typing $Result.
I hope this script will help to understand how to create background jobs. Happy Scripting J
Disclaimer: All posts and opinions on this site are provided AS IS with no warranties. These are our own personal opinions and do not represent our employer’s view in any way.
This article currently have 6,257 views
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
One comment