Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
@milesgratz
milesgratz committed Apr 11, 2017
1 parent ecb59aa commit 94bed91
Show file tree
Hide file tree
Showing 5 changed files with 528 additions and 0 deletions.
55 changes: 55 additions & 0 deletions FastLookup.psm1
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#requires -version 3
<#
.SYNOPSIS
Lookup a value in an array faster than Where-Object
.DESCRIPTION
Improve the speed of looking up a value in an array by creating a hashtable index.
Good for looking up results in very large arrays or CSV files (e.g Import-Csv)
.NOTES
Version: 1.0
Author: Miles Gratz
Creation Date: April 10, 2017
Purpose/Change: Initial script development
.EXAMPLE
$array = 1..10000000
$hashtable = New-FastLookup $array
Measure-Command { $array | Where-Object { $_ -eq 199999 } }
Days : 0
Hours : 0
Minutes : 0
Seconds : 9
Milliseconds : 714
Measure-Command { Get-FastLookup -Value 199999 -Array $array -Table $hashtable }
Days : 0
Hours : 0
Minutes : 0
Seconds : 0
Milliseconds : 65
[NOTE] Performance test on Windows 10 x64 (i5-6200U, 8GB RAM, SSD)
#>

$Functions = @( Get-ChildItem -Path $PSScriptRoot\*.ps1 -ErrorAction SilentlyContinue )

foreach ($Function in $Functions)
{
Try
{
. $Function.FullName
}
Catch
{
Write-Error -Message "Failed to import function $($Function.FullName): $_"
}
}

Export-ModuleMember -Alias 'FastLookup'
Export-ModuleMember -Function $Functions.BaseName
83 changes: 83 additions & 0 deletions Get-FastLookup.ps1
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
function Get-FastLookup {
<#
.SYNOPSIS
Lookup a value in an array faster than Where-Object
.DESCRIPTION
Improve the speed of looking up a value in an array by creating a hashtable index.
Good for looking up results in very large arrays or CSV files (e.g Import-Csv)
.NOTES
Version: 1.0
Author: Miles Gratz
Creation Date: April 10, 2017
Purpose/Change: Initial script development
.PARAMETER Array
A mandatory parameter specifying the array used to create a 'FastLookup' (hashtable index)
.PARAMETER Table
A mandatory parameter specifying the 'FastLookup' created by New-FastLookup
.PARAMETER Value
A mandatory parameter specifying the search criteria (e.g. "Server458")
.OUTPUTS
The object(s) in the array that match the search
.EXAMPLE
PS> $array = 1..10000000
PS> $hashtable = New-FastLookup $array
PS> Measure-Command {
$array | Where-Object { $_ -eq 199999 }
}
Days : 0
Hours : 0
Minutes : 0
Seconds : 9
Milliseconds : 306
PS> Measure-Command {
Get-FastLookup -Value 199999 -Array $array -Table $hashtable
}
Days : 0
Hours : 0
Minutes : 0
Seconds : 0
Milliseconds : 65
[NOTE] Performance test on Windows 10 x64 (i5-6200U, 8GB RAM, SSD)
#>
param(
[Parameter(Mandatory=$true)]
$Value,
[Parameter(Mandatory=$true)]
[Array]$Array,
[Parameter(Mandatory=$true)]
[Hashtable]$Table
)

Try
{
# Lookup Value in hashtable
$Index = $Table[$Value]

# Find quantity of Index values
$IndexQty = ($Index -split ",").Count

# Find objects in Array based on Index
# (if multiple, split into array)
If ($IndexQty -eq 1){ $Array[$Index] }
If ($IndexQty -ge 2){ $Array[$Index -split ","] }

}
Catch
{
$null
}
}
121 changes: 121 additions & 0 deletions New-FastLookup.ps1
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
function New-FastLookup {
<#
.SYNOPSIS
Lookup a value in an array faster than Where-Object
.DESCRIPTION
Improve the speed of looking up a value in an array by creating a hashtable index.
Good for looking up results in very large arrays or CSV files (e.g Import-Csv)
.NOTES
Version: 1.0
Author: Miles Gratz
Creation Date: April 10, 2017
Purpose/Change: Initial script development
.PARAMETER Array
A mandatory parameter specifying input array used to create 'FastLookup'
.PARAMETER Header
An optional parameter specifying the header in the input array used to create 'FastLookup'
.OUTPUTS
A hashtable, listing the values in the array and their corresponding index
.EXAMPLE
PS> $array = 1..10000000
PS> $hashtable = New-FastLookup $array
PS> Measure-Command {
$array | Where-Object { $_ -eq 199999 }
}
Days : 0
Hours : 0
Minutes : 0
Seconds : 9
Milliseconds : 306
PS> Measure-Command {
Get-FastLookup -Value 199999 -Array $array -Table $hashtable
}
Days : 0
Hours : 0
Minutes : 0
Seconds : 0
Milliseconds : 65
[NOTE] Performance test on Windows 10 x64 (i5-6200U, 8GB RAM, SSD)
#>
param(
[Parameter(Mandatory=$true)]
[array]$Array,
$Header
)

# Identify headers in input array
$Headers = $Array | Get-Member -MemberType 'NoteProperty' | Select-Object -ExpandProperty 'Name'

# Define empty hashtable and index
$HashTable = @{}
$Index = 0

#1: Header specified
#2: Header exists in array
If (($Header -ne $null) -and ($Header -in $Headers))
{
# Redefine array with only data from specified Header
$Array = $Array.$Header
}

#1: Header is specified
#2: Header does NOT exist in array
ElseIf (($Header -ne $null) -and ($Header -notin $Headers))
{
# Exit function with error
Write-Error "Specified header ($Header) does not exist in input array."
Break
}

#1: Header is NOT specified
#2: Array contains multiple Headers
ElseIf (($Header -eq $null) -and ($Headers.Count -gt 1))
{
# Exit function with error
Write-Error "Input array requires the -Header parameter (multiple columns detected)."
Break
}

# Loop through array
foreach ($Item in $Array)
{
# Add index of Item to hashtable
# Name Value
# ---- -----
# Server1 953
# Server2 1157
Try
{
$HashTable.Add($Item,$Index)
}

# Duplicate key detected, add to existing value
# Name Value
# ---- -----
# Server1 953
# Server2 1157,3325
Catch
{
$HashTable[$Item] = ($HashTable[$Item],$Index -join ",")
}

# Increment loop
$Index++
}

# Output results
$HashTable
}
99 changes: 99 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,2 +1,101 @@
# FastLookup

PowerShell Module designed to optimize the speed of searching an array.

## How It Works?

The <b>FastLookup</b> module creates a hashtable, combining the Values in an array with the Index of the item in the array. Since the Name (or "Key") of the hashtable must be unique, each duplicate result is appended in the hashtable:

PS> $array

Laptop
Workstation
Laptop
Router

PS> $hashtable

Name Value
---- -----
Laptop 0,2
Workstation 1
Router 3

The larger the array, the more significant the performance increase from using FastLookup instead of Where-Object. Here are some speed differences when looking for an object in an array using Where-Object vs FastLookup (hashtable index) method:

Array with 10000 rows (+6 columns):

Where-Object: 145ms
FastLookup: 25ms

Array with 50000 rows (+6 columns):

Where-Object: 973ms
FastLookup: 53ms

## Example (1 Column Array)

PS> $array = 1..1000000
PS> $hashtable = New-FastLookup $array

PS> Measure-Command {
$array | Where-Object { $_ -eq 199999 }
}

Days : 0
Hours : 0
Minutes : 0
Seconds : 9
Milliseconds : 306

PS> Measure-Command {
Get-FastLookup -Value 199999 -Array $array -Table $hashtable
}

Days : 0
Hours : 0
Minutes : 0
Seconds : 0
Milliseconds : 65
## Example (2+ Column Array)

PS> $array

DeviceName : Device38
Type : Security Camera
IP : 10.10.128.1
VLAN : VLAN85
Location : Los Angeles
Patch Window : Sunday

DeviceName : Device68
Type : Server
IP : 10.10.128.126
VLAN : VLAN15
Location : Chicago
Patch Window : Sunday

PS> $Hashtable = New-FastLookup -Array $Array -Header "Location"
PS> Get-FastLookup -Value "Chicago" -Array $Array -Table $Hashtable

DeviceName : Device68
Type : Server
IP : 10.10.128.126
VLAN : VLAN15
Location : Chicago
Patch Window : Sunday

## Install

Download [FastLookup.zip](https://github.com/milesgratz/FastLookup/releases/download/v1.0/FastLookup.zip) and extract the contents into `C:\Users\[User]\Documents\WindowsPowerShell\modules\Stretch` (you may have to create these directories if they don't exist.)

## Releases
* [v1.0](https://github.com/milesgratz/FastLookup/releases/download/v1.0/FastLookup.zip)

## Contributors
* [Miles Gratz](https://github.com/milesgratz)

## Contact
* Blog: [serveradventures.com](http://www.serveradventures.com)
Loading

0 comments on commit 94bed91

Please sign in to comment.