8 bits

PowerShell: find and remove duplicate items

Based on Get-DuplicateFile in PowerShellPack I have written a simple script to find and remove duplicate files. Files are compared with a SHA1 hash value.

$a = @(get-duplicatefile *.* | foreach {$_.value} | foreach {$_.name})
$b = @(foreach ($i in 1..$a.length) {if ($i%2) {[void]$foreach.moveNext()} $a[$i] })
foreach ($i in $b) {remove-item .\$i}

If you want to search for duplicate files in multiple subdirectories:

$a = @(get-duplicatefile c:\mydirectory -recurse | foreach {$_.value} | foreach {$_.fullname})
$b = @(foreach ($i in 1..$a.length) {if ($i%2) {[void]$foreach.moveNext()} $a[$i] })
foreach ($i in $b) {remove-item $i}

Note: remember to install PowerShellPack first !
If you see this error: Remove-Item : Cannot bind argument to parameter 'Path' because it is null, just run the script again to complete the job.
Warning: it's always wise to make a backup of your important files before you start to remove files. Use this script on your own risk!

PowerShellPack

Microsoft (MSDN) offers an additional PowerShellPack for PowerShell v2, containing several great modules.

PowerShell: copy files from different folders

Sometimes, it can be very handy to copy files from different folders to one central location. For example, you have a lot of photo images in a lot of different directories. You can copy (or move) the .jpg files to one central folder with this function:

Function CopyFiles ([string]$source,[string]$dest,[string]$ext) {
    Foreach ($i in get-childitem $source -recurse -filter *.$ext | foreach {$_.FullName}){
        $fn = $i.substring($i.lastindexof("\"),$i.length - $i.lastindexof("\"))
        $date = Get-Date -format 'yyyMMddhhmmssfff'
        $file = $fn.replace("\","\$date")
            if (Test-Path $dest$fn) {
                Rename-Item $dest$fn $dest$file
            }
        Copy-Item $i $dest
    }
}

You use it like this:

copyfiles c:\folder c:\newfolder jpg

It's just an easy and fast function that I have added to my profile script. I can use it everytime I need to relocate a lot of files from different folders to one central location.

Multidimensional arrays in PowerShell

Like I mentioned here, it's possible to create multidimensional arrays in PowerShell. In fact, PowerShell supports two types of multidimensional arrays: jagged arrays and true multidimensional arrays.

Jagged arrays are normal PowerShell arrays that store arrays as elements. This is very cost-effective storage because dimensions can be of different size.

True multidimensional arrays use .NET to resemble a square matrix.

An example of a multidimensional (jagged) array:

$a = ('mickey','donald','pluto','brutus'),(1,2,3),('prague','paris','london','new york')

PowerShell Hex viewer

After installing the free community extensions you can use PowerShell as a Hex viewer.

Create ZIP archives with PowerShell

It can be a very good idea to store files in a ZIP archive. The ZIP file format is a data compression and archive  format. A ZIP file contains one or more files that have been compressed to reduce file size, or stored as-is.  ZIP archives can be very usefull to backup your important data.

PowerShell offers several easy ways to create ZIP archives.

How to use the PowerShell history

So you’re working in Windows PowerShell and you’ve just typed in a series of commands that perform a very useful task. Those commands are saved in the Windows PowerShell history, but now it’s time to go home. As you know, the moment you exit the Windows PowerShell console that history will be erased. Fortunately, you can save your history in an XML file:

Simple forensics with PowerShell

After installing the Community Extensions, you can easily hash files (MD5) with PowerShell:

Get-Hash file.txt

You can use this for simple forensic purposes. For example, make a text file (aka 'hash database'), containing the hash values of all files in a specific folder:

Get-Hash c:\myfirstfolder\*.* | Format-Table -property hashstring > file1.txt

You can use this newly created hash database to find matches with files in a different folder (or on a different computer). Create a hash database of the files in the other folder:

Get-Hash c:\mysecondfolder\*.jpg | Format-Table -property hashstring > file2.txt

Now, you can use PowerShell to find the matches:

Compare-Object -referenceobject $(get-content c:\myfirstfolder\file1.txt) -differenceobject $(get-content c:\mysecondfolder\file2.txt) -includeequal | where-object {$_.SideIndicator -match "=="}

This is just a very simple example, but it works and it's nice to play with :-)

Installing PowerShell Community Extensions

PowerShell, the full power of .NET, WMI and COM all from a command line. PowerShell has a steep learning curve, much like the tango, but oh, my, when you really start dancing... (Scott Hanselman)

PowerShell Community Extensions (PSCX) is aimed at providing a widely useful set of additional cmdlets, providers, aliases, filters, functions and scripts for Windows PowerShell that members of the community have expressed interest in. You can download the PSCX extensions here.

Functions in PowerShell

Functions consist of four parts:

1) function name
2) input parameters
3) body
4) return values

function myfirstfunctions{
   the code of my function
}

When you have a piece of code that you'll reuse again, it often is wise to put the code in a function.