How to Sync Folders With PowerShell

Why would you need to sync folders?

Copying files from one folder to another is straightforward: Drag them across your window or use a simple command like this:

Copy-Item -Path C:\Folder\* -Destination \\SERVER2\c$\Folder2 

But what if you want to ensure a complete replica of a folder instead of just copying files from one folder to another? That’s when you must sync the two folders. 

Syncing is essentially just copying files, but it’s smarter. Syncing involves the following:

• Reading both folders’ entire contents
• Calculating the differences
• Deciding which files to copy in what direction

Software syncing solutions can work, but they may not allow you to do everything you want. That’s why using a PowerShell script to accomplish syncing is often the best option. 

Joey D’Antoni, principal cloud architect and Microsoft Data Platform MVP at data consultancy firm DesignMind, is an enthusiastic advocate of PowerShell in a data-driven and cloud-focused environment. “PowerShell is a really powerful tool set,” D’Antoni said. “It can be used to do just about anything in the Windows, Azure, or even Linux/MacOS ecosystem.” 

D’Antoni noted that accomplishing a specific goal, such as synchronizing folders or managing file transfers automatically, is a great way to get started with PowerShell. “Figure out the PowerShell commands to accomplish this — write error handling and build looping and logging structures,” he said. “Then you are way ahead with PowerShell.”

How to sync files with PowerShell

PowerShell is more flexible than off-the-shelf software. You can get it to perform whatever functions you want. Better still, you can enhance usability for colleagues by building an interactive menu inside a PowerShell script with point-and-click syncing. 

As with everything in scripting, there are numerous ways to sync files with PowerShell. Some methods incorporate advanced features such as automatic triggers and file hashing. We’ll look at the more straightforward method of using basic PowerShell commands, however, instead of complicated .NET objects. We’ll also manually kick off the script instead of using automatic triggers. 

1. Define the folders’ locations.

Just as you would with websites and other URLs when managing IIS application pools with PowerShell, the first thing you must do when syncing files between folders is define each folder’s location. Here’s how:

$Folder1Path = ‘C:\Folder1’

$Folder2Path = ‘C:\Folder2’

For demonstration purposes, create these two folders on your local drive and put 101 identical text files into each folder.

2. Discover what’s in the folders. 

The next step is to retrieve a list of all files in each folder. That will provide you with two collections you can compare. Use the following commands:

$Folder1Files = Get-ChildItem -Path $Folder1Path

$Folder2Files = Get-ChildItem -Path $Folder2Path

These commands store the contents of each folder in variables, allowing you to analyze and compare the files in $Folder1Path and $Folder2Path for any differences.

3. Compare the folders’ contents. 

Now you must compare the contents of both folders. A helpful PowerShell cmdlet for this is Compare-Object, which allows you to specify two different collections. It will then look at each one and output which items are in the difference collection and which items in the reference collection are different.

Compare-Object -ReferenceObject $Folder1Files -DifferenceObject $Folder2Files

If the collections are identical, Compare-Object will return no output. If you add a file to one of the folders, however, the cmdlet will indicate the differences, showing that the folders are out of sync.

In that example, you created a dummy file in $Folder1, causing the folders to be out of sync. Compare-Object shows that newfile1234.xml is in $Folder1, but not in $Folder2. 

Now you must copy whatever files Compare-Object finds to the folder that doesn’t contain the file. That’s similar to how you would manage file system ACLs with PowerShell to identify and address what’s missing. 

To sync the folders, you must create a script like this:

The output of Compare-Object will now contain all the different files. You’ll then read each file into a for-each loop and check to see which side must copy which files. 

It works well, but you’re not done yet.

How to remove files with PowerShell

You may need to remove files at some point. If a file is removed from one folder, for example, you may want to remove the file in the other folder instead of copying over the new file.

In this example, we’ll remove a file from $Folder2, and Compare-Object can spot the file to be removed.

Next you must remove the file (or files) that Compare-Object finds:

The file will be removed. 

Syncing folders is like managing user profiles in PowerShell. In both cases, you ensure consistency through automation. With user profiles, you ensure each user has the same settings; with syncing folders, you ensure each folder has the same files.

D’Antoni emphasized the importance of testing when rolling out changes. “Automation is awesome, but very powerful,” he said. “If you can change something on thousands of servers at once, that’s awesome. But you can also break thousands of servers very quickly. Pro tip: PowerShell has a -whatif flag to test your code without actually running it. Even after that, always test on a subset of systems.”

Expert advice on PowerShell

D’Antoni regularly uses PowerShell to automate tasks such as managing cloud infrastructure, orchestrating data workflows, and configuring environments to support advanced analytics solutions. The same principles can be applied to simpler but equally important tasks, such as syncing folders efficiently and accurately. Consider the following advice:

• Use PowerShell modules to streamline operations. D’Antoni highlighted the importance of using PowerShell modules to streamline operations. “As you build a collection of PowerShell scripts for your usage, using modules allows you to include helper functions — reusable utilities that simplify larger workflows or common tasks,” he said. “Modules also make it easier to deploy code to other computers by bundling related scripts together, improving efficiency and consistency across your environments.”
• PowerShell is a robust automation tool. D’Antoni shared insights from his time at Comcast, where he served as principal architect for the firm’s SQL Server solutions. “At Comcast, I worked with a team to build a private cloud,” he said. “PowerShell drove all of our automation in the Windows space — all of the pre- and post-deployment configuration. That was a lot of dev work, but another good example is the dbatools module, an open-source module that helps administrators automate complex SQL Server database-related tasks.” D’Antoni also emphasized PowerShell’s versatility. “It was also really effective for managing smaller, essential tasks like syncing folders for greater consistency and efficiency,” he said. 
• Just start coding. D’Antoni advised PowerShell beginners to dive in. “Just start writing code,” he said. “Find your problem and start with it. PowerShell in a Month of Lunches is an excellent book that you can start learning with.”

Mark Fairlie contributed to this article.