How does FileWatcher Comparison work

Introduction

This article provides a summary of how FileWatcher's comparison functionality operates. The contents of this article apply both to standard comparisons and comparisons against a different folder.

How does comparison work?

FileWatcher's comparison process will identify unchanged, modified and missing files and folder. Additionally, it will also flag up files and folders that are unknown or not included in the original reference inventory.

FileWatcher does not compare the contents of an inventory directly against filesystem objects, but another inventory. The comparison process is two fold:

1. Create a temporary inventory of the target folder.
2. Compare the original inventory against the temporary inventory

Once the temporary inventory has been successfully generated, FileWatcher will compare the two. If an entry is found in both, then FileWatcher will compare:

• Size in bytes
• Date/Time Created
• Date/Time last modified
• Checksum (files only)

If any of the above fail to match, then item is determined to have modified.

If an item is found in one inventory, but not the other, then following rules apply:

• Item in reference inventory, but not temporary inventory - determined to be missing
• Item in temporary inventory but not reference inventory - unknown item

Once a comparison process has completed, FileWatcher will output a summary of the comparison to both console and log. See examples of comparison output below for reference.

Types of comparison

Operationally, there are two types of comparison: standard and against a different folder. A standard comparison is where an inventory is compared against the original inventoried folder. You may also compare an inventory against a different folder.

Possible Issues

Large file sets

There are potential issues with large file sets (large number of files and folders). Firstly, it is theoretically possible for someone to add, modify or delete an object within the target folder after that folder has been processed for the temporary inventory. Secondly, large file sets may take a considerable period of time to compare, especially if they contain large files.

Large files

The most time-consuming item within the comparison process is the generating of file checksums for the temporary inventory. If a file is substantial in size (1GB+) - the checksum calculation may take several seconds or even minutes to complete depending on system resources.

To mitigate these issues, it is recommended that FileWatcher is run at period of low system utilisation.

Troubleshooting

As with creating an inventory, comparison operations also generate a full verbose log file. The path and name of this file may be specified at execution time or will automatically be placed within the same folder as the FileWatcher application. If a file already exists at the same path, the log will be appended.

Examples of comparison output

For reference, the outputs of two test comparisons are included below. Both of these examples show what will appear in the console and within the comparison log file.

Successful comparison

In the example below, the comparison has completed successfully and has not detected any missing, changed or unknown files or folders.

18 JAN 2019 19:29:05     Information    Writing log to 'InventoryCompare-18-Jan-2019_192905.log'
18 JAN 2019 19:29:05     Information    No comparison target defined, defaulting to 'D:\Testing\FolderToInventory' as defined in the base or reference inventory.  Comparison mode changed to fullPath.
18 JAN 2019 19:29:05     Information    Generating temporary inventory for folder 'D:\Testing\FolderToInventory'
18 JAN 2019 19:29:05     Information    Successfully generated inventory for folder 'D:\Testing\FolderToInventory'
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (18-Jan-2019 11:26:56)
18 JAN 2019 19:29:05     Information    - Filesizes match: 5.14 MB
18 JAN 2019 19:29:05     Information    - Checksums match:
18 JAN 2019 19:29:05     Information    - Result: folder unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\03_hd.png'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (16-May-2011 23:36:23)
18 JAN 2019 19:29:05     Information    - Filesizes match: 1.63 MB
18 JAN 2019 19:29:05     Information    - Checksums match: A0766BB2869543D8EB4DF610DF962EADF2D60AE0B1E7657E980057813DCE455E
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\04_hd.png'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (16-May-2011 23:36:50)
18 JAN 2019 19:29:05     Information    - Filesizes match: 1.76 MB
18 JAN 2019 19:29:05     Information    - Checksums match: 995D98D2D085E2530E78B8C3EC279FA2C4469C5E58B7F571C41982E327D852D6
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\05_hd.png'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (16-May-2011 23:37:51)
18 JAN 2019 19:29:05     Information    - Filesizes match: 1.76 MB
18 JAN 2019 19:29:05     Information    - Checksums match: 995D98D2D085E2530E78B8C3EC279FA2C4469C5E58B7F571C41982E327D852D6
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - Filesizes match: 1.77 MB
18 JAN 2019 19:29:05     Information    - Checksums match:
18 JAN 2019 19:29:05     Information    - Result: folder unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\A_Bar_at_the_Folies-Bergere_(Manet).jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:16:41)
18 JAN 2019 19:29:05     Information    - Filesizes match: 203.30 KB
18 JAN 2019 19:29:05     Information    - Checksums match: 1F034E98228CE91A2F8540648FC562CCD538C5D1F71F0C3E4AEDBC8211437BEC
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\A_Vanitas_Still_Life_(Claesz).jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:17:52)
18 JAN 2019 19:29:05     Information    - Filesizes match: 105.31 KB
18 JAN 2019 19:29:05     Information    - Checksums match: 0F4DD093C504E6A2755A818F3701E765541B71A35BB860575940E9E03619076C
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\A_Young_Girl_Reading_(Fragonard).jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:17:32)
18 JAN 2019 19:29:05     Information    - Filesizes match: 185.51 KB
18 JAN 2019 19:29:05     Information    - Checksums match: BC8C85C95EA95B492795DA98A658EC059E95B978FDCC4A91685555E534E80F34
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\Adoration_of_the_Magi_(Mantegna).jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:18:11)
18 JAN 2019 19:29:05     Information    - Filesizes match: 199.68 KB
18 JAN 2019 19:29:05     Information    - Checksums match: 32AB43FDE36C455C41C1EE84AACA159DE471F7795AF5415196FCDEEE1E9C82FB
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\Alexander's_Victory_(Altdorfer).jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:18:25)
18 JAN 2019 19:29:05     Information    - Filesizes match: 264.67 KB
18 JAN 2019 19:29:05     Information    - Checksums match: 4CC2468933C22BA82E6F266784588A7A6465275348D2F0FA412C8E0C86ED1390
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\American_Gothic_(Wood).jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:17:19)
18 JAN 2019 19:29:05     Information    - Filesizes match: 185.07 KB
18 JAN 2019 19:29:05     Information    - Checksums match: 0272AAE3167526FCAEEFD4169CA8B3F6FFE43918330C4BF760AAED859267AE36
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\Aristotle_Contemplating_the_Bust_of_Homer_(Rembrandt).jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:16:46)
18 JAN 2019 19:29:05     Information    - Filesizes match: 189.24 KB
18 JAN 2019 19:29:05     Information    - Checksums match: 232D2C082D059FE8B2F0AEDC37FC54CC4E370C658E68347683413D42B383E7E8
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\Bathing_at_Asniers.jpg'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:19:52)
18 JAN 2019 19:29:05     Information    - Filesizes match: 240.38 KB
18 JAN 2019 19:29:05     Information    - Checksums match: AD30BCE18057AA5ACBF2D4FAB2CCDA3A9E2D75520039E9D4C42C6869618239E7
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Found matching entry 'D:\Testing\FolderToInventory\Classic Fine Art\Thumbs.db'
18 JAN 2019 19:29:05     Information    - created datetimes match (18-Jan-2019 11:26:37)
18 JAN 2019 19:29:05     Information    - lastModified datetimes match (08-Feb-2008 00:23:23)
18 JAN 2019 19:29:05     Information    - Filesizes match: 240.00 KB
18 JAN 2019 19:29:05     Information    - Checksums match: 6D9E368A90D4CD284A7BF5BE420C38B31137E851EEF7199C331BFCF03F2CB318
18 JAN 2019 19:29:05     Information    - Result: file unchanged
18 JAN 2019 19:29:05     Information    Comparison completed, duration 75 milliseconds
18 JAN 2019 19:29:05     Information    - Unchanged Entries
18 JAN 2019 19:29:05     Information    - Folders: 2
18 JAN 2019 19:29:05     Information    - Files: 12
18 JAN 2019 19:29:05     Information    - Total: 14
18 JAN 2019 19:29:05     Information    - Changed Entries
18 JAN 2019 19:29:05     Information    - Folders: 0
18 JAN 2019 19:29:05     Information    - Files: 0
18 JAN 2019 19:29:05     Information    - Total: 0
18 JAN 2019 19:29:05     Information    - Missing Entries
18 JAN 2019 19:29:05     Information    - Folders: 0
18 JAN 2019 19:29:05     Information    - Files: 0
18 JAN 2019 19:29:05     Information    - Total: 0
18 JAN 2019 19:29:05     Information    - Unknown Entries
18 JAN 2019 19:29:05     Information    - Folders: 0
18 JAN 2019 19:29:05     Information    - Files: 0
18 JAN 2019 19:29:05     Information    - Total: 0

Failed comparison

In this comparison, there are a number of missing items and a couple of unrecognised (i.e. new) items.

25 JAN 2019 16:23:25     Information    Writing log to 'InventoryCompare-25-Jan-2019_162325.log'
25 JAN 2019 16:23:25     Information    Generating temporary inventory for folder 'D:\Testing\MyFolder2'
25 JAN 2019 16:23:25     Information    Successfully generated inventory for folder 'D:\Testing\MyFolder2'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\03_hd.png'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\04_hd.png'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\05_hd.png'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\A_Bar_at_the_Folies-Bergere_(Manet).jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\A_Vanitas_Still_Life_(Claesz).jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\A_Young_Girl_Reading_(Fragonard).jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\Adoration_of_the_Magi_(Mantegna).jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\Alexander's_Victory_(Altdorfer).jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\American_Gothic_(Wood).jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\Aristotle_Contemplating_the_Bust_of_Homer_(Rembrandt).jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\Bathing_at_Asniers.jpg'
25 JAN 2019 16:23:25     Information    Missing entry 'D:\Testing\FolderToInventory\Classic Fine Art\Thumbs.db'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\04_hd.png'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\45-degree-fabric-light.png'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\73351_v0_600x.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\b83136607970012f2fdf00163e41dd5b.gif'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\A_Bar_at_the_Folies-Bergere_(Manet).jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\A_Vanitas_Still_Life_(Claesz).jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\A_Young_Girl_Reading_(Fragonard).jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\Adoration_of_the_Magi_(Mantegna).jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\Alexander's_Victory_(Altdorfer).jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\American_Gothic_(Wood).jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\Aristotle_Contemplating_the_Bust_of_Homer_(Rembrandt).jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\Bathing_at_Asniers.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\Classic Fine Art\Thumbs.db'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded folder'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded folder\03_hd.png'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded folder\04_hd.png'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded folder\05_hd.png'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded images'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded images\blade runner.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded images\Screenshot (1).png'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\excluded images\shepard-fairey-war-is-over.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\070209165144a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\070209165806a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\181107185053a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\181107190544a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\181107190846a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\181107192356a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\201209183923a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\201209184017a.jpg'
25 JAN 2019 16:23:25     Information    Unknown entry 'D:\Testing\MyFolder2\ship-photos.de\201209184249a.jpg'
25 JAN 2019 16:23:25     Information    Comparison completed, duration 111 milliseconds
25 JAN 2019 16:23:25     Information    - Unchanged Entries
25 JAN 2019 16:23:25     Information    - Folders: 0
25 JAN 2019 16:23:25     Information    - Files: 0
25 JAN 2019 16:23:25     Information    - Total: 0
25 JAN 2019 16:23:25     Information    - Changed Entries
25 JAN 2019 16:23:25     Information    - Folders: 0
25 JAN 2019 16:23:25     Information    - Files: 0
25 JAN 2019 16:23:25     Information    - Total: 0
25 JAN 2019 16:23:25     Information    - Missing Entries
25 JAN 2019 16:23:25     Information    - Folders: 2
25 JAN 2019 16:23:25     Information    - Files: 12
25 JAN 2019 16:23:25     Information    - Total: 14
25 JAN 2019 16:23:25     Information    - Unknown Entries
25 JAN 2019 16:23:25     Information    - Folders: 5
25 JAN 2019 16:23:25     Information    - Files: 28
25 JAN 2019 16:23:25     Information    - Total: 33

Further Information

Step-by-step guides are available for standard comparisons and comparisons against a different folder.