Jump to content

Sky Slate Blueberry Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate
Photo

Crazy Scripting : FolderSpy v0.96 [ Synchronous ]


  • Please log in to reply
22 replies to this topic
SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005
I am experimenting with various versions of FolderSpy and this particular version calls ReadDirectoryChangesW in Synchronous mode.

ReadDirectoryChangesW requires OS >= Win 2000

The problem with synchronous mode in AHK is that AHK is single threaded and ReadDirectoryChangesW can halt/freeze every other operation when it is waiting for an event in the watched folder. Halt means: you cannot start an another thread via hotkey /settimer .. the GUI ( if any ) would not respond for any messages ( including mouse ).. not even a right click on tray icon would work!

To overcome the said effect, the script calls ReadDirectoryChangesW in a seperate thread using CreateThread() and RegisterCallback().
When called in synchronous mode, ReadDirectoryChangesW sets the variable nReadLen ( with number of bytes ) when it exits the wait state.
Therefore, the script calls ReadDirectoryChangesW in a seperate thread and loops to check the variable nReadLen to ascertain whether the wait state has ended.

An interesting fact is that ReadDirectoryChangesW can spy without enough access rights to the folder.

Irrespective of mode, ReadDirectoryChangesW keeps updating the structure and it is upto the user to decide on how frequently to call the ReadDirectoryChangesW. The user may even call it once per hour but the structure size should be large enough to hold all the folder events that may occur during the period, else it will reset and start from the beginning of the Structure. This script uses a 64KB structure.

Disclaimer: I would not say that the script is perfect as it had to undergo a lot of cosmetic changes in last few days in order to make it legible for the average user. I have been testing it over months and had no problems except that I had to clear the ListView when it grows very large ( owing to my system's meagre 256MB RAM ). The script may be considered more as a demo for multi-threading. I will post an another simple script using asynchronous mode, which should be enough for most of the needs.

To stress-test this script, press the Start button and load any MSDN page ( like this ) in Internet Explorer. I get around 2000+ notifications in a few seconds!

Posted Image

; * Folder Spy v0.97 *   by Skan  || Created : 14-Apr-2007  /// LastModified : 06-Sep-2007

#Persistent
SetBatchLines, -1
Process, Priority,, High
OnExit, ShutApp
Menu, Tray, Icon, Shell32.dll, 4
Menu, Tray, Tip , FolderSpy

WatchFolder  := A_Temp . "orary Internet Files"
WatchSubDirs := True

Loop %WatchFolder%, 1 
      WatchFolder := A_LoopFileLongPath
DllCall( "shlwapi\PathAddBackslashA", UInt,&Watchfolder )

CBA_ReadDir := RegisterCallback("ReadDirectoryChanges")

; FILE_NOTIFY_INFORMATION : http://msdn2.microsoft.com/en-us/library/aa364391.aspx

SizeOf_FNI := ( 64KB := 1024 * 64 )
VarSetCapacity( FILE_NOTIFY_INFORMATION, SizeOf_FNI, 0 ) 
PointerFNI := &FILE_NOTIFY_INFORMATION

Gui, Margin, 5, 5
Gui, Add, Edit     , x5   w574 h22 +ReadOnly vWatchFolder, %WatchFolder%
Gui, Add, Button   , x+5  w25  h22 gSelectFolder vBrowseButton, ...
Gui, Add, CheckBox , x+20      h22 vWatchSubDirs Checked 0x20, Sub Folders
Gui, Add, ListView , x5   w700 h480 +Grid vSpyLV
                   , Time|Event|File/Folder Name|Size-KB|TimeStamp [Mod]|Attrib
Gui, Add, Button   , x550  w64  h22 +Default vClear gClearListView, Clear
Gui, Add, Button   , x+10  w64  h22 +Default vStartStop gStartStop, Start

LV_ModifyCol( 1, "54" )
LV_ModifyCol( 2, "75  Center " )
LV_ModifyCol( 3, "327        " )
LV_ModifyCol( 4, "55  Integer" )
LV_ModifyCol( 5, "120        " )
LV_ModifyCol( 6, "46         " )

Gui, Add, StatusBar
SB_SetParts( 100, 500 )
GuiControl, Focus, StartStop
Gui, Show, , FolderSpy v0.96

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -
Return ;                                                  [| End of Auto-execute section |]
;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -


WatchFolder:

  GuiControlGet, WatchFolder
  GuiControlGet, WatchSubDirs

  ; CreateFile: http://msdn2.microsoft.com/en-us/library/aa914735.aspx

  hDir := DllCall( "CreateFile"
                 , Str  , WatchFolder
                 , UInt , ( FILE_LIST_DIRECTORY := 0x1 ) 
                 , UInt , ( FILE_SHARE_READ     := 0x1 ) 
                        | ( FILE_SHARE_WRITE    := 0x2 ) 
                        | ( FILE_SHARE_DELETE   := 0x4 )
                 , UInt , 0
                 , UInt , ( OPEN_EXISTING := 0x3 )
                 , UInt , ( FILE_FLAG_BACKUP_SEMANTICS := 0x2000000  ) 
                        | ( FILE_FLAG_OVERLAPPED       := 0x40000000 )
                 , UInt , 0 ) 

  Loop {

  nReadLen  := 0
  hThreadId := 0

  ; CreateThread   : http://msdn2.microsoft.com/en-us/library/ms682453.aspx

  hThread   := DllCall( "CreateThread", UInt,0, UInt,0, UInt,CBA_ReadDir
                       , UInt,0, UInt,0, UIntP,hThreadId )
         Loop { 
                If nReadLen 
                            {
                              GoSub, Decode_FILE_NOTIFY_INFORMATION
                              Break
                            }
                If !Watch
                    Break
                Sleep 100
              }

  ; TerminateThread : http://msdn2.microsoft.com/en-us/library/ms686717.aspx
  ; CloseHandle     : http://msdn2.microsoft.com/en-us/library/ms724211.aspx

  DllCall( "TerminateThread", UInt,hThread, UInt,0 )
  DllCall( "CloseHandle", UInt,hThread )  

  If !Watch
      Break
}

Return

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -

SelectFolder:

 FileSelectFolder, SelFolder, *%WatchFolder%, , Select Watch Folder
 If ( SelFolder = "" ) 
      Return

 Loop %SelFolder%, 1 
    WatchFolder := A_LoopFileLongPath
 DllCall( "shlwapi\PathAddBackslashA", UInt,&Watchfolder )

 GuiControl,, WatchFolder, %WatchFolder%
 GuiControl, Focus, StartStop

 DllCall( "CloseHandle", UInt,hThread ) 
 SetTimer, StartStop, -1

Return

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -

StartStop:

  Watch := !Watch
  If ( Watch ) {
                 GuiControl,Disable, BrowseButton
                 GuiControl,Disable, WatchSubDirs
                 GuiControl,, StartStop, Stop
                 SetTimer, WatchFolder, -1 
  }  Else      {
                 GuiControl,Enable, BrowseButton
                 GuiControl,Enable, WatchSubDirs
                 GuiControl,, StartStop, Start
  } 

Return

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -

ReadDirectoryChanges() {       ;    http://msdn2.microsoft.com/en-us/library/aa365465.aspx

 Global hDir,PointerFNI, Sizeof_FNI, WatchSubdirs, nReadlen
 Return DllCall( "ReadDirectoryChangesW"
                , UInt , hDir
                , UInt , PointerFNI
                , UInt , SizeOf_FNI
                , UInt , WatchSubDirs
                , UInt , ( FILE_NOTIFY_CHANGE_FILE_NAME   := 0x1   )
                       | ( FILE_NOTIFY_CHANGE_DIR_NAME    := 0x2   ) 
                       | ( FILE_NOTIFY_CHANGE_ATTRIBUTES  := 0x4   )
                       | ( FILE_NOTIFY_CHANGE_SIZE        := 0x8   )
                       | ( FILE_NOTIFY_CHANGE_LAST_WRITE  := 0x10  )
                       | ( FILE_NOTIFY_CHANGE_LAST_ACCESS := 0x20  )
                       | ( FILE_NOTIFY_CHANGE_CREATION    := 0x40  )
                       | ( FILE_NOTIFY_CHANGE_SECURITY    := 0x100 )
                , UIntP, nReadLen
                , UInt , 0
                , UInt , 0  )
}

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -


Decode_FILE_NOTIFY_INFORMATION:

  PointerFNI := &FILE_NOTIFY_INFORMATION

  Loop {

    NextEntry   := NumGet( PointerFNI + 0  )
    Action      := NumGet( PointerFNI + 4  )
    FileNameLen := NumGet( PointerFNI + 8  )
    FileNamePtr :=       ( PointerFNI + 12 ) 


    If ( Action = 0x1 )                            ; FILE_ACTION_ADDED    
       Event := "New File"

    If ( Action = 0x2 )                            ; FILE_ACTION_REMOVED  
       Event := "Deleted"

    If ( Action = 0x3 )                            ; FILE_ACTION_MODIFIED  
       Event := "Modified"

    If ( Action = 0x4 )                            ; FILE_ACTION_RENAMED_OLD_NAME 
       Event := "Renamed Fm"

    If ( Action = 0x5 )                            ; FILE_ACTION_RENAMED_NEW_NAME 
       Event := "Renamed To" 

    VarSetCapacity( FileNameANSI, FileNameLen )
    DllCall( "WideCharToMultiByte", UInt,0, UInt,0, UInt,FileNamePtr, UInt
           , FileNameLen,  Str,FileNameANSI, UInt,FileNameLen, UInt,0, UInt,0 )

    File := SubStr( FileNameANSI, 1, FileNameLen/2 )
    FullPath := WatchFolder . File
    FileGetAttrib, Attr, %FullPath%
    FormatTime, Time  , %A_Now%, HH:mm:ss

    If ( FileExist( FullPath ) = "" )
     {
       LV_Insert( 1, "", Time, Event, File )
       Sb_SetText( "`t" LV_GetCount() )
     }
    Else
    Loop %FullPath%
     {
       FormatTime, TStamp, %A_LoopFileTimeModified%, yyyy-MM-dd  HH:mm:ss
       LV_Insert( 1, "", Time, Event, File, A_LoopFileSizeKB, TStamp, A_LoopFileAttrib )
       Sb_SetText( "`t" LV_GetCount() )
     }

    If !NextEntry
       Break
    Else
       PointerFNI := PointerFNI + NextEntry
  }

Return 

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -

GuiClose:
ShutApp:

  DllCall( "CloseHandle", UInt,hDir )
  DllCall( "TerminateThread", UInt,hThread, UInt,0 )
  DllCall( "CloseHandle", UInt,hThread ) 
  ExitApp

Return

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -

ClearListView:

 LV_Delete() , Sb_SetText("")

Return

;:   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -

:)

System Monitor
  • Members
  • 508 posts
  • Last active: Mar 26 2012 05:13 AM
  • Joined: 09 Mar 2007
Cool!

tic
  • Members
  • 1934 posts
  • Last active: May 30 2018 08:13 PM
  • Joined: 22 Apr 2007
8)
pretty cool skan.....

I'd asked for help with this a while ago but couldnt get it to work, and now youve just gone and done it all!

I like it a lot and would love to use it in my programs :D

poetbox
  • Members
  • 113 posts
  • Last active: Sep 11 2013 08:05 AM
  • Joined: 07 Jan 2007
well down!
I like it! : 8) :lol: :p :p
Your anothor scripts from *.tw cannot be download now.

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

Your anothor scripts from *.tw cannot be download now.


:?: :roll:

poetbox
  • Members
  • 113 posts
  • Last active: Sep 11 2013 08:05 AM
  • Joined: 07 Jan 2007
Its http://www.autohotke...topic21630.html :p :lol:

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005
I confirm you that it works for me .. and BTW it is Windows SPY. :)

tic
  • Members
  • 1934 posts
  • Last active: May 30 2018 08:13 PM
  • Joined: 22 Apr 2007
little question skan. how would you go about monitoring more than 1 drive? say you wish to watch the C:\ D:\ and E:\ drive simultaneously. cheers

majkinetor !
  • Guests
  • Last active:
  • Joined: --
start script 3 times :D

tic
  • Members
  • 1934 posts
  • Last active: May 30 2018 08:13 PM
  • Joined: 22 Apr 2007

start script 3 times :D


:? thats not really a viable option, as I wish to integrate it into one of my programs (MOVEit) and the user can currently specify as many rules as they want, so it wouldn't really be an option to run a new exe as many times as there are rules, as the exe would need to be created first and then there would be loads running!

majkinetor
  • Moderators
  • 4512 posts
  • Last active: May 20 2019 07:41 AM
  • Joined: 24 May 2006
That is the only option for you as this script creates thread. As far as my tests can say, creating threads in AHK is bug prone and if I remember correctly Skan said that creating more then 3 will crash AHK. In my case results are even worst, single thread randomly crashes AHK, which seems to depend on what function you execute.

Doing it without threads will block your script.

So, the only viable solution is to use original script aunch it N times externaly, and transmit data to your script. Check out IPC module for details about interprocess communication.

This is ofcourse far from sophisticated as N processes will run for N "rules", but we are out of domain of AHK here.

Perhaps AHK can be updated to support creating threads in safe way, the way that wouldn't crash it, via some command similar to RegisterCallback, but that is for the wish list, if possible at all.
Posted Image

SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

say you wish to watch the C:\ D:\ and E:\ drive simultaneously


You may call the ReadDirectoryChangesW as many times in Asynchronous mode with a SetTimer or in a Loop. In Asynchronous mode, there is no wait state and the structure FILE_NOTIFY_INFORMATION will contain folder events that occured between the calls.
However, you have to use your own methods to decide whether some folder event had occured during the function calls:

I am able to think of two methods:

1) Hash the stucture after every call and if changed call the routine Decode_FILE_NOTIFY_INFORMATION
2) Use FindNextChangeNotification / FindNextChangeNotification to detect changes and on detection call ReadDirectoryChangesW followed by Decode_FILE_NOTIFY_INFORMATION

Synchronous Vs Asynchronous
Which method you really need depends on your actual usage.
Synchronous method is superior when timing is very critical.
For example some X program fileinstalls a file, loads it and deletes it.
In Synchronous mode, I might be able to duplicate a copy of the file before it is deleted.

Otherwise, most of the time Asynchronous mode should suffice

Let me know if you are interested and I will post an example for Asynchronous mode.

:)

Joy2DWorld
  • Members
  • 562 posts
  • Last active: Jun 30 2014 07:48 PM
  • Joined: 04 Dec 2006

how would you go about monitoring more than 1 drive?


likely there are better ways, and did not extensively test it, but seemed to work fine for me:



set up multiple callback functions, each using a DIFFERENT refrence for the directory to watch, but using all else the same.


call each of them (multiple thread creation) same as single is called now.



worked for me with 10 threads at once... no crash... etc.



likely ways to improve upon it,

note that seems many standard AHK functions/commands will cause termination of thread after execution of that one command......
Joyce Jamce

tic
  • Members
  • 1934 posts
  • Last active: May 30 2018 08:13 PM
  • Joined: 22 Apr 2007

Let me know if you are interested and I will post an example for Asynchronous mode.


I am extremely interested!

set up multiple callback functions, each using a DIFFERENT refrence for the directory to watch, but using all else the same.


I went and simplified skans original script to just show the changes as tooltips to make it easier for me to understand his code. would you be able to show me how you created N threads.

#Persistent
SetBatchLines, -1
Process, Priority,, High
OnExit, ShutApp

WatchFolder  := "C:\"
WatchSubDirs := "1"

EventString := "New File,Deleted,Modified,Renamed From,Renamed To"
StringSplit, EventArray, EventString, `,

DllCall("shlwapi\PathAddBackslashA", UInt, &Watchfolder)

CBA_ReadDir := RegisterCallback("ReadDirectoryChanges")

SizeOf_FNI := ( 64KB := 1024 * 64 )
VarSetCapacity( FILE_NOTIFY_INFORMATION, SizeOf_FNI, 0 )
PointerFNI := &FILE_NOTIFY_INFORMATION

hDir := DllCall( "CreateFile", Str  , WatchFolder, UInt, "1", UInt , "7", UInt, 0, UInt, "3", UInt, "1107296256", UInt , 0 )

Loop 
{
	nReadLen  := 0
	hThreadId := 0

	hThread   := DllCall( "CreateThread", UInt, 0, UInt, 0, UInt, CBA_ReadDir, UInt, 0, UInt,0, UIntP,hThreadId )
					   
	Loop 
	{
		If nReadLen
		{
			PointerFNI := &FILE_NOTIFY_INFORMATION

			Loop 
			{
				NextEntry   := NumGet( PointerFNI + 0  )
				Action      := NumGet( PointerFNI + 4  )
				FileNameLen := NumGet( PointerFNI + 8  )
				FileNamePtr :=       ( PointerFNI + 12 )
	
				Event := EventArray%Action%	   

				VarSetCapacity( FileNameANSI, FileNameLen )
				DllCall( "WideCharToMultiByte", UInt,0, UInt,0, UInt,FileNamePtr, UInt, FileNameLen,  Str, FileNameANSI, UInt,FileNameLen, UInt,0, UInt,0 )

				File := SubStr( FileNameANSI, 1, FileNameLen/2 )
				FullPath := WatchFolder . File
				FileGetAttrib, Attr, %FullPath%
				FormatTime, Time  , %A_Now%, HH:mm:ss

				If !FileExist(FullPath)
				Tooltip, %Time%`n%Event%`n%File%
				Else
				{
					Loop %FullPath%
					{
						FormatTime, TStamp, %A_LoopFileTimeModified%, yyyy-MM-dd  HH:mm:ss
						Tooltip, %Time%`n%Event%`n%File%`n%A_LoopFileSizeKB%`n%TStamp%`n%A_LoopFileAttrib%
					}
				}	

				If !NextEntry
				Break
				Else
				PointerFNI += NextEntry
			}
			Break
		}            
	
		Sleep 100
	}

	DllCall( "TerminateThread", UInt,hThread, UInt,0 )
	DllCall( "CloseHandle", UInt,hThread ) 
}
Return

ReadDirectoryChanges() 
{
	Global hDir,PointerFNI, Sizeof_FNI, WatchSubdirs, nReadlen
	Return DllCall( "ReadDirectoryChangesW", UInt, hDir, UInt, PointerFNI, UInt, SizeOf_FNI, UInt, WatchSubDirs, UInt, "375", UIntP, nReadLen, UInt, 0, UInt, 0 )
}

GuiClose:
ShutApp:
DllCall( "CloseHandle", UInt,hDir )
DllCall( "TerminateThread", UInt,hThread, UInt,0 )
DllCall( "CloseHandle", UInt,hThread )
ExitApp
Return


SKAN
  • Administrators
  • 9115 posts
  • Last active:
  • Joined: 26 Dec 2005

likely there are better ways, and did not extensively test it, but seemed to work fine for me:set up multiple callback functions, each using a DIFFERENT refrence for the directory to watch, but using all else the same.


I tried callback but it never worked for me. Can you post some working example. It would be very useful.

@tic: Please allow me sometime.

:)