Page 1 of 1

Need help debugging AutoHotFlow v1

Posted: 02 May 2018, 14:47
by bichlepa
This appeal for help is addressed to experienced AHK user and AHK developer.

Most of the features in AutoHotFlow v1 are implemented but I am struggling with crashes. I've spent over 30 hours to find and to fix them but can't find the reasons.
I use AHK_H because it allows real multi-threading. Possibly I can't find them because there may be bugs in AHK_H itself.

Please help me to fix the crashes! I really don't want to give up this project!

Quick start:
  • Clone the GitHub project. Use the branch "AutoHotFlow-1.0".
  • Run AutoHotFlow.ahk. It will start the actual process using the AHK-H binary, which is also in the repository.
  • Try to use it. You will see that it will sometimes freeze or crash.
I've updated the document Documentation\Code documentation.odt. You may read it to get a rough idea about the used threads.
If you need more documentation, I'll write it.
If you have hints how I could find the problems on myself, I'll try it.

Re: Need help debugging AutoHotFlow v1

Posted: 19 May 2018, 08:47
by joedf
what line does it crash on?

Re: Need help debugging AutoHotFlow v1

Posted: 19 May 2018, 09:11
by bichlepa
I have no idea on why and on which line it crashes. Sometimes it just hangs up until I kill it. Sometimes I get a message from Windows that AutoHotkey has stopped working.
Most likely it has something to do with multi-threading. Something like dead lock when using critical sections or missing mutex protection when accessing shared critical objects.

Re: Need help debugging AutoHotFlow v1

Posted: 19 May 2018, 13:36
by joedf
If you can, possible document steps to reproduce it otherwise it may be impossible to fix. You could try running it with a debugger?

Re: Need help debugging AutoHotFlow v1

Posted: 20 May 2018, 04:27
by bichlepa
There is no fix point at which it crashes. It randomly occurs while using it.
The fastest way to reproduce it, is to start AutoHotFlow, enable the flow "Huge test script" and run it several times. While it runs, you may try to open an other flow for edit. At that point it often hangs up.

I did not figure out how to use a debugger yet. I already tried to download Visual Studio and the source codes of AutoHotkey_H. With some help of HokeyIt I was able to build it, but I was not able to debug it yet.

Re: Need help debugging AutoHotFlow v1

Posted: 20 May 2018, 09:34
by joedf
Are you using scite4ahk or AhkStudio?

Re: Need help debugging AutoHotFlow v1

Posted: 20 May 2018, 09:40
by bichlepa
I use scite4ahk.

Re: Need help debugging AutoHotFlow v1  Topic is solved

Posted: 12 Sep 2020, 06:17
by bichlepa
After a long break I came up with new ideas and decided to give an other try to solve the crashes.
And ... I succeeded! :superhappy:

These three steps have led to success.
  1. Secure all shared variable access with a critical section.
    Since I use multithreading (AutoHotkey_H) I use shared variables to share information between the threads. I think, one of the problems was, that I sometimes took a sub-object of a critical object and interacted with it (for example inside for-loops). Now I wrote some functions in a single file where all access to shared variables are secured with a critical section.
  2. Do not use multiple critical sections.
    I used multiple critical sections to protect different resources. One of them, as mentioned, for access to shared variables. And I used an other for access to log files. I found out that the crashes were signitificantly rarer when I disabled the file access to logs. After I deleted the critical section for files and used insted the other critical section to protect file access, the crashes were gone.
    I think that is the reason how it caused crashes:

    Code: Select all

    Thread1
    pseudo-thread 1^--------EnterCS1-------- ← pseudo-thread gets interrupted by other pseudo-thread
    pseudo-thread 2                         --------EnterCS2 ← blocks here
    
    Thread2
    pseudo-thread 1^------EnterCS2------ ← pseudo-thread gets interrupted by other pseudo-thread
    pseudo-thread 2                       --------EnterCS1 ← blocks here
  3. On exit: prevent close threads while it is in critical section
    To achieve this, the main thread, which initiates an exit, writes a shared variable with the message to all threads that they have to exit. I wrote custom function wrapper for EnterCriticalSection() and LeaveCriticalSection:

    Code: Select all

    ; exit routine inside _EnterCriticalSection:
    ; when the application closes, the main thread informs all threads that they must stop
    ; We have to make sure, that the threads close, while not blocking the critical section (excluding main thread).
    ; to do so, each thread keeps track whether it blocks the critical section (with variable criticalSectionCounter)
    ; Whenever the is going to block the critical section, it checks whether it has to stop.
    ; If so, the thread sends a message to main thread that it can now be killed and at same time stops the pseudo ahk thread
    _EnterCriticalSection()
    {
    	global criticalSectionCounter
    	global API_Main_Thread_Stopped_sent
    	
    	; if below exit code already executed, do not enter critical section
    	if API_Main_Thread_Stopped_sent
    	{
    		exit  ; stop the pseudo-ahk-thread
    	}
    
    	EnterCriticalSection(_cs_shared)
    	if (criticalSectionCounter == 0 and (_share.exiting or _exiting) and not _ahkThreadID="Main")
    	{
    		; send the message to main only once
    		if not API_Main_Thread_Stopped_sent
    		{
    			API_Main_Thread_Stopped_sent := true
    			API_Main_Thread_Stopped(_ahkThreadID)
    		}
    
    		; Leave critical section 
    		LeaveCriticalSection(_cs_shared)
    		exit ; stop the pseudo-ahk-thread
    	}
    
    	criticalSectionCounter++
    }
    
    _LeaveCriticalSection()
    {
    	global criticalSectionCounter
    	criticalSectionCounter--
    	
    	LeaveCriticalSection(_cs_shared)
    }
Now I can continue my work on AutoHotFlow! :dance: