Jump to content


Photo

Autohotkey with preffered edition Dragon Naturally speaking


  • Please log in to reply
39 replies to this topic

#1 Q

Q
  • Members
  • 24 posts

Posted 21 February 2008 - 09:31 PM

Hi,

The professional version of Dragon Naturally speaking (DNS) is the only version that offers linking macros with proper voice command phrases.
I mean launching macros with meaningful spoken phrases like "go to my project folder 1", not just hot key presses like "Press Alt Alpha".

But Boo Hoo Woo it's like around 400 to 500 pritish pounds :-(
The preferred edition is however a fraction the cost at around 70 pounds. With the preferred edition you can only however paste text and graphics, there is no macro support.

Whey hey, I have however figured out how using Autohotkey to provide free and fast macro support for the Preferred edition and I have tried it on version 9.5 on vista (NOTE my method won't work for the standard edition).

Basically it relies on scrapping the results box and then firing off the respective macro. So in tools options view of DNS you need to set the results box so that it is always displayed.

Instructions:-

1) In tools options view of DNS you need to set the results box so that it is always displayed.

2)For each macro you want to set up. In the DNS command browser enter a new text command with the name as the phrase you want to launch your macro with and enter the ¬ character only in the content box.
You have to enter the ¬ character as DNS will not let you save it with no characters in it. The ¬ will be sucked out by the secound script on running the macro.

3) After DNS is running run the following two scrips. Note there are just two macro examples in the first script named "click top left" and "click some where else", you will want to add more. Be sure in step 2) above to have added in the commands "click top left" and "click some where else".
The label that will be jumped to run the macro in the first script is shown in a tool tip at the top of the screen. The secound script monitors for the ¬ character and strips this out.

4) If it fails to work try reloading the first script when the DNS results box is showing.

5) This does work well and is quick but please post any improvements here or better Ideas to link autohotkey to the non-professional versions of dragon naturally speaking.

6) Please let me know here how you get on

Note: at time of writting (correct me if I am wrong) I think that DNS does not run on the 64bit vista version just on the 32bit one. Most computers sold with vista however use the 32bit version even if the underlying chips are 64bit.

SCRIPT 1:-


#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
CoordMode, ToolTip, Screen

WinGet, active_id, ID, A

Loop,
{

; This will visit all windows on the entire system 
WinGet, id, list,,, Program Manager

  Loop, %id%
  {
      this_id := id%A_Index%
      WinGetClass, this_class, ahk_id %this_id%
      WinGetTitle, this_title, ahk_id %this_id%
       len1:=StrLen(this_title)
      StrA:= Substr(this_title, 1, len1)

      if  (this_class="#32770" and StrA<>"DragonBar"){
           Loop,
          {

             WinGetTitle, this_title, ahk_id %this_id%
              len1:=StrLen(this_title)
              StrB:= Substr(this_title, 1, len1)
              StringLower, StrB, StrB
	      Endchar:=SubStr(StrB, len1, 1)	
              if (Endchar=" " and len1>1){
                  len1:=len1 - 1
                  StrB:= Substr(StrB, 1, len1)
                  }
              StringSplit, StrArray, StrB, %A_Space%, %A_Space% 
              StringReplace, StrB, StrB, %A_SPACE%,, All
              StringReplace, StrB, StrB, ...,, All
	     
              if (strlast<>StrB){
                   strlast:=StrB
                     if (StrLen(StrB)<>0){
                       char1:=SubStr(StrB, 1, 1)
                       if (char1>="a" and char1<="z"){
			 if (IsLabel(StrB)){
		             gosub %StrB%     
                             ToolTip, %StrB% (custom), 600, 5
			     } else {
                             ToolTip, %StrB%, 600, 5
                             }
                        }
                     }
           
              }
            
              
              Sleep, 20
          }
      }
  }

Sleep, 5000
}

clicktopleft:
;-----------------------------------------------------------------------
CoordMode Pixel, screen
CoordMode, Mouse, Screen
MouseMove, 0, 0, 0
MouseClick, left

return

clicksomewhereelse:
;-----------------------------------------------------------------------
CoordMode Pixel, screen
CoordMode, Mouse, Screen
MouseMove, 100, 200, 0
MouseClick, left

return





SCRIPT2 put this in a seperate script and run seperately
#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.

#NoTrayIcon


loop,
 {

;clipWait
sleep, 20

char1:=SubStr(clipboard, 1, 1) 
len1:=StrLen(clipboard)

if (char1="¬" and len1>1){
   str1:=SubStr(clipboard, 2, len1-1)
  clipboard = %str1%
}
if (char1="¬" and len1=1){
  clipboard =
}

}


#2 porky

porky
  • Guests

Posted 14 March 2008 - 10:21 AM

Q,

Your ideas are exactly what I'm looking for. I am a new dragon preferred user and am immediately frustrated by the lack of support for some of the more basic navigation. I have been looking at the know Brainer software to supplement Dragon but even this is $200!

I'll spend sometime this weekend trying your suggestions; have you had any other interest or progress.

Regards,

Porky

#3 Guests

  • Guests

Posted 26 March 2008 - 03:51 PM

I tried to do something like this myself -nice job! AHK is my weapon of choice (I also have to say that many of the voice rec communities are much more contentious than this forum), but you guys should also keep Vocola (which interfaces with a Python module) in mind - very similar capabilities to Knowbrainer (but free!), very easy to build macros. I was able to do most of what I needed with it. Unimacro is also free and uses Python, and is much more powerful, but also much more complicated.

#4 KnowBrainer

KnowBrainer
  • Guests

Posted 17 April 2008 - 04:00 PM

Q,

Your ideas are exactly what I'm looking for. I am a new dragon preferred user and am immediately frustrated by the lack of support for some of the more basic navigation. I have been looking at the know Brainer software to supplement Dragon but even this is $200!

I'll spend sometime this weekend trying your suggestions; have you had any other interest or progress.

Regards,

Porky


Before you abandon KnowBrainer 2007 completely, you might be interested to know that KnowBrainer is now freely available to the physically disabled who cannot otherwise afford it. If you'd like to know more about KnowBrainer or qualify for a free copy read our What is KnowBrainer

Lunis Orcutt - Developer of KnowBrainer &
Host of the Http://www.KnowBrainer.com Speech Recognition Forum

#5 EdScriptNewbie

EdScriptNewbie
  • Members
  • 117 posts

Posted 18 April 2008 - 05:38 PM

THANKS for creating this.

I have a zillion hotkeys, hotstrings etc in my AutoHotKey.ini. I don't want to copy the ones I need into SCRIPT 1 b/c I'm always improving them.
1. What do I put in SCRIPT 1 to call a hotkey or hotstring from AutoHotKey.ini? It's always on, but somehow DNS Preferred blocks it.

Also, yes, I too wish for a DNS forum with the helpful participants, kindly tone and easy searching of this superlative forum, but the ones I've tried seem more consistent with Nuance's far-less-than-helpful tone, plus contentious as mentioned.
2. Can anyone point to a good DNS forum?

THANKS AGAIN

#6 Hasso

Hasso
  • Members
  • 324 posts

Posted 21 May 2008 - 04:17 PM

I only know a good one in German, maybe it's useful for someone: http://forum.oasa-sp...ewforum.php?f=3.

#7 Volante

Volante
  • Guests

Posted 01 June 2008 - 02:57 AM

I put together a simple script to use Dragon Naturally Speaking to drive voice commands using a simple input Dialog box that auto executes after four seconds.

I have tied this function to my mouse-wheel button or to a "press tilde" command, if working with no mouse. The script can then be edited to basically match any text generated by voice as a command. Since the dialog box is user activated, it can be controlled from any context, and you can determine in advance whether you want the command to execute in the current application or be more explicit about executing specific actions.

; Voice Commands - Lets you control your computer using DNS
MButton::
~::
    InputBox, command, Voice Command,,,300,100,0,0,,4

    ; Command 1
    if command = text of your command
    {
        code you want to execute
    }

    ;Command 2
    if command = some other text command
    {
        code you want to execute
    }

return

Basically, if you mess up and there is no match to your voice command, the system does nothing because there is no match. Simple.. but works well.

#8 Kmarvin

Kmarvin
  • Members
  • 3 posts

Posted 13 October 2008 - 01:47 AM

Nice work-arounds for DNS preffered's lack off support for real commands or even text substitutions that send keystrokes. When I was stumped by this issue, I found Natpython & Vocola here: http://speechwiki.org/NL/HomePage.html. These, respectively, use the NatLink utility of DNS to accept python scripts and convert DNS (v3) scripting language to python.

Keystroke commands are easy to write and they CAN call hotstrings/hotkeys in autohotkey. Vocola also supports variables, captured words and alternatives. Moreover, the DNS script command AppBringUp can pass parameters to autohotkey.

I select tools in a toolbar (which DNS8 does not support) with the following code:

Icon_finder.ahk
; This script searches the entire screen or a limited area of the screen for an icon in a supported graphic format
; It is designed to be called by another program as required using a command line shell type call 
; -- AppBringUp in the Vocola-NatLink implementation of the Naturally Speaking script language 
;
; eg. AppBringUp(icon,"AutoHotkey /icon_finder.ahk param1 param2 param3 param4 param5 param6",6,dir) for a full-screen search or
; AppBringUp(icon,"AutoHotkey /icon_finder.ahk param1 param2 param3 param4 param5 param6 param7 param8 param9 param10 param11",6,dir)
; for a limited area search
;
; C: is prefixed to /icon_finder.ahk when this call is made, I don't know which program does it.
; Icon and Param1 are the name of the icon file.
; The quoted part is the command line parameter in AppBringUp (quotes required).
; The next number specifies the state for the window (6 is minimized/inactive).
; Finally, the call has a working directoy parameter, dir. This permits simplification of param1 because A_WorkingDir
; below then contains the path to the icon file (a path passed in param1 throws vocola for a loop).
;
; The the autohotkey script reads the command line parameters param1-param11 as follows:
; In the command line parameter a param2 = "L" means the next four parameters specify the upper left and lower right coordinates
; of the area to be searched as fractions of the full-screen width and height. If L is not set a full-screen search is accomplished
; (fractional coordinates must not be sent either in this case).
; The mouse is offset from the upper left corner of the icon the next two parameters and clicked by following one (0-2 clicks may be
; sent). The remaining parameters get the mouse off the icon (toward the upper left, if negative or lower right if positive), and
; allow for color mismatch in the icon (0-255), respectively.

#NoTrayIcon

LimitArea = %2%
X2 := (A_ScreenWidth)
Y2 := (A_ScreenHeight)

IfNotInString, LimitArea, S
{
	WinGetPos, X, Y, Width, Height, A
	X2 := width
	Y2 := height
}
else
{
	CoordMode, Pixel, Screen
	CoordMode, Mouse, Screen
	LimitArea = "L"
}
IfInString, LimitArea, L
{
	c1 = %3%
	c2 = %4%
	c3 = %5%
	c4 = %6%
	X1 := (X2*c1)
	Y1 := (Y2*c2)
	X2 := (X2*c3)
	Y2 := (Y2*c4) 
	ShiftX = %7%
	ShiftY = %8%
	Clicks = %9%
	ShiftBack = %10%
	MatchColor = %11%
}
else
{	X1 := 0
	Y1 := 0
	ShiftX = %2%
	ShiftY = %3%
	Clicks = %4%
	ShiftBack = %5%
	MatchColor = %6%
}

ImageSearch, FoundX, FoundY, %X1%,%Y1%,%X2%,%Y2%, *%MatchColor% %A_WorkingDir%\%1%

if ErrorLevel = 2
	MsgBox  Could not conduct the search for %A_WorkingDir%\%1%.
else if ErrorLevel = 1
	MsgBox Icon %A_WorkingDir%\%1% could not be found on the screen.
else
{	MouseMove FoundX, FoundY
	Click %ShiftX%, %ShiftY%, %Clicks%, relative
	If ShiftBack != 0
		MouseMove (FoundX+ShiftBack), (FoundY+ShiftBack)
}
return

The vocola code that calls this looks like this:
# Icon finder calls for icon based clicking

# IconFinder opens the process from DNS as "icon" in a minimized
# window ("6"), passes the ahk commandline ($i), and path to the icon
# ($p). CallLimitedIconFinder assembles the command line
# so that white space doesn't foul the AppBringUp call. The voice
# commands are "your icon" and "another tool" defined
# as bit maps "your_icon" and "another_tool".

IconFinder (i,p) :=
	AppBringUp(icon, $i, 6, $p);
	
CallLimitedIconFinder (icon, flag, yupper, xleft, ylower, xright, xshift, yshift, clicks, offset, match, path) :=
	IconFinder("AutoHotKey /icon_finder.ahk "$icon".bmp "$flag" "$yupper" "$xleft" "$ylower" "$xright" "$xshift" "$yshift" "$clicks" "$offset" "$match, $path);

(your icon = your_icon | another tool = another_tool) = CallLimitedIconFinder($1, S, 0.75, 0.75, 1, 1, 5, 5, 0, 0, 50,"C:\My Scripts\icons");

Icon bitmaps can be cut from screens captured using CaptureScreen (Sean's script http://www.autohotke...topic18146.html using paint. This is automated somewhat with this code in front of the CaptureScreen code:
;
; Script Function:
;  Capture icons for later selection by icon based search 	
;

#NoEnv ; Recommended for performance and compatibility with future AutoHotkey releases.
#NoTrayIcon 
SendMode Input ; Recommended for new scripts due to its superior speed and reliability.

; Interface script to permit Dragon Naturally Speaking via NatPython/Vocola to issue a call to CaptureScreen() to capture an icon
; to select with ImageSearch in subsequent calls to the IconFinder script

CoordMode, Mouse, Screen
MouseGetPos, xpos, ypos
xshift = %1%
yshift = %2%  
PosX1 := xpos - xshift/2
PosY1 := ypos - yshift/2
PosX2 := xpos + xshift/2
PosY2 := ypos + yshift/2
C1 = %PosX1%,%PosY1%,%PosX2%,%PosY2%
IconArchive = %A_WorkingDir%\%3%
MouseMove, 0, 0
CaptureScreen(c1, false, IconArchive)
MouseMove xpos, ypos
SplashImage, %IconArchive%, H150 W400,,This is how it looks!
FileSelectFile, SelectedFile, S, temp, Save As, bitmaps (*.bmp)
If SelectedFile =
	SplashImage, Off
Else
{
	FileCopy, %IconArchive%, %SelectedFile%.bmp, 1
SplashImage, Off
}
Return

; CaptureScreen goes here

Then call from vocola:
# Capture a bit map around the mouse pointer for use with IconFinder.ahk

<n> := 0..99;# n is a variable values 1-99
capture (a) <n> by <n> icon  = AppBringUp(IconCapture, "AutoHotKey /IconCapture.ahk $2 $3 temp.bmp", 6, "C:\My Scripts\icons"); # save temporay n by n bmp in specified path

edit the last icon = AppBringUp(Paint,"\windows\system32\mspaint temp.bmp",1,"C:\My Scripts\icons");# shows the bmp in paint

edit an icon = AppBringUp(Paint,"\windows\system32\mspaint temp.bmp",1,"C:\My Scripts\icons")
	SendKeys {alt+f}o;# as above but opens "open" dialog too

The above approach allows me to run software with a very highly graphical interface by voice. A little cross pollination of autohotkey and vocola-natpython seems to help. See what you think.

#9 Kmarvin

Kmarvin
  • Members
  • 3 posts

Posted 13 October 2008 - 09:17 PM

Sorry the following link is more useful for natpython and vocola as the project has been picked up by new people!http://www.speechcom...ng.com/forum/52

#10 EdScriptNewbie

EdScriptNewbie
  • Members
  • 117 posts

Posted 14 October 2008 - 11:44 PM

Keith, thanks for posting what appears to be some very interesting work. I use DNS (preferred) a lot, and I use ahk, but I haven't been able to make them get along. I know nothing of python, etc, so your post is Way over my head, technology-wise. if I were going to just try something out along these lines, where might I start? Thanks. ...Ed

#11 nekto

nekto
  • Guests

Posted 17 October 2008 - 10:39 AM

http://www.speechcom...g.com/node/1773

Here is a script that make Vocola to execute hotkeys when you say code phrase. So I guess something as simple as

fire my first script = "{Alt+j}" ;
fire my second script = "{ctrl+l}" ;

should work as long as you define scripts for Alt+J and Ctrl+L in AHK.

Under linux atm, could not check.

#12 EdScriptNewbie

EdScriptNewbie
  • Members
  • 117 posts

Posted 17 October 2008 - 01:25 PM

so simple. thanks!

#13 nekto

nekto
  • Guests

Posted 17 October 2008 - 04:52 PM

My pleasure :)

Just in case installer is here: http://sourceforge.n...ojects/natlink/

Download section, first you install second package (pythonfornatlink), after that you should install first one (natlink).

Vocola files you need to modify: http://vocola.net/UsingVocola.html
Or simply locate and edit "in_vocola.vcl".

Good Luck! 8)

#14 Q

Q
  • Members
  • 24 posts

Posted 17 October 2008 - 07:42 PM

Hi there,

Thanks for the info about natlink and vocola, does the latest verison 3.5 work with all versions of DNS? I mean DNS Versions 8, 9 and 10?

The example above gives a way to press hotkeys to run AHK.

Is there a vocola command to run a file so you could simply just run the .AHK file?

#15 Kmarvin

Kmarvin
  • Members
  • 3 posts

Posted 05 November 2008 - 04:03 AM

Hi,
The command you want in vocola (DNS scripting language) is:

speak this phrase = AppBringUp(script name,"C:\my scripts\script.ahk",n,WorkingDir);

where "script name" is what you want DNS to know your script as while it is running (so DNS can "Switch to ..." it by name), the path to your script and its name calls ahk and runs the script. (My previous post had autohotkey explicitly in the second parameter along with the script path and name as a command line parameter. That was more complicated than necessary.)

Parameters n and WorkingDir can just be omitted; n is a single digit -- 1 = normal active, 2= minimized active, 3= maximized active, 4 = normal inactive, 6 = minimized inactive; "WorkingDir" sets a new working directory.

The present version Vocola/NatPython (3.5) works with DNS8.1. Reports on the speechcomputing forum noted previously indicate it also works w/ DNS9.5, but some problems were noted with DNS10.

I had a little bit of a go with Q's code. The following modified code id's the results box only once, eliminating the continiuous loops. It also Sends keystrokes (with modifier keys) to the active window. So in DNS Command Editor write a custom command with content ":::!f" and it will click the file menu when said (no new ahk subroutine required). Define "move results box" with content ":::" and it will put the results box at the top of the screen. Define your own commands as before but use ::: as content (::: can be said). (Note that DNS "Dictation Box" also uses the clipboard, so starting with ::: will cause modifier keystrokes to be sent as modifiers). Hope you don't mind Q, but your code makes low overhead for simple commands.

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
CoordMode, ToolTip, Screen
#Persistent

loop
{	
	this_id := WinExist("... ahk_class #32770","","Dragon")
	If this_id > 0 
	break
}

OnClipBoardChange:
;WinGet, this_id, ID, ahk_class #32770,, Dragon
WinGetTitle, this_title, ahk_id %this_id%

	; strip leading placeholder from clipboard
		charLPH:=SubStr(clipboard, 1, 3)
		len1:=StrLen(clipboard)
		if (charLPH=":::" and len1>3)
		{
			str1:=SubStr(clipboard, 4, len1-3)
			clipboard = 
			Send %str1%

		}
		if (charLPH=":::" and len1=3)
		{
			clipboard =
		}
		else 
		{
		ToolTip, %SubName%, 900, 5
		SubName = %this_title%
		strlast:=""
		return
		}
	; extract the command from the tool tip, call the subroutine, and ToolTip the type
		len1:=StrLen(this_title)
		StrB:= Substr(this_title, 1, len1)
		StringLower, StrB, StrB
		Endchar:=SubStr(StrB, len1, 1)   
		if (Endchar=" " and len1>1)
		{
			len1:=len1 - 1
			StrB:= Substr(StrB, 1, len1)
		}
		StringSplit, StrArray, StrB, %A_Space%, %A_Space%
		StringReplace, StrB, StrB, %A_SPACE%,, All
		StringReplace, StrB, StrB, ...,, All
        
		if (strlast<>StrB)
		{
			strlast:=StrB
			if (StrLen(StrB)<>0)
			{
				char1:=SubStr(StrB, 1, 1)
				if (char1>="a" and char1<="z")
				{
					if (IsLabel(StrB))
					{
						gosub %StrB%
						SubName = %StrB% (Custom)     
						;ToolTip, %StrB% (custom), 900, 5
					} 
					else
					{
					SubName = %this_title%
					;ToolTip, please write %this_title%, 900, 5
					}
                        	}
			}
		}
return

moveresultsbox:
;-----------------------------------------------------------------------
WinMove, ahk_id %this_id%,, 600, 5

return

clicktopleft:
;-----------------------------------------------------------------------
CoordMode Pixel, screen
CoordMode, Mouse, Screen
MouseMove, 0, 0, 0
MouseClick, left

return

clicksomewhereelse:
;-----------------------------------------------------------------------
CoordMode Pixel, screen
CoordMode, Mouse, Screen
MouseMove, 100, 200, 0
MouseClick, left

return

God speed,