AutoHotkey Community

Hi AHKers!
Here's another app for teachers of students with disabilities (autism in particular) I've made. This app hopefully replaces unacceptably expensive equipment that turn text and images to speech/sound to help someone communicate.
For example ($7895!!!):https://store.prentrom.com/product_info.php/cPath/11/products_id/53


It is designed for the large amount of (cheap) tablet pc's which will hopefully be out before christmas (here in Australia at least). Basically you assign sounds/speech to images you've chosen and then touch them to play the sound or speak the text.

Other communicators aren't fully configurable like this is, it is able to be used with a vast variety of ability levels and so can be used to scaffold a long term learning experience.

Here's a live demonstration of the program using the inbuilt 5 button demo
Demonstration video

What I need:
-Testers with tablets/touchscreens to see if everything works! {I don't have one yet } I'm particularly curious about the performance of the text to speech functions.
-Someone experienced wanting to try to get this running with ironahk under linux and mac
-My code is "latenight logic" code - if anyone cares to improve it, be my guest
-If someone has a tablet and can actually use this with someone who needs it please give feedback (I'll do my own testing after christmas


Thanks:
Go to the forum for all the code I've borrowed and learned from

Here are some screenshots of the program in use

You can choose from a wide variety of button arrangements



Here are the arrangement options for 5 buttons



Click on an image-button to set it up




Once you choose a picture to use as a button you can choose
1.Text 2 speech
2.Play a sound
3.Record a new sound
4.Nothing (to be used as distractions)

An example of how you COULD choose to set it up - I have chosen to use text2speech with each picture so a user could communicate, e.g. "Apples please" (the auslan symbols are for "milk" and "drink")

You can also save any setup you make


IMPORTANT!:
To escape at anytime you can press alt and q together
To exit during usage you can either press the exit button (if available) or follow the instructions and press the red triangles in the corners of the screen as quickly as you can before they reappear.


here's the download link for the app The Open Communicator 2.0

Installer
The Open Communicator 2.0 Installer

_________________
Inventing problems that need solutions...

Open Communicator
MouseTrainer

Wow, Evan, I think it's great that you're using AHK to help people with special needs. I'd offer to test, but I don't have a tablet or touchscreen. I doubt that many AHKers do, but there's a website called reddit that I bet could be used to round up quite a few testers, especially if you commit to keeping it open source. They like helping out, may already know about AHK, and are very tech/gadget savvy.

Hi Joe! Thanks for your encouragement, I've had a little look at reddit and the general vibe I get doesn't motivate me to be part of it, not sure why tho. I have alot of respect for the AHK forum (especially since they'll be able to help out with ahk solutions more expertly) and I'll hope for a little bit longer that some people give it a go. If it ends up nowhere I'll give reddit another look, thanks mate.

Evs

Sounds good. You probably know what's best for your project. I'm a member of the site and would be happy to post for you, so drop me a PM if you want me to. I'd bet that they get 100-1000 times as many pageviews as the AHK forums (owned by Conde Nast) and many of them pride themselves on helping out worthy causes (there's lots of trolls, too), so if you want someone with the right hardware, I feel like you'd have great luck there. You also might like it more if you visited one of the smaller subredits--for example the education subreddit:
http://www.reddit.com/r/education/

And, yeah, the AHK community is pretty cool, too...

Morning evandevon (or maybe not if you're in Australia)

I've downloaded the Open Communicator and popped it onto my Toughbook which has touchscreen and can be transformed into a tablet.

I've only had a brief go with it so far and everything seems to be working.

I do have a couple of observations though. Selecting a file with the standard windows explorer is quite difficult when using the touchscreen. Maybe some sort of gallery where you can choose which picture to display would be good?
The text to speech had quite a delay on it but I haven't had a chance to test it properly as I'm in the office at the moment and was getting some funny looks.
Also the red triangles in the corner gave me some problems. I couldn't click them for the life of me. I'm sure there's a technique I'm missing but it wasn't immediately obvious.

As I say this was only a brief test. I'll have a more in depth play with it later on and report back. Is there anything in particular you'd like testing or shall I just try everything out?

Hello again,

I've done a bit more testing.

I still can't get the red corner thing to work. Don't know what I'm doing but the only way I can exit is with alt+q (just figured out that i need to get rid of all the corners to exit. That makes sense now and works well.) The two red triangles in the top corners have white lines underneath them as though they are underlined. Is this intentional?

When choosing that a button should have text-to-speech associated I repeatedly type the word into the box on the right only to have to type again on the software keyboard when I choose the computer speech radio box. I'd suggest making the preview edit box read-only or default to using the hardware keyboard for that edit box and have another button with which you can access the software keyboard

I also noticed that the text input at the bottom of the 15 button configuration won't let me input words correctly. I tried typing 'Hello' but the cursor keeps skipping back to the beginning so 'Hello' ends up as 'ollHe' (which the text-to-voice dutifully reads).

That's all I've got for now. I'll continue to test and report back.

Hi Sam! Thanks so much for playing with it for me!
Thanks Jo, I'll give reddit another look after working through the bugs.

The red triangles at the top must be slightly moving, they are 4 images within 4 gui's with transparency and if the image shifts inside the gui at all it tends to leave thin lines on the borders - I'll fix it.

That gallery idea is a good one, I was hoping that tablet pc's would somehow detect when in tablet mode and make it easier to use with fingers - but since that doesn't seem to happen I'll put one in. Is the difficulty mainly that the images aren't shown in thumbnails? Or is it navigating to the files?

Since getting rid of all the corners wasn't clear can you suggest how I could change the instructions/procedure to make it more intuitive? I could change the exiting procedure but it has to be difficult for kids using it to accidentally start.

I used to have a button that loaded the on screen keyboard but I removed it since it seemed that (the one user I tested with) somehow couldn't see it amongst the options so I made it auto launch. I'll put it back in and make the editbox auto-select the asociated radio button so you don't have to select it.

I'm assuming your using the hardware keyboard to type in the text in the 15 button display? I think I know the problem but give it a go following the steps below:
The button configs with sentence-2-speech boxes at the bottom (8/10/15/alphabet) should work like this.
1.User presses a picture
2.The FILENAME of the picture gets printed and any actions get performed
3.Other pictures are pressed until sentence is built
4.User presses the speak button which speaks the sentence. Then the sentence box is cleared.

Txt2speech performance: What are the specs of ur com? How's the audio file play performance? I found an app that does txt to .wav and I could use soundplay so maybe I should use that so that com specs aren't an issue?

I'm imagining the typical hardware for this to run on is going to be purely touch screen so keep this in mind when using it.

Thanks so much for testing this for me! I really appreciate it!
Evs

Hey Evs.

I can give another thumbs up for Reddit. I lurk there quite a bit and the community is, for the most part, made up of good people who I'm sure would be able to help.

The red corner idea is a good one. As you say it needs to be difficult to do by accident. The only way I can think to make it more obvious is to have an animated finger on the instruction rather than one in each corner. Also I find it hard to get right into the corners on my Toughbook so normally end up using the touch-pad mouse to exit so I'd request that they be made a bit bigger too.

It may be that other tablets do detect when they are in tablet mode but I think a gallery would look nicer anyway. What is happening on the Toughbook is that the normal FileSeclectFile explorer is opening. It is possible to put it into Thumbnail mode but with my chunky fingers it's hard to do anything on an interface that small.
In my minds eye I can see a simple Gui with a dropdown list to choose the image category, an image holder to display and a slider along the bottom to navigate. It would fit more with the feel of the program.

The problem I was having with the text box makes sense. I was using the Hardware keyboard which it isn't designed for. I've tried it out in the way you suggested and it works well. The only trouble I encountered is that when using 15 buttons it will quite often put the filename into the speech box twice. I've only noticed this with the 15 button option but it does happen quite a lot.

The Txt2Speech performance is generally good. There are a few things (like the number 5) which sound a bit odd but with some creative spelling I can usually get it sounding OK.
The responsiveness is good. This is the Computer I'm testing on Panasonic Toughbook CF-19 (it is a nice perk of my job). I'd be interested to see what difference the txt to .wav generator makes.

Sam

PS. worth noting that my toughbook has XP installed and not the advertised Windows 7.

Hi sam!
I've uploaded a newer version back at the original post.
Changes:
- Added intro splash screen
- Allow the user to use a keyboard to type in text2speech text more easily
- animated the exit instructions with slightly longer duration
- made the exit corners bigger by %50 and should have fixed the white lines issue
- added the demo function (first gui) for reliability of file locations (just choose it and it should load)
- fixed a few bugs that caused the program to freeze


Coming Soon:
-A tablet [fat finger;)] friendly gallery for an easier image selection process
-txt2wav for low spec tablets to use soundplay

Let me know how the changes make it better/worse. Thanks again for all your feedback!
Evs

_________________
Inventing problems that need solutions...

Open Communicator
MouseTrainer

Will it work with a wacom graphics tablet?

_________________
KeyMce/GenMce - mackie emulator for pc keyboard/Convert your controller to mackie.
Midi I/O - Want to play with midi/ahk?

I don't see why not, they seem to be able to work with anything, though i do not have one to test with. They seem pretty damn cool tho!

Also added a demonstration video to the top post

evs

_________________
Inventing problems that need solutions...

Open Communicator
MouseTrainer

Hi Evan,

Sorry about the delay in replying. I've had a couple of deadlines to meet and wasn't able to devote enough time to a proper test the Open Communicator. I've done that now though and would agree that it seems more stable.

The animation to indicate how to exit using the red triangles is spot on. Much easier to understand. The bigger red corners are good for me too although I'm still getting the white line underneath the top two.

I would suggest that users be allowed to exit the Credits and Exit instructions splash screens with a click as I could foresee getting impatient waiting for them to load after I'd been using it a while.

The Demo Screen works really well and shows off all the functions available.

I look forward to testing the Chunky Digit Gallery.

Sam

Hi Sam!
No worries! I'm in the middle of renovations and haven't had much spare time either

I've played a bit with the main program but have only really been working on the image gallery program. The only main program change is that now you can exit the exit instructions with a click anywhere.

re: The white lines - What screen resolution are you using? Mine seems to be fine on all resolutions available. Don't know what else it could be.

Here's the link for the Image Gallery
Forum link
I put it into it's own thread since I guess others might be interested in it but not the main program and so may never see it if it was here.
Eventually I'll add an option to copy and paste files and create new folders but this is really just a proof of concept. let me know how it works by finger. Probably best to post your feedback there rather than here for that particular part of the program unless it has more to do with it as part of a whole.

Thanks so much again for making the time to test this out! If you ever need a hand with a world domination attempt I'm your man!
Evs

_________________
Inventing problems that need solutions...

Open Communicator
MouseTrainer

Update: 16/02/2011
Update to 2.0 (link at end of original post)
Fixed a few bugs, Now you can record a new sound using MS recorder and save it with only using the onscreen keyboard/controls. Earlier an actual keyboard was required to type the file name.

Future 2.1 Release
A fast, scrollable and touch based image and sound gallery. A whole project on it's own really. The basics are there in the previous post if anyone is interested in looking.

Evs

_________________
Inventing problems that need solutions...

Open Communicator
MouseTrainer