AutoHotkey Community

[Laszlo]

I keep struggling with Unicode character entry. Some applications, like PowerPoint of MS Office XP, Win2K Notepad or all the ones with standard Windows edit controls can handle Unicode well, but the Alt-Numpad method does not work to enter the characters. Alt-945 should produce α, but these applications show an ANSI character as if only the last byte of the character code was typed. The same happens with Send {ASC 945} from AutoHotKey.

It is especially painful in PowerPoint, where the nice characters could really spice up your presentations. The Insert / Symbol method is very slow.

There should be a way to send Unicode to PowerPoint and Notepad (without the complicated way of pasting it from the Clipboard or from the symbol table). Some keyboard drivers do it (like Keyman). Does anyone know, how?

[Chris]

If you haven't already, try "Transform Unicode". It allows a script to put Unicode characters directly onto the clipboard.

[Laszlo]

This is what I wanted to avoid. Since we cannot save/restore graphic clipboards, the sidefects are serious.

[Laszlo]

The AutoHotKey sample code to get the Unicode clipboard converted to an ASCI string:

[Laszlo]

The following works with Notepad and WordPad

[Chris]

[Laszlo]

I don't think a Unicode version of AutoHotKey is important, but extending the Send command to handle {ASC 945} in Unicode applications is.

[jonny]

[BoBo]

[Laszlo]

Hey, I just work here! You could have guessed that I am Hungarian, why should I otherwise bother to type letters like Ő. Have you seen a native American doing this? (Actually, I have. Knuth made TeX capable of handling diacritical letters, like the á and ő in the name Pál Erdős, for proper bibliographic references.)

Once I tried the German version of the MS Word macro language (Visual Basic). The keywords, commands were in German. It was just horrible. You had to remember two sets of reserved words and their shorthands and spellings, because other versions of Visual Basic still required English for their reserved words. You have to learn the programming language anyway, so it makes not much of a difference if you type your program with English commands or in your native language. (English is even easier than writing a program, e.g. in Chinese. I guess.)

But the input/output of the program you write must be able to handle foreign languages. That is, the strings have to be allowed to contain all the funny characters, maybe in the form of my favorite {ASC 945} and the commands Input, Send… should process them correctly. This is all what I am asking for.

[Laszlo]

After performing some experiments with the clipboard-workaround for pasting Unicode characters, I start to understand, that Unicoding is a hard problem. Some of the difficulties:
- Some Unicode characters are double width, so the insertion point has to move accordingly.
- Others are half width.
Entering them and then moving the insertion point to the left causes really funny behaviors of the editors (Notepad, WordPad, MS Word). Sometimes the caret moves two positions, sometimes it lands in the middle of the double-width character, etc.
- Some Unicode characters are meant for writing from right to left. The editors try to be smart, so the insertion point is left to the left of them.
- Some characters are interpreted by the editors as a command, and menus start to drop down, windows change, etc. With a simple loop of sending characters to WordPad I managed to crash Windows, corrupt open files, close applications without saving unfinished work, and so on.

It looks like the technology is not mature enough. At least in Windows. Cut-and-pasting characters using the Windows standard tools should not cause havoc to the OS. Still, diacritical letters and math symbols look safe, so Chris, let us have our fun with easier sending them from a script. It is so entertaining to see Windows go bazark…

[Chris]

[Laszlo]

Well, for XP Microsoft says the following:
http://www.microsoft.com/globaldev/DrIntl/columns/019/default.mspx

[Chris]

Thanks for the info. I think TextOut would not work because it simply draws the text you specify -- it doesn't actually send any characters anywhere, nor would it be appropriate to use it to draw text in a window belonging to some other process, such as Notepad.

[Laszlo]

I found a workaround for the Unicode-to-PowerPoint problem. It is not for the fainthearted, so those of you in this category stop reading right now. For the rest of you, fasten your seatbelts…

The Send command simulates keystrokes or sends ANSI characters to the active window. If an application does not support the ALT-NumKeys method, or some other keyboard-only way for entering Unicode characters, we are out of luck. The solution is to let Windows itself generate those Unicode characters and send them to the active application. Since AHK can control keystrokes, defining a special keyboard layout, with all the Unicode characters needed will solve our problems. AHK scripts will send the right keyboard entries and the keyboard driver sends the corresponding Unicode characters to the application.

The good news is that Microsoft provides the right tools for free (for Win2K, XP and Windows Server 2003), the Microsoft Keyboard Layout Creator (MSKLC):
http://www.microsoft.com/globaldev/tools/msklc.mspx

You start with loading your current keyboard layout into MSKLC. Find a key, which you can designate to be a "dead key". It means, it does not produce an output until one more key is pressed. If this key is defined to modify itself such a way, that it gives the original output, all what you loose is that this key has to be pressed twice to get its original character.

Having this dead key, we can assign as many Unicode characters to (modified) keys as we want or run out of keys. If that happens, we have to define a second dead key, too.

If you only need a few Unicode characters, this will be sufficient. Under the MSKLC menu item "Project" select "Build DLL and Setup Package". It creates and installable package, the keyboard name with .msi extension. Double clicking on it installs the new keyboard driver, what you can use after selecting it in the Control Panel / Keyboard option. The new keyboard happily sends the Unicode characters to all application ever after. (Some of them don't know, what to do with Unicode, so be careful.)

However, this method requires the user to remember which key is modified to give what Unicode character. If you misfire, you have to backspace and try it again. This is why I prefer the post modifier method. You press a key first, than modify it with a hotkey in a cycle, the last change always visible on screen, until you find what you need. Many small cycles work better than the dead key method. Fortunately, now an AHK script can easily produce those Unicode characters, which we just assigned to modified keyboard keys: the script sends the dead key and the base key to be modified. And we have its powerful scripting language for doing fancy things, like activating, de-activating Unicode characters in different windows, provide different tray icons to remind the user what key macros are active, selecting subsets of characters, remap the keyboard layout with a hotkey, etc.

My computer is used by other family members, too, so I would hear screams and receive abuse when they find a (dead) key behave unexpectedly. Fortunately, we have a 101-key keyboard, and MSKLC still lets you assign functions to the missing 102-th key found on other keyboards. So I chose this key to be the dead one, so the keyboard behaves exactly as before. No accidental Greek characters or stuttering keys. The dead key is only accessible as a virtual key from AHK or its siblings.

In the AHK script we have to use the corresponding virtual key with the right scan code. In my case it was E2. It is not easy to find, so keep this valuable information in secret, or give it away for really big money only. (Using the original keyboard driver I sent all the possible 256 hex values, one by one with Send {vkxx}. The 102-th key will produce a "\", so it narrows down to a choice between 2. The neighboring key scan codes reveal the real "\", which leaves you the scan code for the missing keyboard key.)