Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Is there a way to simulate a mouse click anywhere on a screen in Dictate ?

I'm using Dictate because of hand problems. Unfortunately, it does not seem to have the ability as supplied to simulate mouse clicks anywhere on the screen as Dragon Naturally Speaking does on the Windows side.( I do not mean by this the ability to select an item in the menu, which It does have the built in ability to do.) It can, however, so I read, be scripted to do a lot of things. Is there an Applescript out there that would allow for the simulation of mouse clicks anywhere on the screen (as opposed to just on a UI item)?

2 gHz Core Duo iMac w 2 gigs RAM, Mac OS X (10.5.5)

Posted on Nov 10, 2008 6:37 AM

Reply
22 replies

Nov 10, 2008 9:54 AM in response to Everardo Verguizas

Hi Everardo

Extra Suites from the link below has the ability to mouse click

http://osaxen.com/files/extrasuites1.1.html

using something like the below you can move the mouse to a location on your monitor and have extra suites click for you.
------------
tell application "Extra Suites"
ES move mouse {700, 270}
ES click mouse
end tell

-------------


Budgie

Nov 16, 2008 7:05 AM in response to Everardo Verguizas

Everardo,

You might have options using your Mac's own built-in Speech Recognition framework.

The following example scripts are designed to perform the indicated click-actions via spoken command when your mouse cursor is already situated at a desired spot -- hovering over a piece of text, a link, an application icon, a folder, empty space on the desktop, etc. Moving your cursor to the particular spot is another matter. According to the information found on this page, under 'Make Your Move,' however, it would seem that Dictate does allow you to move the cursor via voice. My hope would be that you can use the scripts below in conjunction with Dictate.

Paste each of the following blocks of code into your AppleScript Script Editor. In each case, choose File > Save As > File Format: application, and leave the Options unchecked. Place aliases for the saved scripts into your Home > Library > Speech > Speakable Items folder:

+1) This script performs a single-click, good for clicking links. When saving, you can name it "Click." The code:+

*tell application "System Events"*
*keystroke tab using command down*
*keystroke (ASCII character 53)*
*end tell*

+2) This one performs a double-click, good for opening applications or folders, and for those occasions when you want to highlight a single word on a page. Name it "Double Click."+

*tell application "System Events"*
*keystroke (ASCII character 53)*
*keystroke (ASCII character 53)*
*end tell*

+3) Likewise, triple-click, for highlighting a line or an entire paragraph of text. Name this one "Triple Click."+

*tell application "System Events"*
*keystroke (ASCII character 53)*
*keystroke (ASCII character 53)*
*keystroke (ASCII character 53)*
*end tell*

+4) This script performs a control-click (right-click) on an item or on a piece of text, useful when you want to bring up a contextual menu. Name it "Control-Click" (or "Right-Click").+

*tell application "System Events"*
*keystroke tab using command down*
*key down control*
*keystroke (ASCII character 53)*
*key up control*
*end tell*

Once the scripts are properly saved and their aliases placed in your Speakable Items folder, these additional steps need to be taken for them to work:

1) GUI Scripting must be turned on. Under System Preferences > Universal Access, check "Enable access for assistive devices."

2) Mouse Keys must be turned on. System Preferences > Universal Access > Mouse, select the Mouse Keys: On radio button.

3) In System Preferences > Speech > Speech Recognition > Settings, select the Speakable Items: On radio button. To activate a voice command entirely hands-free, select Listening Method: "Listen Continuously with keyword." You then have the option whether to actually use the keyword; if you'd prefer not to use a keyword, select Keyword is: "Optional before commands." If necessary, click the Calibrate button to adjust the microphone input volume.

You should be all set at this point. If you choose not to use a spoken keyword or modifier key, then, thereafter, when your mouse cursor is hovering over a desired spot on the screen, simply speak the name of the saved script application -- "Click," "Double-Click," "Triple-Click," or "Control-Click" (or "Right-Click") -- and the corresponding action should complete.

These scripts worked for me, using Mac OS 10.4.11. Your results may vary... Good luck.

Nov 16, 2008 3:18 PM in response to Andrew99

Andrew,

I am certainly going to try out your suggestions. Thank you for a very thorough description of what needs to be done. I understand that, until I try this out with Dictate, I won't know for sure whether it works. I really appreciate your taking the time, however, to walk me through the procedure in such detail.

I probably won't be able to try this out for a day or two, but, when I do, whether it works or not, I will post a complete description of what happened so that if there are others in the same situation they can benefit from your suggestions. I will also post the results on MacSpeech's forum. Thanks very much again.
Everardo

Nov 16, 2008 5:57 PM in response to Everardo Verguizas

Andrew,
I couldn't resist waiting so I tried, following your detailed instructions, to create a " mouse click" in Dictate. I couldn't get it to work from the regular script editor. However, Dictate provides its own script editor and so, following MacSpeech's instructions, I copied and pasted your script into the script editor, compiled it, named it "mouse click," and then attempted to use the command (in global commands). I did this only after setting all the switches in system preferences as you explained. It did work. However, I'm not sure that this is definitive yet, since I see the the round speakable items window is open at the same time as MacSpeech's control window. So I'm going to try to stress test this as much as possible.

I'll continue testing this as time permits.

Everardo

Nov 16, 2008 8:27 PM in response to Everardo Verguizas

...I see the the round speakable items window is open at the same time as MacSpeech's control window.


Hmm... If both Dictate and your Mac's Speech Recognition are active at the same time I can see a potential for conflict, yes. If each program tried to respond to a voice command common to both programs, something unpredictable could result. Problem.

You might see if you can avoid this by using the keyword approach referred to in the earlier post. Under System Preferences > Speech > Speech Recognition > Settings, set the Listening Method to "Listen continuously with keyword," and choose "Keyword is: Required before each command." That way Dictate would assume handling the bulk of your voice commands, while your Mac's Speech Recognition would only respond to those commands preceded by a keyword. For example, if by chance Dictate were set up to use a command common to Speakable Items, "Get My Mail," for instance, Dictate should respond to this spoken command rather than your Mac's Speech Recognition. Your Mac's Speech Recognition wouldn't recognize the command unless you had said, " Computer, get my mail." ("Computer" is the default keyword, which can be changed.)

Of course, if you have a concern that the round Speech Feedback window is creating clutter on your screen, it can be minimized to the dock without affecting performance.

I couldn't get it to work from the regular script editor.


None of the example scripts are meant to be run from the Script Editor, of course, but they should be compiled and saved from there. If saved properly, an AppleScript application icon should result. It's an alias of this icon you'd want to place into the Speakable Items folder.

That said, I'm glad to hear you had the success you did using the method that worked for you. Continued good luck!

Andrew99

Nov 27, 2008 1:09 PM in response to Everardo Verguizas

Platform: 2.2 gHz iMac using Mac OS X 10.5.5; Dictate v. 1.2.1 Build

Okay, so, here, after a great deal of research, is a procedure for implementing a single mouse click anywhere on the screen. Since it’s a procedure, I’m detailing each step.

!. Download the scripting addition (sometimes known as an OSAX) xTool.sax from: http://lestang.org/osax/XTool/XTool-2.0.dmg.tgz. You will have to decompress the file (StuffIt) and double-click it to get it to unfold into a .dmg (disk image) file. In this file you will find the file Xtool.osax. Drag it into your MacHD (or whatever the name of your topmost volume is)/library/ScriptingAdditions (no spaces in the name) folder. The purpose of this is to place it into a folder where Applescript is expecting to find scripting additions. In other words, it’s hard-coded and you can’t just drop it anywhere because Applescript may not find it. Also, there may be more than one Library folder in other directories. Under 10.5.5. which I’m running, I can tell you it works where I placed it.

The effect all of this has is that when you create your script in Dictate’s New Commands window (Script Editor), Dictate will know what the commands “click mouse’ and “position of the mouse” mean. These are not standard Applsecript commands but are additions, hence the name Scripting Additions.

2. Start Dictate and in the File drop-down select New Command. This will take you to a window with a pane on the left with names such as Applications, Global, and several others. Click on Global (so that the command will be recognized no matter what application your are in). To the right of the pane you will see a number of commands listed by name and, underneath, a (very small) “+” (and a minus sign and a small icon). Ignore everything but the “+”. Click on the “+”.

Below several boxes will appear: Command, Description, Context, Type.
In Command Name type in the name you want Dictate to recognize to do the click. (I call mine Mouse click.) In Description you can type something like Single-clciks the mouse. In the Context drop-down make sure to select Global. In the Type drop-down, select Applescript.

Paste the following into Text Edit (orTexEdit Plus,if you have that):
click mouse {(position of the mouse), [1]}

For reasons unknown to me, you can’t just select that text, copy, and paste it into Dictate’s Script Editor winodw. You will have to select it in Text Edit and drag it into the Source text box.

Click on the Compile button at the left bottom. After it finishes compiling (a matter of seconds if everything is normal), click on the Run button with your mouse. If all has gone well, you will very likely see the Run button get pressed repeatedly. Take your mouse off the button so it stops blinking like mad. (The Mouse click Appplescript is repeatedly looking at the position it find itself and clicking again, and again.)

While your mileage may vary, I have been able to simulate single mouse-click using this workaround.

So far, I have not been able to simulate a double-click reliably. There is, I believe, an issue with the amount of time between the two clicks (a parameter of the click mouse command above which I did not go into here). For anyone out there who wants to work on that, I would be glad to hear your results. I’ve tried values up to 2000 in the DELAY parameter after the TIMES parm. Here is what the dictionary shows for this command.

click mouse?v
click mouse [point] : where to do the mouse click
[times integer] : how many times you want to clik the mouse
[delay integer] : delay between two mouse clicks
[using primary button/secondary button/middle button] : which mouse button


If I get this to work I will let my fellow sufferers know.

Thanks to Hiroto and Andrew99 in particular for their help.

Nov 28, 2008 1:48 AM in response to Everardo Verguizas

Hello

I'm not sure at all, for I cannot test XTool 2.0 in my environment, but it seems to me the correct syntax of single-clicking at the current postion of the mouse pointer would be -

click mouse (position of the mouse) times 1

Or -

click mouse (position of the mouse)

Or -

click mouse

The last one might simply work IF the direct parameter of 'click mouse' command - 'point' - is optional (because it's in brackets in dictionary) and its default value is the current position of the mouse and 'times' parameter is optional and its default value is 1.

* Generally speaking, parameter enclosed in brackets in dictionary is optional. Also, there're two kinds of command parameters: one is direct parameter which should immediately follow the command and another is labeled parameter which has leading label to the parameter value.

E.g.
click mouse point
times integer

indicates the command has direct parameter whose value is of class 'point' and labeled parameter whose label is 'times' and value is of class 'integer'.



---
As for double-clicking, it might be -

click mouse (position of the mouse) times 2 delay 200

Or -

click mouse times 2 delay 200

I assumed the unit of 'delay' parameter is millisecond, but it may be wrong.
I'd look into the dictionary and documentation of the OSAX if any.
If they state nothing helpful or even something incorrect which often happens, we'd need to experiment on the command and find the proper syntax and parameter by trial and error...

Good luck,
H

Nov 28, 2008 6:27 AM in response to Hiroto

Hiroto,
You propose alternate, possibly correct syntaxes instead of the example I gave. However, as far as I can determine the command I described is the one responsible for the on-screen results I've seen.

To be as accurate as possible, here's what I've done.

Checked to make sure that that Speech in System Preference was not set to have Speech Recognition on, but instead was set to off. It was off, so that the mouse clicks on screen could not have been the result of calling Speech Recognition ( the approach Andrew99 has suggested and which I had tried, with the results I've mentioned before).

Also, I had deleted the previous Dictate command I had attempted, using the ExtraSuites osax Budgie had suggested.

As a result of these two things I think I can rule out that something else in my computer is executing the mouse clicks from within Dictate.

The results I've seen on screen are as follows:

Nov 28, 2008 7:02 AM in response to Everardo Verguizas

Hmmm..I've tried your suggestion that I just use the click mouse command and I have to admit that it appears to work just fine. So it looks as though my syntax only works accidentally and just using click mouse will do the trick. I'll go back and try your suggested syntax on double-clicking and see if I have better luck.

Always glad to hear from helpful people like you who know better than I do.

Everardo

Is there a way to simulate a mouse click anywhere on a screen in Dictate ?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.