Calling User32.dll mouse_event

Hi,

I don't know if this is the right place to ask, but here goes:

I want to control the mouse using Matlab (version R2010B). I tried the java robot code, but it does not work in my application. It has to be done by calling the user32.dll. I can move the mouse using "setCursorPos". But now I also want to emulate a mouse click (left button only for now).

So basically my question is: How do I successfully call mouse_event in user32 to perform a left click ?

Here is my code so far:

clc

screenSize = get(0, 'screensize');

if(libisloaded('user32')==0)

      disp('loading dll...')
      loadlibrary('C:\WINDOWS\system32\user32.dll','user32.h');
  else
      disp('dll already loaded')
  end

calllib('user32','SetCursorPos',screenSize(3)/2,screenSize(4)/2); %center cursor on screen

So far so good. But when I add something like this at the end, it just crashes and I have to restart matlab:

MouseEventLeftDown = 2;
MouseEventLeftUp = 4;

calllib('user32','mouse_event',MouseEventLeftDown,0,0,0,0);
calllib('user32','mouse_event',MouseEventLeftUp,0,0,0,0);

Thanks in advance,

Eelco

Answer 1

[Eelco Galestien]

Hi,

@jan: As I said, the java robot solution does not work in my application. I want to control the mouse in a video game. The game runs in full screen mode and somehow it overrides the java robot.

Anyway, the solution I came up with is this: First create a mex file, include "windows.h" and "winbase.h" Not sure if you really need both. They don't have to be in the Matlab working directory. Matlab should find them in windows/system.

Next pass some numbers inside the mex to the function SetCursorPos(x,y). X and Y should be int32 if I recall correctly. This positions the mouse at the pixel coordinate (x,y) on the screen. I used get(0, 'ScreenSize') in the m-file that calls the mex file because the video game changed the screen resolution on start up.

Last I used the process thread to simulate a mouse click. Inside the mex file I wrote: PostMessage(GetForegroundWindow(),WM_LBUTTONDOWN,0,0); Sleep(20); PostMessage(GetForegroundWindow(),WM_LBUTTONUP,0,0); Sleep(20);

If I understand correctly this posts the code for a mouse click in the message queue of the active window. I used postmessage, because I didn't want to wait for it to actually do so. I you want to wait until the message is actually posted, use sendmessage instead. WM_LBUTTONDOWN and UP are already defined in the header files so they are automatically the right type. GetForegroundWindow() is a pointer the the active window, the application inside which I wanted to click the mouse.

Compile the mex-file and call it from an m-file or from the workspace. The postmessage and sendmessage functions allow you to send anything to any process. It does not even need to be in focus. Just create a pointer to the application you want to control.

[Eelco Galestien]

forgot to mention, the Sleep(20); is very important. It gives the computer the necessary time to actually process your command. Without Sleep(20) you will get very strange behavior, or none at all.

Answer 2

[Jayson]

HI I met the same problem. When I input the word "loadlibrary('C:\WINDOWS\system32\user32.dll','user32.h')", I got back the message "Could not find file user32.h", I dont know why, can anyone help me?

Answer 3

[victoria vincze]

Hy

I'm about to try to do something like this, but your solution doesn't work for me. It cannot find user32.h. Does somebody know how can I solve this?

V

[Jan]

Did you install a C-compiler?

[victoria vincze]

Of Course.

I've a mex file, which sets the cursor, but it also works incorrectly cause it sets the cursor on the left upper corner instead of the center of the screen.

Answer 4

[Chirag Gupta]

I haven't looked at user32.h, but I will take a guess with checking the function syntax in the header file for mouse_event.

When you pass MouseEventLeftDown = 2; you are actually passing a double value (MATLAB defaults to double) [Check using whos]. If the mouse_event expects an int32, it will crash. Try and cast the value to the type expected.

calllib('user32','mouse_event',int32(MouseEventLeftDown),0,0,0,0);

[Jan]

@Chirag: But the last argument is a ULONG_PTR. Does the DOUBLE 0 match this on 32-bit Matlab's also?

[Chirag Gupta]

@Jan: I haven't seen the header file. But if it's a ULONG_PTR then the best course is to create a libpointer of type uint32.

Answer 5

[deng]

clc
screenSize = get(0, 'screensize');
if(libisloaded('user32')==0)
    disp('loading dll...')
    loadlibrary('C:\WINDOWS\system32\user32.dll','user32.h');
else
    disp('dll already loaded')
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
loading dll...
??? Error using ==> loadlibrary>lFullPath at 553
Could not find file user32.h.

Error in ==> loadlibrary at 221
        header=lFullPath(header);

Error in ==> matlabdiaoyongwindowshanshu at 5
    loadlibrary('C:\WINDOWS\system32\user32.dll','user32.h');

[Walter Roberson]

Which operating system are you using, and are you running the operating system in 32 bit or 64 bit mode? Are you using 32 bit MATLAB or 64 bit?