hexidecimal from matlab to c++ from a calllib function

There is a function in a c library i am using, and one of the input arguments is a ulong. In the header file the constant I wish to use is being referenced by the hexadecimal number 0xffffffff. So ideally i should be able to convert that into a decimal number and use that number as the input argument. i.e calllib('libname', 'functionname', str2double(hex2dec(ffffffff))). I am unsure if this is working correctly though. Also from the documentation for the library I am using it says "the symbols _WINDOWS and LITTLE_ENDIAN_PLATFORM must be defined" which leads me to think i may be running into issues with something like that.

Anything helps!

Also if you have information if its easier to just link this library to a c compiled program (turn into a .mex64 later) that might be easier