Is there a maximum input size for mex? Segmentation Fault with large inputs

Hi,

I have been developing an application in which i need to read then send large strings from matlab to mex. I cannot split up the string.

I found it was crashing occasionally with a segmentation fault and i have been able to replicate it ( reasonably reliably - it does not crash everytime ) with the mex script below:

 /****************************************************************
 This function is testing the reading in a string in mex
 ***************************************************************/
 #include "mex.h"
 #include "matrix.h"
 #include <signal.h>
 #include <setjmp.h>
 #include <stdlib.h>
 #include <gpib/ib.h>
 #include <string.h>

 void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
 {
   /* INPUTS AND OUTPUT */
   int ud = mxGetScalar(prhs[0]);
   char * array = mxArrayToString(prhs[1]);
   int len = mxGetScalar(prhs[2])+1;  
   int * ret;
   plhs[0] = mxCreateNumericMatrix(1, 1, mxINT32_CLASS, mxREAL);
   ret = (int*)mxGetPr(plhs[0]);
   ret[0]= -1;
   mexPrintf("Length = %i\n",len);
   char * buffer = mxMalloc(len);
   if(buffer == NULL || array == NULL){
     ret[0] = -2;
     return;
   }

   int i;
   mexPrintf("Test 0\n");
   for(i=0; i<(len-1); i++){
     buffer[i] = array[i];
     /* CRASH IN HERE */
   }
   mexPrintf("Test 1\n");
   buffer[len-1] = '\0';
   mexPrintf("Test 2\n");
   mexPrintf("%s\n",buffer);

   mxFree(buffer);
   return;
 }

I compiled with mex (gcc-4.3) and the -g flag and I tested it with the following code:

 input = char(mod(1:1500000,256));
 mex_test(16,input,length(input))

The output for a crash is:

 Test 0
 ***CRASH***

Here is a link to the crash info: http://pastebin.com/BVPwVihC

I have a work around of writing the data to a file using matlab functions and then reading from it in the c file. This seems to be fine with the large strings.

Btw, for my application the length wouldn't be above 1.5e6 characters

I don't think there are any errors in my mex code as it seems to be fine for smaller arrays, but if there are any suggestions i would be grateful

Thanks, Stephen

[James Tursa]

I would advise against using "input" as the name of a variable, since there is a built-in MATLAB function of that same name.

Answer 1

[James Tursa]

There is a bug in the mxArrayToString API function. It only copies the characters up to the first null character. It does not allocate memory for the entire size of the input mxArray and copy the entire contents as expected per the doc. So your loop is accessing invalid memory after a certain point because array was built from a MATLAB character string that contained null (0) characters. You should report this bug to TMW via the official bug report link. In the meantime, you can use a simple bare-bones substitute function, e.g.,

   char *myArrayToString(mxArray *mx)
   {
     char *cp = NULL;
     short *sp; // Assumes a short is 2-bytes
     mwSize i, n;
     if( mx && mxIsChar(mx) ) {
       n = mxGetNumberOfElements(mx);
       sp = (short *) mxGetData(mx);
       cp = (char *) mxMalloc(n+1);
       for( i=0; i<n; i++ ) {
         cp[i] = sp[i];
       }
       cp[n] = '\0';
     }
     return cp;
   }

Thanks for pointing out this bug.

Answer 2

[James Tursa]

What is the connection between len and the size of prhs[1]? How do we know this isn't simply a user input error? I would suggest more robust checking code. E.g.,

char * array = mxArrayToString(prhs[1]);
int len = mxGetNumberOfElements(prhs[1])+1;

Btw, in a mex routine out-of-memory errors never return control back to your program with a NULL pointer ... they simply abort back to the calling routine. Thus the following if-test will never be true:

if(buffer == NULL || array == NULL){

If your strings are so large, why are you printing out buffer to the screen?

Finally, you don't need special code to put the '\0' character at the end of buffer ... mxArrayToString will put one at the end of array for you automatically, so you can simply copy that character as part of your loop and delete the special code entirely (unless it is there for some other purpose that is not shown).

[Stephen]

This is not the function i was using. I created it to narrow down the problem. It is not a user input problem: see

"... and I tested it with the following code:

 input = char(mod(1:1500000,256));
 mex_test(16,input,length(input)) "

"Btw, in a mex routine out-of-memory errors never return control back to your program with a NULL pointer ... they simply abort back to the calling routine."

I realized that they shouldn't but i was trying to find the source of the segmentation faults, so i thought just in case.

"If your strings are so large, why are you printing out buffer to the screen?"

No real purpose. it is a testing function. I do not in my application

"Finally, you don't need special code to put the '\0' character at the end of buffer ... mxArrayToString will put one at the end of array for you automatically, so you can simply copy that character as part of your loop and delete the special code entirely (unless it is there for some other purpose that is not shown)."

I didn't know that. So this would imply the length it 1 longer that the 'length(input)'? Hence length + 1?

That makes that code unnecessary, but it shouldn't effect anything else.

Any idea about the segmentation faults?

[James Tursa]

P.S. You are leaking the memory behind array. In this case it is only a temporary leak since array is on the garbage collection list.