Nobody can possibly answer your question: "why... unwanted stuff...?" Only the author of the binary file format can tell you that. --More specifically, those 4 bytes actually exist in the file. MATLAB isn't misbehaving in any way.
I can tell you that you've already solved your problem. Simply skip 4 bytes and proceed. I bet they are useful somehow, but not likely as an int32. Try other 4-byte formats like float32, or 4*char. Maybe it's a float32 datestamp?