text is a built-in function so you can't use that as a variable name. Anyway, it's not text - it's an image so call it grayImage or something more descriptive.
You need to define keyinverse. You don't have it. You have something similar kinverse, but that's a different variable.
You need to define m and n. You really need to use descriptive variable names so it doesn't look like an impenetrable alaphbet soup of code. And add LOTS more comments. Here's a little bit further but this code still needs MAJOR improvement:
close all; clear;clc; workspace;
grayImage = imread('cameraman.tif');
grayImage = double(grayImage);
[m,n]=size(grayImage);
grayImage=reshape(grayImage,1,m*n);
key = [2 3;1 4]; %key
keyinverse= [6 15; 5 16];%inverse matrix
[ new_code,original_image ] = encrypt_decrypt( key, keyinverse,grayImage );
figure(1),imshow(new_code);
figure(2),imshow(original_image);
fprintf('Done running %s.m.\n', mfilename);
function [ new_code,original_image ] = encrypt_decrypt( k, kinv, grayImage )
a=reshape(grayImage,2,(length(grayImage))/2); %convert to two rows
code=mod(k*(a),256);
new_code=reshape(code,1,length(grayImage)); %reshape to one row
new_code=uint8(new_code);
new_code=reshape(new_code,m,n);
text2=new_code;
a =double(text2);
text2=reshape(text2,1,m*n);
b=reshape(a,2,length(text2)/2);
msg=mod(k*(b),256);
new_msg=reshape(msg,1,length(text2));
original_image=uint8(new_msg);
original_image=reshape(original_image,m,n);
end
