Do the xyz values correspond to voxel locations? If so, simply use a for loop to stuff the value into the appropriate voxel.
image3d = zeros(maxRow, maxCol, maxSlice, 'uint8');
image3d(y(k), x(k), z(k)) = grayLevel(k);
If x, y, and z are floating point or have values way different than the number of voxel dimensions you want, then you can use round(rescale()) to turn them into voxel coordinates before passing to image3d as indexes.