Two things:
The thing you're doing here
is not solarization. This is just the additive inverse or complement of the source image. If you look closely at the example result, the behavior is dissimilar.
Also, the example result does not appear to have any mask applied to it at all. The background content is still there. I have a feeling that the only reason you think you need a mask is because you're doing inversion instead of solarization.
While the example image doesn't look like what you might get from some solarize filters, it can be replicated. I'm sure that nothing needs to be specific here, so I'm just going to contrive a filter based on a general assumption of behavior and desired aesthetics. What your example shows is a filter wherein the intensity response is inverted only toward the top end. Something like this:
inpict = linspace(0,1,100);
solar = ((1-os)*inpict + os).*(1 - kinv*inpict.^ks);
avg0 = mean(inpict,'all');
avgs = mean(solar,'all');
h1 = plot(inpict,inpict); hold on
legend([h1 h2], {'original','solarized'},'location','northwest');
Note that the filter curve will raise the midtones. This is due to the rescaling done on the last line. If that's undesired, simply don't rescale, but understand that the image will be generally dark. You can adjust the parameters to suit your expectations.
This can be applied to the image just the same:
inpict = imread('rose.png');
inpict = im2double(inpict);
solar = (inpict + os).*(1 - kinv*inpict.^ks);
avg0 = mean(inpict,'all');
avgs = mean(solar,'all');
Note that there is no need for masking to suppress the background.