Problem going back from frequency domain to time domain using IFFT after some processing in frequency domain

So, I'm trying to do some processing in the frequency domain, the reason is that I have to measure a very fast pulse and in order to do it accurately I need to compensate for the attenuation in high frequencies induced by cables and some attenuators. The pulse I am trying to measure has a risetime of 150ps with 4-5 ns pulse width, I am using a 100Gs/s scope so sampling frquency is not a problem, here is a plot of my data 50000 samples and its fourier transform after shifting(magnitude and phase), note that i am zooming the plot to +/- 6GHz

Now I have the S parameters of my set up measured using a VNA, which gives me the attenuation of the cables and attenuators from 300kHz to 6GHz, I am plotting the S21 parameters, the VNA give me complex nombers as ReS21 and ImS21, here I plot magnitude and phase.

As you can see the attenuation is quite large -70db and that is because i am measuring a quite large voltage, around 10Kv so I have to use large attenuators to measure it with a scope, but this attenuators and cables also induce additional attenuation in higer frequencies, I want to reconstruct the signal and use this attenuation data to compensate for the cables and attenuators.

So far I have been able to reconstruct the original pulse without applying this compensation but as soon as I apply this compensation and try to go back to time domain instead of get real numbers I get complex numbers, my guess is that there is a problem with the phase but I am not sure what is happening.

Also in the range of 300Mhz to 6GHz my signal.

Here the code:

%% FFT computing and plotting
L = size(Data(),1);                 % Length of signal
DataFFT2S=zeros(L,2);               % creating array 
DataFFT2S(:,2)=fftshift(fft(Data(:,2)));  %perform FFT and shift
T = Data(2,1);                      % Sampling period 
Fs=1/T;                             %sampling frequency
df=Fs/L;                            %frequency resolution
% calculate unshifted frequency vector
f = (0:(L-1))*df;                   %%frq vector
% move all frequencies that are greater than fs/2 to the negative side of the axis
f(f >= Fs/2) = f(f >= Fs/2) - Fs;
f=fftshift(f);
DataFFT2S(:,1)=f;%positive and negative freq vector

DataFFT2S is a 50000X2 array and is shifted as shown inthe picture above

Now I import the VNA data and do an interpolation to get the real and imaginary parts for every frequency in DataFFT2S

%% correction from VNA and attenuators
filename='attenuators+cable.csv';
VNAData = csvread(filename,1,0); %columns named bFreq,ReS11,ImS11,ReS21,ImS21,ReS12,ImS12,ReS22,ImS22
VNAfreq= VNAData(:,1);
ReS12=VNAData(:,4);
ImS12=VNAData(:,5);
VNAAtt=ReS12+(i*ImS12);
%Interpolating the VNA results for the FFT frequencies
size1=int64(6e9/df);
VNAInterp=zeros(size1+1,2);
VNAInterp(:,1)=DataFFT2S(L/2+1:L/2+size1+1,1); %just the positive frequencies from 0 to 6GHz
VNAInterp(:,2)=interp1(VNAfreq,VNAAtt,VNAInterp(:,1),'spline','extrap');

the VNA gives me 201 data points, after the interpolation I get 61 data points, VNAInterp is a 61X2 complex array and it's magnitude and phase are also shown above, the comparison between the interpolated data and the original one is shown here blue original orange interpolated/extrapolated

Now I divide the original fourier transform by the interpolated data from the VNA only in the range from 0 to 6GHz, the positive frequencies are same as negative frequencies but flippend and conjugated but excluding the DC component.

%data correction
DataFFT2S(L/2+1:L/2+1+size1,2)=DataFFT2S(L/2+1:L/2+1+size1,2)./VNAInterp(:,2);%
DataFFT2S(L/2+1-size1:L/2,2)=flip(conj(DataFFT2S(L/2+2:L/2+1+size1,2)));

here the plots of magnitude and phase of the result

so far it looks symmetric and i souldnt have any problem going back to time domain right? well i get complex numbers and i cant reconstruct the signal, if i plot the magnitude of that looks like this

%% Back to Time domain
DataFFT2S(:,2)=ifftshift(DataFFT2S(:,2));
DataFFT2S(:,2)=ifft(DataFFT2S(:,2));

Is not even close to the shape of the original pulse, my guess is that something is going on with the phase but im not sure.

HELP please