Quick question about imagesc, colormaps and colorbars

Hello, I have a question about an imagesc plot I create animations for, and why I get such a wildly changing colorbar.

Basically, what I do to set it up is set up a n x n x 10 matrix of images, which I first plot the first slice of (:,:,1) using iamgesc. For each subsequent frame of the animation, I set the cdata property of the imagesc plot I made to the new data values.

After that, I find the highest and lowest values along the 10 frames. After I do that, I set both the caxis and the Limits property of the colorbar to those limits.

Now to my problem. Here is a gif of the animation I created , the problem is for some reason the colormap isn't consistent... its like its redrawing the map every slice. I tried everything from manually declaring the climits and caxis every frame, but i either get the color of the colorbar changing with the ticks/range the same or the tick labels of the colorbar changing with the colors the same.

Any ideas on how best to address this? Thanks!