GUI freezes on click of datacursor on image that has attached ButtonDownFcn

I have a Gui with two axes. One contains an image which has a ButtonDownFcn attached to it (to the image, not the axes). The other has a plot. Ideally, I'd like to be able to use the datacursor on the plot and still be able to click the image, but since I haven't been able to do that, I just added a button to turn the datacursor on and off. However, with the datacursor on, clicking on the image results in MATLAB freezing. Eventually the datacursor does appear on the image and it unfreezes. What's causing this, and what can I do about it?