Cannot clear mex with an existing DesktopWindowTarget

I'm trying to create a window and display a user-supplied image using the C++ MEX function: (Just a small snippet of code to illustrate my purpose)

```C++

#include"pch.h"

using namespace matlab::data;

CompositionHost* CH;

void MexFunction::operator()(ArgumentList& outputs, ArgumentList& inputs)

{

if (inputs.empty())

delete CH;

else

{

CH = new CompositionHost;

WindowHandle = CreateWindowExA(WS_EX_LAYERED, "STATIC", NULL, WS_VISIBLE | WS_POPUP, 0, 0, 500, 500, NULL, NULL, GetModuleHandle(NULL), NULL);

CH->Initialize(WindowHandle);

buffer_ptr_t<uint8_t>PixelBuffer = TypedArray<uint8_t>(std::move(inputs[0])).release();

CH->AddElement(PixelBuffer.get());

outputs[0] = ArrayFactory().createScalar((uint64_t)WindowHandle);

}

}

```

The entire project is compiled as MexWindow.mexw64. When the user enters a random uint8 array, it is able to successfully display a random image on the screen:

MexWindow(randi(255,[4,128,128],'uint8'));

Then, without providing any arguments, call MexWindow again and the image will be cleared:

MexWindow;

It is safe to clear the MEX at this point:

clear mex

But the problem is that if the user forgets to clear the image first, and go straight to clear mex:

MexWindow(randi(255,[4,128,128],'uint8'));
clear mex

This will cause MATLAB to freeze permanently. Even if I clean up manually in DllMain:

```C++

#include "pch.h"

HWND WindowHandle;

BOOL APIENTRY DllMain( HMODULE hModule,

DWORD ul_reason_for_call,

LPVOID lpReserved

)

{

if (ul_reason_for_call == DLL_PROCESS_DETACH)

{

delete CH;

DestroyWindow(WindowHandle);

}

return TRUE;

}

```

Still can't stop MATLAB from stucking. After some research, I found out that it was the winrt::Windows::UI::Composition::Desktop::DesktopWindowTarget member in CompositionHost that was causing the problem. However, the user must actively issue a command to destruct this object. If the destructor is called in the DllMain function after clear mex, MATLAB will still freeze. It seems that this object just can't destruct after clear mex. But you also can't destroy it immediately after drawing, because that will also clear the drawn image at the same time.

What could be the cause of this stuck? Is there any way to avoid it?

This problem only occurs in MATLAB. I have tried within general Win32 applications: dynamic loading of DLLs, creating windows and drawing graphics, unloading DLLs, and releasing resources in DllMain, but not encountering this problem.

Answer 1

[埃博拉酱]

Although I don't know the principle, I solved the problem. Simply move the resource release code into the destructor of MexFunction. Don't do anything in DllMain.