Unexpected error calling cuDNN: CUDNN_STATUS_BAD_PARAM.

7 visualizaciones (últimos 30 días)
Scott Stearns
Scott Stearns el 20 de Mzo. de 2021
Comentada: Tom Van den heuvel el 21 de Sept. de 2021
Hi,
This error stops training when the 'ExecutionEnvironment' is 'parallel', 'multi-gpu', or 'gpu'. Training is running uninterrupted when set to 'cpu'. I'm running code for the first time on Ubuntu 20.04.2 LTS system with Intel i9 12 core cpu and 2x 3070 gpu's. It indicates only 12 workers and seems to not recognize the gpus.
Any suggestions and help is welcome.
Thank-you

Respuesta aceptada

Joss Knight
Joss Knight el 21 de Mzo. de 2021
Editada: Joss Knight el 23 de Mzo. de 2021
After some investigation (see thread below), this problem seems to be limited to RTX 3080 and 3070 and Linux. It can be worked around by disabling tensor cores. Restart MATLAB and run
setenv NVIDIA_TF32_OVERRIDE 0
before you do anything else. Further investigations are under way to look for a solution that doesn't require this workaround, which will reduce performance.
Original answer:
Are you running MATLAB release R2021a? The 3070 is not supported on earlier releases.
  47 comentarios
Joss Knight
Joss Knight el 21 de Sept. de 2021
Editada: Joss Knight el 21 de Sept. de 2021
This is fixed in the next update of MATLAB R2021a, however you'd be better off simply downloading R2021b which will be out in a week or so.
Unfortunately NVIDIA weren't able to provide us with a fix that has no effect on performance, but we can at least limit the workaround to the problematic convolutions. A proper fix will arrive with the next CUDA upgrade.
We've never seen this problem on Windows.
Tom Van den heuvel
Tom Van den heuvel el 21 de Sept. de 2021
Thx for the update!

Iniciar sesión para comentar.

Más respuestas (0)

Categorías

Más información sobre Parallel and Cloud en Help Center y File Exchange.

Etiquetas

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by