Getting python terminal output

I am calling a python function from Matlab that usually to take a couple of hours to finish a run. If the function is run as a proper Python script from Windows terminal, it prints out the operations being done etc. i.e. displays a log:

Cleaning Sig_Traces 1
Loaded Matrix_Weights
Iteration 1 in progress.
Checking for fit ...

So, at any point of time, I have a clue about the progress. Now, how to make this displayable in Matlab's own terminal (or wherever - I don't care) when calling the function from Matlab? I cannot be really staring at a "Busy" footer in Matlab for hours.

[Aboltabol]

As of over three years ago, the answer appears to "Not Possible".

[Delprat Sebastien]

You cannot easily get the print output.

However, if you use the logging module, you can use this in your python code to send repports to matlab. This snipet may have to be reworked but you should have the overall idea

import logging


HANDLER = logging.StreamHandler()
FORMATTER = logging.Formatter('[%(levelname)s] %(filename)s - %(lineno)d - %(message)s')
HANDLER.setFormatter(FORMATTER)

LOGGER = logging.getLogger('YOUR MODULE NAME')    # <= modify here
LOGGER.addHandler(HANDLER)
LOGGER.setLevel(logging.WARNING) # pick whatever level suits

% Then in your code (pick one of these)
LOGGER.info('my message')
LOGGER.warning('my message')
LOGGER.error('my message')

Answer 1

[Walter Roberson]

In the special case where you are running Python from Windows, then you could use the .NET assembly System.Diagnostics.Process to run the python executable while being able to access its standard output.

[Aboltabol]

I apologize for not being aware of this; can you point to any documentation? Btw, I wish to emphasize that my input to the python function is from Matlab and I need to capture back the output, for downstream processing, again in Matlab.

[Walter Roberson]

Mathworks does not provide much information on the topic because Mathwork's only concern is being able to interface with .NET assemblies in general; what you can do with those assemblies is up to Microsoft and the people who write the assemblies.

The following posts may be useful

https://www.mathworks.com/matlabcentral/answers/583955-get-the-status-using-system-command-when-program-has-been-closed#answer_485480

https://www.mathworks.com/matlabcentral/answers/586706-how-to-redirect-standard-input-and-standard-output-using-net-system-diagnostics-process#answer_487475

Answer 2

[Al Danial]

The issue is that matlab buffers Python's print statements and only flushes this buffer--meaning print to the Python text in the matlab command window--when enough text has been added. All you need to do is print a bunch of useless text from Python to fill the buffer before printing your progress message. The best "useless" text is that which has no negative impact on the output you want to see. To that end I use a large number of copies of a space followed by a backspace, so a net zero of cursor movement, and in the end your text appears as it would normally.

Here's sample Python code that does this:

import time
def compute(n_iter):
for i in range(int(n_iter)):
    time.sleep(0.4)
    x = i*i / 7.3 - 0.2
    print(' \b'*4000, end='')  # <- this is the trick
    print(f'{i:3d} {x: 10.6f}')

The second to last line fills the print buffer with no effect on the position of the "real" text which is to be printed (on the last line in this example). Tested on Linux with 2022b.

[Walter Roberson]

However this is not relevant to the behaviour of .NET System.Diagnostics.Process.

.NET -- 4K buffer size, prone to deadlock

https://stackoverflow.com/questions/43204624/what-are-buffer-sizes-of-process-start-for-stdout-and-stderr

But .NET notifications can be delayed until newline is received

https://www.codeproject.com/Articles/170017/Solving-Problems-of-Monitoring-Standard-Output-and