Read process output while it is running

Let’s say I have this code in src/Proc.hx:

class Proc {
    public static function main() {
        trace('one');
        Sys.sleep(2);
        trace('two');
        Sys.sleep(2);
        trace('three');
        Sys.sleep(2);
        return 1;
    }
}

Compile it like so:

haxe -cp src -main Proc -cpp bin

and when you run it:

./bin/Proc

you get:

src/Proc.hx:3: one
src/Proc.hx:5: two
src/Proc.hx:7: three

with 2 second delay in between each line.

Now I would like to do this from different Haxe code/app:

  • start (run, execute) the bin/Proc binary via Sys.io.Process
  • read its stdout as it becomes available
  • print to this app’s stdout whatever we got from the running Proc binary
  • repeat until Proc terminates

Basically, I’d like to be able to read and printout stdout of the started (long-running) process, while it is running and not after it has finished.

This:

    public static function main() {
        trace('click');
        var p = new Process( './bin/Proc' );

        trace('tick');
        while( true ) {
            Sys.sleep(1);

            trace('tock');
            if( p.exitCode(false) != null ) {
                break;
            }
        }
    }

does not work as I would expect it to:

  • have ‘click’ printed out
  • immediately followed by ‘tick’ and then
  • six(ish) ‘tock’ printed every second until the created process exits

I get “command not supported” (on exitCode(false) line) exception on Neko and a cryptic Error : ValueException after first ‘tock’ is printed out on C++ target. I guess neither of them like call to exitCode(false).

How would one go about doing this for Neko or C++ target? Neither of those seems to support non-blocking call to Sys.io.Process#exitCode() method and I have no idea how to check if process is still running otherwise. From my experiments, I’m not even sure if, when you create an instance of the Sys.io.Process on either target, application blocks until created process finishes or if the process runs in parallel with the app that started it.

So, how would I start a long-running process and then move on with the rest of the work and check from time to time the process’ stdout and if it has finished on Neko or C++ targets?

Thanks!

I think you will need to create a new thread that will run the blocking Process api and then communicate the results to the main thread.

Here’s an example of how to do this restructuring the application to run an event loop in the main thread:

import sys.thread.Thread;

enum EventLoopMessage {
	Tick;
	Tock;
	ProcResult(text:String);
}

class ThreadTest {
	public static function main() {
		var mainThread = Thread.current();

		Thread.create( () -> {
			while(true) {
				mainThread.sendMessage(Tick);
				Sys.sleep(1);
			}
		});

		Thread.create( () -> {
			while(true) {
				mainThread.sendMessage(Tock);
				Sys.sleep(1);
			}
		});

		Thread.create( () -> {
			// You could run your process in this thread.
			Sys.sleep(10);
			mainThread.sendMessage(ProcResult("This simulates the process end result"));
		});

		// All the main thread does is handle the event loop.
		while( true ) {
			var message : EventLoopMessage = Thread.readMessage(true); // Wait for the next event in a blocking way.
			switch message {
				case Tick:
					trace("Tick");
				case Tock:
					trace("Tock");
				case ProcResult(text):
					trace('Process result: $text');
					return;
			}
		}
	}
}

@basro, thanks for the suggestion!

Maybe me giving a source of the Proc class was a bit misleading: I need to run a number of external processes, hence Sys.io.Process, and not a Haxe code that I can include as a part of the app.

All of the binaries I need to run communicate their current status via stdout and will take some time to finish (seconds, minutes, mostly even longer than that), so I also need to be able to read whatever they output to stdout as that output is generated and not after the work has been done and external process has terminated.

Yes, I think understood. But because the API is blocking you still need to use threads.

Here’s the example modified to show how to read the stdout of a process:

import haxe.io.Eof;
import sys.io.Process;
import sys.thread.Thread;

enum EventLoopMessage {
	Tick;
	Tock;
	ProcData(text:String);
	ProcResult(text:String);
}

class ThreadTest {
	public static function main() {
		var mainThread = Thread.current();

		Thread.create( () -> {
			while(true) {
				mainThread.sendMessage(Tick);
				Sys.sleep(1);
			}
		});

		Thread.create( () -> {
			while(true) {
				mainThread.sendMessage(Tock);
				Sys.sleep(1);
			}
		});

		Thread.create( () -> {
			var p = new Process("ping www.google.com");
			try {
				while ( true ) {
					var line = p.stdout.readLine();
					// You might want to parse the data here.
					mainThread.sendMessage(ProcData(line));
				}
			}catch(e: Eof) {
				mainThread.sendMessage(ProcResult("Stdout was closed"));
			}
		});

		// All the main thread does is handle the event loop.
		while( true ) {
			var message : EventLoopMessage = Thread.readMessage(true); // Wait for the next event in a blocking way.
			switch message {
				case Tick:
					trace("Tick");
				case Tock:
					trace("Tock");
				case ProcData(text):
					trace('Process data: $text');
				case ProcResult(text):
					trace('Process result: $text');
					return;
			}
		}
	}
}

I think there’s an ongoing effort to implement a new async API, so in the future you might be able to do this without having to run your own event loop and threads.

Non-blocking Process.exitCode() is implemented for a few targets.
You better read stdout/stderr until EOF and then check exitCode in blocking mode.

ValueException should be converted to a human-readable message upon a crash by hxcpp runtime.
If it’s not the case, please, submit an issue to the Haxe repo with a reproducible sample.
Meanwhile to get information from ValueException you can catch it and use this API: haxe.Exception - Haxe 4.2.1 API

Something like this should get you started, obviously there is lots of room for improvment here, and i cant remember exactly how to get the main thread to wait on the other threads to be compete, but it should start you off at least :slight_smile:

package;

import sys.io.Process;
import sys.thread.Thread;

class Main {
	static function main() {
        var thread = Thread.create(run);
        thread.sendMessage(Thread.current());
        
        while (true) {
            try {
                if (Thread.readMessage(false) == "COMPLETE") {
                    break;
                }
            } catch (e) {
                trace(e);
            }
            Sys.sleep(.1);
        }
        trace("ended"); // for some reason it never gets here, so im doing something wrong with the messaging, but i cant remember what
	}
    
    static function run() {
        var mainThread = Thread.readMessage(true);
        trace("----------------------------------> Thread started");
        
        //var p = new Process("cmd /c dir /s c:"); // use this to recuse dirs from c: so it will take ages, so you know its working
        var p = new Process("cmd /c dir c:");
        var stdout = p.stdout;
        while (true) {
            try {
                var b = stdout.readByte();
                Sys.print(String.fromCharCode(b));
            } catch (e) {
                break;
            }
        }
        
        trace("----------------------------------> Thread ended");
        mainThread.sendMessage("COMPLETE");
    }
}

Cheers,
Ian

EDIT: just realised the code above is basically the same, but a better version of the same… :smiley:
My bad, ill learn to read one day :smiley:

2 Likes