Drop-in Fiber Replacement for Threads

I have a program that requires a large number of tasks being run concurrently, but not in parallel. When I started working on this program I explored some options (Callback based, turned out to be too impractical to be viable; tink.async, turned out to completely mess up IDE autocompletion and @async/@await crept its way into every function, also slowed compilation down to a crawl). I eventually ended up simulating Fibers with the standard Thread API. This worked fine for a while, but as the complexity of processed data grows the performance starts to become unbearable.

Long story short: I have a handful of classes which create and coordinate Threads, but I have to replace them with Green Threads/Fibers/whatever you wanna call 'em. Anything that resembles async/await or callback-functionality is out of the questions as I would have to rewrite tens of thousands of LoC. Can anyone think of a conceivable way to do this without switching to one of the super-slow target platforms?

1 Like

I’m curious about why your Fiber thread approach slowed to a crawl, I guess thousands of concurrent thread were being created? Or is all your async code getting executed on the main thread?

Maybe there’s a tweak to your Fiber implementation to improve the performance – when you say performance becomes unbearable is that because the UI/display thread gets hammered? Or the data-processing is slower than it should be?

The first approach that comes to mind would be to implement a job system – you queue your jobs with the job manager (maybe a job here is a callback) and the job manager decides how to execute: it could say have a pool of 4 threads each with their own job queue and distribute jobs between them

What does your implementation and async usage look like right now? (Feel free to drop in some sample code or web links)

The thread count is very high, and so is the number of context switches. It can get into the thousands range very easily. That is despite already keeping threads alive and reusing them wherever possible.

There is no UI component at all, its just a transpiler that is run on the command line, but it takes a long time for the processing to occur. I already used a profiler to determine that Thread.create is in fact a hot spot.

Unfortunately the described job system won’t work here, because jobs have to suspend their execution midway many times to wait for some other unknown task to finish, so they all need their own stacks.

Most of the async stuff is isolated into this class:

Outside code can enque tasks and awaitValues but has otherwise no idea that anything is asynchronously happening (everything looks synchronous).

EDIT: I would like all the code to be executed on the main thread, but without actual Fibers that is not possible. Right now I’m spawning threads to take on the tasks.