Does the cpp HTTP API do persistent connections?

1

If I make multiple consecutive HTTP requests to the same host using the C++ backend, is there any way to persuade the runtime to reuse the same socket for each connection?

Context: I’m trying to fetch email from a JMAP mail server, and while JMAP is a pretty decent protocol mostly, in order to fetch the raw email data you have to do a distinct HTTP requests for each message, and I have 450,000. Using a persistent connection would make things substantially faster.

2

Hi David - your question caught my interest because I may need to do something similar in the near future (with a large number of http(s) requests going to a single endpoint), and so I tinkered with some short programs and read the docs and source code a bit.

May I first ask for clarification - am I correct in understanding that you are primarily concerned with avoiding repeated TCP and TLS/SSL handshakes, and not simply with reusing the same IP+TCP port combination?

My personal finding from experimentation is the latter is possible but the former does not seem to be possible out-of-box without having to roll one’s own Http class over top of sockets. It appears that the Http class funnels requests through the customRequest method (whether that is called directly or not), which does allow a specific socket to be optionally provided, but the source code (and my observation) indicates that it always closes the provided or created socket at the end of each request.

So if one wanted to simply reuse the same port number, that is entirely possible by simply reinstantiating the socket (I was able to do this on the CPP target against a locally-hosted Node.js Fastify server about 10000 times in the span of a second or two, with only a few missed requests), but this unfortunately would not avoid having to repeat handshakes (which is what I’m assuming you are really trying to avoid).

I’d be very curious if a more seasoned Haxe user than myself has a novel solution for this.

3

Yes, the idea is to avoid the handshakes. You end up with much better throughput (especially if you can also request pipelining, where multiple requests are sent at once and then the server responds with all the combined data at once).

Having done some playing and looking at the source code I can confirm that yes, it always closes and reopens the socket. This gives me a throughput of approximately one message a second.

I did try using a thread pool to do multiple requests simultaneously this can boost throughput to about 50 messages a second, which is a lot better. This is a bit antisocial but Fastmail haven’t blocked me yet; probably because after transferring about 400 messages something inside the haxe network code deadlocks…

Looking at the stack traces, I can see the network threads waiting on hx::ExitGCFreeZone, my master thread waiting on the semaphore I’m using to wait for completion, and the garbage collector waiting to start. My gut feeling is that the semaphore code isn’t GC-aware and isn’t waking up when the garbage collector is requesting it, causing the deadlock.

1 Like
4

Yup. From the C++ runtime, in the Mutex implementation:

        void Acquire()
        {
                hx::EnterGCFreeZone();
                mMutex.Lock();
                hx::ExitGCFreeZone();
        }

But in the Semaphore code:

  void Acquire() {
#if HX_WINDOWS
        WaitForSingleObject(sem, INFINITE);
#elif defined(POSIX_SEMAPHORE)
    sem_wait(&sem);
#elif defined(APPLE_SEMAPHORE)
    dispatch_semaphore_wait(sem, DISPATCH_TIME_FOREVER);
#endif
  }

I patched in the relevant calls to EnterGCFreeZone() and ExitGCFreeZone() without really understanding what they do and it all seems to work now…

5

Wow, that is actually really useful behavior to know about. I can foresee encountering that myself when I return to the project I’m working on where I’ll have to make a large number of requests simultaneously.

For whatever it’s worth, when I ran some experimental multithreaded http request tests for the Haxe cpp target a month or two ago, I go almost the exact speed you mentioned - I was getting about 100 requests per 1 to 1.5 seconds with 32 threads (increasing thread count beyond that didn’t result in meaningful improvements for me in that particular test). I was testing against the public httpbin.org/anything endpoint.

6