Go2hx Work in Progress

[PXshadow]

go2hx

---

A Go to Haxe transpiler and spiritual sucessor to the Tardisgo project with notable design improvements.

Improvements:

• AST instead of SSA types that allows much nicer looking final generated code, and less overhead.
• Haxe focused/maintained, example: low level parts of the Go’s standard library is being written by hand, in Haxe (os,fmt,math etc)
• Haxe has improved immensly since 2016, specfically for this project: unicode support, rest arguments, module level functions etc.
• Language features in Go and not in Haxe are supported via macros such as multireturns, defers, gotos, pointers etc.

Current state:

• Can generate all expressions/statements, except one’s releating to goroutines
• Able to run simple scripts that use parts of the standard library that are written.
• About 8% of the low level std is written in Haxe now.
• Macro language features, most are working and some need improvements.
• Very much a work in progress and still requires much more tests to pass before it can begin taking shots at full Go libraries.

Goals:

• 20 times the amount of Haxe libraries across all Haxe targets.
• Make Haxe much more competitive as a server sided language.
• Clean and fast exucting generated code.
• full acess to the Golangs test suite, thereby transpile the tests and test them in Haxe
• Ease of use following dts2hx’s example.

So you may be asking what does the code generation look like? Something like this:

PXshadow/go2hx

Most of the remaining work will be in Haxe either by writing parts of Go’s standard library, or tweaking and working on macros,

If that sounds interesting to you I’m very much looking for devs to work with this on, feel free to contact me (here/discord/etc).

P.S
Super big shout out to the Haxe compiler team, this project could not have been possible to make a year ago and am fully indebeted by the amazing strides you guys have taken.

Cheers and to a year of Haxe in 2021

[nadako (Dan Korostelev)]

Whoa, this is very impressive! Definitely going to check this out!

[RealyUniqueName (Aleksandr Kuzmenko)]

Impressive!
Have you compared output code size and performance to gopherjs?

[PXshadow]

Thank you so much! I have done comparison of output size for gopherjs vs go2hx, it’s pretty striking how insanely massive gopherjs here is the output:

https://gist.github.com/PXshadow/7c3ab1568a529d4e583563bf14906f27#file-go2hx-js

• gopherjs: 1080 kb, 20k loc
• go2hx: 4kb, 158 loc

This is what Gopherjs wrote on code size here:
" Why is GopherJS generated code so big?

GopherJS reimpliments most of the Go runtime in Javascript, so that’s a start. A lot of things import the fmt and reflect packages, which represents a sizable chunk of code. In general, GopherJS code is so large for reasons similar to the reason Go executables are so large: they’re statically linked. In the Go case, literally; in the GopherJS case, metaphorically speaking, but the idea is similar."

Haxe’s DCE seems much stronger then gopherjs’s and the design diffrence of writing the low level part of Go’s std greatly minimizes the code size as well. I have not dealt with inputted go code using reflect yet so that may throw a wrench in DCE, however I’m sure macros can fill the gaps and explicity set to keep fields and classes.

As for performance, I have not tested it out against gopherjs yet, if you have any benchmark tests let me know. I think gopherjs will be more comparable performance wise the more mature go2hx gets, such as comparing coroutines, http requests, json encoding etc.

[alex_dja (Alexander Djafarov)]

Cool stuff! It would also be nice to have golang as a haxe target:)

[PXshadow]

It would be nice to have a golang target I agree. Go’s assembly table generation is very neat and it would be amazing to easily produce one file executables for Haxe on all operating systems.The down side with only doing that approach is target locking the libraries to Go, and having a target that doesnt have that great of a c bindings system performance wise, which would hinder thing such as haxe’s future use of libuv. If there’s anything else I’m missing let me know

[rivetweb (X Y)]

this is transpiler haxe to go
or go to haxe?

[PXshadow]

The latter, it converts the go programming language into haxe code.

[darmie ('Damilare Darmie Akinlaja)]

I will try to use this to convert the LifeVM project to Haxe… Perhaps my dream of having WebAssembly run smoothly on Haxe will come sooner.

I have a feeling that some standard go library like TeeReader will pose a problem.

[PXshadow]

Great to hear! WebAssembly running smoothly in Haxe would be a dream come true for me as well, currently the transpiler is no where near close enough to generate libraries that use any of the standard library at the moment. The current approach is to use a mix of automatically transpiled and hand written code for the standard library (for parts that extensively use syscalls, internals or the runtime) io.TeeReader from a quick glance looks like it makes sense to be completly auto compiled. When you attempt the conversion please feel free to open as many issues on the github you run into, and any help on the compiler is more than welcome. I look forward to the results.

[rainy]

Is it possible for Haxe to compile Go?

[PXshadow]

If you mean without the depedency of Go to compile Haxe to Go then yes when the bootstrap process gets finished and the 6% of the project in Go can be compiled into Haxe code. If you mean a Go target for Haxe, it’s possible however it’s not something I’m working on.

[PXshadow]

It’s been about 8 months since the initial post, so a progress update is due!

Quick recap of project:

go2hx is a golang to Haxe source-to-source compiler (like Haxe its self) written in Haxe with the main objective to compile as many golang libraries into Haxe as possible for use by Haxe developers.

The reason for this objective is because Haxe lacks a large collection of libraries to be able to use across its many targets portably. Requiring an unfortanate choice of library access vs portability.

Progress:

• The compiler has become quite a bit more sophisticated ditching a lot of the macro exprs that would work as a poor filler for not knowing type info. Now the compiler directly gets the golang’s ast type info and uses it for type analysis to generate much more proper idiomatic code.

• Entire tinygo test suite passing except for tests that use time/runtime standard libraries.

• The go2hx reflection standard library is able to handle almost all runtime inspection.

• Basic types have greatly improved from go’s slices/arrays/maps/pointers and numbers.

• A decent portion of go2hx’s go language generation is spec compliant, and more tests coming to increase it.

• Can now install go2hx with Haxelib via haxelib git go2hx https://github.com/go2hx/go2hx

• Created some github wiki articles example

Current status:

• 1000 tests, test suite is being setup to run for github actions (hoping to multi-thread but might not be feasible at the moment).

• Working towards compiling simple libraries and building up a list of supported ones to increase adoption of the compiler before more spec compliance is met.

• More work on writing parts of go’s standard library to pass tests is being done.

Interesting showcase parts:

Modified the haxe-test-adapter to be able to set the line number of tests and offset to match an array entry.

Built a simple go2hx REPL that takes golang code as input and outputs Haxe code and runs it.

Any feedback is extremely appreciated, you can write here and/or create an issue on the repo.

I hope the REPL gives a nice introduction to the compiler and allows some early on experimentation.

Thanks for taking the time to read, the generous kind words have really helped keep the drive strong to finish this project and bring it to life, I look forward to sharing many more posts in the future (in smaller time intervals then the current precedent!)

[PXshadow]

Long overdue update below:

4 years of go2hx

A little backstory:

This project has been the essence of my dream for Haxe…

Haxe with a large and robust cross-target library ecosystem.

This idea has so captivating for me that I have been working on this project for years with very little reward because in my head it must be completed.

Especially now given my professional work is now stuck with using Golang on the back-end and JS/TS on the front-end, for me personally a world without a completed go2hx would be a tragedy.

Ecosystem size differences on github: 7.7k Haxe vs 309k Go

So what now?

Q1 2024 will be marked as the release of go2hx because it needs to happen, otherwise the project will die.

The release will be bumpy, and almost all 3rd party libraries and most stdlibs will still not work, but a release will give more life to the project and a timeline will help guide the way.

The life will come from interested devs using it, and also seeing the vision and what this project could mean for Haxe with more contributions.

Contribution

The compiler simply needs more devs improving it, the project is big with plenty of exciting parts, it needs specialized interest in specific problems or features. You there reading, could be the exact right person for one of them, for example!

• You like passing small tests and making improvements to a compiler written in Haxe?
• Interested in C libraries and want access to thousands of them with cgo?
• Don’t like the Go dependency of go2hx and want to try out bootstrapping?
• Target X (JS, JVM etc) is not supported and you want to use go2hx libs with it?
• Pointers are all simulated, and you think it would be great to use the native pointer for X target?
• Love Go routines and want to make them better in Haxe/go2hx?
• Performance is bad for X case and you want to make it faster?

If one of these or something else interests you, reach out to me, here, on github or on the Haxe discord

Finally I’d like to say I am super grateful for the Haxe team for building such a wonderful and stable language, none of it would be possible without them and the continued support.