Error defining typedef in macro context

Hello everyone.

I am currently building a form generation library that auto-generates code for both the client and server for use on websites. I have managed to successfully parse a simple text file into what’s called CField's and from here can begin to generate code for either JavaScript (simple typedef's) or PHP (classes extending sys.db.Object).

I am attempting to test the Context.defineType() function on a simple typedef for JavaScript context but having the following error when compiling:

C:\HaxeToolkit\haxe\std/haxe/macro/Context.hx:471: characters 2-27 : Invalid type definition
source/client/Parser.hx:246: characters 8-38 : Called from
source/client/Parser.hx:175: characters 16-36 : Called from
--macro:1: character 0 : Called from
Aborted

I’m not entirely sure why this is happening. I have tested the parsing and I know it parses the text correctly.

I have the full code for this macro below:

#if macro
package;

import haxe.macro.Context;
import haxe.macro.Expr;
import haxe.macro.Type;

import sys.FileSystem;
import sys.io.File;

class CField
{

    public var type:String;
    public var fieldName:String;
    public var hidden:Bool;
    public var autoincrement:Bool;
    public var dbtype:String;

    public var options:Map<String, String>;

    public function new()
    {
        options = new Map<String, String>();
    }

}

class Parser
{

    private var _fields:Array<CField>;
    public var fields(get, never):Array<CField>;
    function get_fields() return _fields;

    public var tableName:String;

    public function new(file:String)
    {
        _fields = [];

        parse(file);
    }

    public function parse(file:String)
    {
        var start = file.lastIndexOf('/') + 1;
        tableName = file.substr(start, file.lastIndexOf('.') - start);


        var content = File.getContent(file);
        var lines = content.split("\r\n");
        for (i in 0...lines.length)
        {
            var line = lines[i];
            var firstWord = true;
            var field = new CField();

            var wordData = getNextWord(line);
            var remaining = wordData.remaining;
            var isAString = false;
            var stringValue = "";

            // check if the remainder of the line has any more text to parse
            while (wordData.word != "")
            {
                // word begins with '@' symbol and is the first word,
                // it is an identifier representing the type of an object
                if (wordData.word.indexOf("@") == 0 && firstWord && !isAString)
                {
                    field.type = wordData.word.substr(1);
                }
                // word begins with ':' symbol, denoting an option
                else if (wordData.word.indexOf(":") == 0 && !isAString)
                {
                    // options must be part of a field identifier
                    if (line.indexOf("@") != 0 || firstWord)
                    {
                        throw '$file ($i): Field Options cannot appear where a field has not been declared.';
                    }

                    // key value pair, add the option to `options`.
                    if (wordData.word.indexOf("=") > -1)
                    {
                        var kv = wordData.word.split("=");
                        field.options.set(tableName + "_" + kv[0].substr(1), kv[1]);
                    }
                    // if no '=' sign exists, it is a variable.
                    // check if it exists in the field and apply it
                    else
                    {
                        var value = wordData.word.substr(1);
                        // the actual field type we're checking must be a boolean,
                        // otherwise this won't work.
                        if (Reflect.hasField(field, value))
                        {
                            Reflect.setField(field, value, true);
                        }
                    }
                }
                // check if the word begins with '"', if so it's a string.
                // for now, we assume it's only for naming the field.
                else if (wordData.word.indexOf('"') == 0)
                {
                    // if the same word also ends in quotations, just grab the text in between.
                    if (wordData.word.lastIndexOf('"') == wordData.word.length - 1)
                    {
                        field.fieldName = wordData.word.substr(1, wordData.word.length - 2);
                    }
                    else
                    {
                        isAString = true;
                        stringValue = wordData.word.substr(1);
                    }
                }
                else if (isAString)
                {
                    if (wordData.word.indexOf('"') == wordData.word.length - 1)
                    {
                        isAString = false;
                        stringValue += wordData.word.substr(0, wordData.word.length - 1);
                        field.fieldName = stringValue;
                        stringValue = "";
                    }
                    else
                    {
                        stringValue += wordData.word;
                    }
                }

                firstWord = false;
                wordData = getNextWord(remaining);
                remaining = wordData.remaining;
            }

            _fields.push(field);
        }
    }

    function getNextWord(line:String)
    {
        var word = "";
        var remaining = "";
        for (i in 0...line.length)
        {
            if (line.charAt(i) == " ")
            {
                remaining = line.substr(i + 1);
                break;
            }
            else
                word += line.charAt(i);
        }

        return { word: word, remaining: remaining };
    }


/*
* Begin Macros
*/

    public static function build(folder:String)
    {
        var files = FileSystem.readDirectory(folder);
        for (i in 0...files.length)
        {
            var f = files[i];
            var isDir = FileSystem.isDirectory(folder + f);

            if (!isDir)
            {
                // for now, we are only checking the root of the specified folder
                var parser = new Parser(folder + f);
                build_shared(parser);

            }
        }
    }

    private static function build_server(folder:String)
    {
        // generates a class that extends `sys.db.Object` and their relevant
        // database types.

        
    }

    private static function build_shared(parser:Parser)
    {
        // generates a typedef to be used in client and server context
        // Typedefs can be directly used to add structure
        // for JSON objects.

        var fields = parser.fields;
        var _tfields = new Array<Field>();
        for (i in 0...fields.length)
        {
            var f:CField = fields[i];
            switch (f.type)
            {
                case "ID", "Numeric":
                    _tfields.push({
                        kind: FVar(macro:Int),
                        name: f.fieldName,
                        pos: Context.currentPos(),
                        access: [APublic],
                        meta: [{
                            name: ":optional",
                            pos: Context.currentPos()
                        }]
                    });
                case "TextField", "TextArea", "Markdown", "Code":
                    _tfields.push({
                        kind: FVar(macro:String),
                        name: f.fieldName,
                        pos: Context.currentPos(),
                        access: [APublic],
                        meta: [{
                            name: ":optional",
                            pos: Context.currentPos()
                        }]
                    });
                default:
                    _tfields.push({
                        kind: FVar(macro:Dynamic),
                        name: f.fieldName,
                        pos: Context.currentPos(),
                        access: [APublic],
                        meta: [{
                            name: ":optional",
                            pos: Context.currentPos()
                        }]
                    });
            }
        }

        var definition:TypeDefinition = {
            fields: [],
            kind: TDAlias(TAnonymous(_tfields)),
            name: "T" + parser.tableName,
            pack: ["shared"],
            pos: Context.currentPos()
        };

        Context.defineType(definition);
    }

/*
* End Macros
*/

}

#end

I suppose the focus of attention is on the very last few lines where the problem lies, but I’ve never really used TypeDefinition before. I would much appreciate some assistance.

The TypeDefinition looks fine.

You should try with kind: TDAlias(TAnonymous([])), the fields are the only things that looks like they could be a problem, though they do look fine too.

Okay, interesting. So changing to code to:

var definition:TypeDefinition = {
            fields: [],
            kind: TDAlias(TAnonymous([])),
            name: "T" + parser.tableName,
            pack: ["shared"],
            pos: Context.currentPos()
        };

allows to macro to compile, but as expected the following error comes up:

source/client/Main.hx:11: characters 8-15 : shared.TUser has no field ID

Attempting to add fields to the type definition in any way causes the error. Also, I am using --macro Parser.build("data/") at compile time to use initialisation macros. Would it be better to use the @:build() command?

A build macro would work too, I don’t think there’s much difference here.

Try tracing your _tfields aray, I guess there’s something wrong with it,
maybe one of them has a name that is null?

Also not sure if you have the correct FVar, it should have a second argument haxe.macro.FieldType - Haxe 4.2.1 API though if it compile I guess you do?

Ah yes.

Well, this is the result of the trace:

source/client/Parser.hx:239: [{ kind => FVar(TPath({ name => Int, pack => [], params => [] }),null), meta => [{ name => :optional, pos => #pos(--macro:1: character 0) }], name => ID, pos => #pos(--macro:1: character 0), access => [APublic] },{ kind => FVar(TPath({ name => Dynamic, pack => [], params => [] }),null), meta => [{ name => :optional, pos => #pos(--macro:1: character 0) }], name => null, pos => #pos(--macro:1: character 0), access => [APublic] }]

The second parameter of FVar is indeed null, so I suspect the Context needs an expression there? What expression would it need to complete the kind because I’ve never needed to do this in the past (that being said, I’ve never used defineType before until now).

I found the problem! It had nothing to do with the code, it had to do with the fact that I had an empty line in the file and I interpreted it as a field… It was therefore creating an empty field with a null name that was then being added to the type definition. It’s working now.

I’ve corrected the interpretation to check if the name is valid and a type exists.