How to visit a certain sequence of AST nodes in Aurelia source code via Babel.js 7+?

[HamedFathi]

Hello,
I need to parse some Aurelia source code files.
For this purpose I chose Babel.js 7+.
The AST result completely covers my needs.

import babel = require("@babel/parser");
let ast = babel.parse(code, {
  sourceType: "module",
  plugins: [
    'jsx',
    'typescript',
    'classProperties',
    ['decorators', { decoratorsBeforeExport: true }]
  ]
});

The source code is:

export class RgbToHexValueConverter {
    toView(rgb: IColor): string {
        return "#" + (
            (1 << 24) + (rgb.r << 16) + (rgb.g << 8) + rgb.b).toString(16).slice(1);
    }

    fromView(hex: string): IColor {
        let exp = /^#?([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})$/i,
        result = exp.exec(hex);
        return {
            r: parseInt(result[1], 16),
            g: parseInt(result[2], 16),
            b: parseInt(result[3], 16)
        };
    }

    get fullName(): string {
        return "THENAME";
    }
}

class Test {
    getInfo(input: any): any {}
    getData(input: any): any {}
}

interface IColor {
    r: number;
    g: number;
    b: number;
}

To achieve the result I found a way that is named jsonpath

$..program.body[0].declaration.body.body[0].key.name 

But I want to use native Visitors. Is it possible?

I want to implement a scenario with visitors :

If you foundExportNamedDeclaration then go to declaration to find ClassDeclaration type.
If you don’t, Are you in ClassDeclaration ?
If yes, then go to body and find ClassBody then again body
And for each ClassMethod go to the key and return the name.
With this you can get all name methods in a class.

ExportNamedDeclaration > ClassDeclaration > ClassBody > 
ClassMethod > Identifier > [getInfo, getData]

How can I get above result by visitors? Or what is the best/correct way to get info form AST in Babel.js 7+?

[bigopon]

Here is an example with your source code in AST explorer: https://astexplorer.net/#/gist/e9f3a3a1dc8373d3307f61260b59288e/d1114d8766a971052bde59f3519efa0d95c112bc

You can see at bottom left panel, how visitor pattern is being used:

Maybe start experiencing with that?

[HamedFathi]

@bigopon

Thanks, I know about this

export default function (babel) {
  const { types: t } = babel;
  
  return {
    name: "ast-transform", // not required
    visitor: {
      Identifier(path) {
        path.node.name = path.node.name
      }
    }
  };
}

My problem is about chaining between visitors. In your sample, Identifier() returns everything in AST but I need a chain of visitors. (a related and limited result)

Visit ExportNamedDeclaration after that find ClassDeclaration inside it and again find the node inside it and so on.

Identifier in your sample returns many ids but I want to have nested visitors. something like

ExportNamedDeclaration(path)
{
    ClassDeclaration(path)
    {
            ClassBody (path)
               {
                     ClassMethod (path)
                     {
                           ...
                    }
             }
    }
}

In fact I want to decrease the scope of path and send it to another visitor.
I want to find related tokens
Class Name > Methods of this Class > parameters of each methods
I think this scenario is not linear, it is nested/conditional about some scoped and related tokens but all examples I found in the web was linear!

export default function (babel) {
  const { types: t } = babel;
  
  return {
    name: "ast-transform", // not required
    visitor: {
      ClassMethod (path) {
        path.node.name = path.node.name
      }
    }
  };
}

With above code I can find all methods but I can not get some related information like
Was it in Export class?
Was it in class X or class Y?

[huochunpeng]

Use my ast-matcher which is designed for normal people (me!) to do AST analysis.

const babel = require('@babel/parser');
const astMatcher = require('ast-matcher');

astMatcher.setParser(code => {
  const babelAst = babel.parse(code, {
    sourceType: "module",
    plugins: [
      'jsx',
      'typescript',
      'classProperties',
      ['decorators', { decoratorsBeforeExport: true }]
    ]
  });
  // babel ast tree got different shape than other estree parsers.
  // I found I need this to match other parsers.
  return babelAst && babelAst.program;
});

const code = `
export class RgbToHexValueConverter {
    toView(rgb: IColor): string {
        return "#" + (
            (1 << 24) + (rgb.r << 16) + (rgb.g << 8) + rgb.b).toString(16).slice(1);
    }

    fromView(hex: string): IColor {
        let exp = /^#?([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})$/i,
        result = exp.exec(hex);
        return {
            r: parseInt(result[1], 16),
            g: parseInt(result[2], 16),
            b: parseInt(result[3], 16)
        };
    }

    get fullName(): string {
        return "THENAME";
    }
}

class Test {
    getInfo(input: any): any {}
    getData(input: any): any {}
}

interface IColor {
    r: number;
    g: number;
    b: number;
}
`;

const matcher = astMatcher('export class __any_name { __anl_body }');

// return an array of matches.
// if you change above pattern from 'export class...' to 'class...',
// you can get two matches.
matcher(code).forEach(r => {
  // r.match.name is the ast node captured by __any_name
  console.log(r.match.name.name);
  // r.match.body is array of ast nodes captured by __anl_body
  console.log(r.match.body.map(n => n.key.name)) 
});

// Then you can use another pattern to work on every node in body.
// anotherMatch = astMatcher('...');
// anotherMatch(body[0])

The terminal output:

⋊> ~/p/tt node index.js                                                                                                                                                      13:00:09
RgbToHexValueConverter
[ 'toView', 'fromView', 'fullName' ]

More doc here: https://github.com/dumberjs/ast-matcher

[HamedFathi]

@huochunpeng
Thank you as always,
None of the members of the Aurelia team are normal, You are awesome.

[huochunpeng]

Thx.

ast-matcher was created because the laziness of me. When I study bundler, I read lots of AST analysis code, they are very very tedious. I told myself that’s too much for me to comprehend, I have to find way to reduce the complexity.

ast-matcher is only 200 lines of code, but it took me few days (if not more than a week) to get it right. In return, it saved me much more time.

[zewa666]

We’re using tsquery (https://github.com/phenomnomnominal/tsquery) in our custom tslint rules which makes querying the ast for complex expressions easier. Since you’re targeting ts code that might be a fit

[HamedFathi]

@zewa666

Thanks,

@fkleuver encouraged me to use Typescript compiler API against Babel. Recently I found ts-morph too.
I am working on a proposal based on personal ideas for building a document generator on Aurelia-based projects, something like Compodoc for Angular so I need gathering some information from the source code via parsers.

[brandonseydel]

ts-morph looks like something that would make writing aot pretty damn simple. Good abstraction over typescript apis.

[fkleuver]

The binding / type checking APIs are a bit hairy and hard to reach, but typescript already has a lot of built-in API surface for traversing and transforming the AST.
I’d much rather see a good abstraction over the control flow analysis APIs but those seem to be tucked away to be used by the type checker only. A bit of a shame.

Also, ts-morph looks like it would use heaps and heaps of extra memory and slow stuff down on very large projects. So many layers of objects…

It would be nice not to have to invent another wheel for our AOT but I also don’t want to build an AOT that is unbearably slow and causes medium-sized to large projects take half an hour to build (encounted such case with Angular recently)