有没有一个简单的例子使用antlr4从java源代码创建AST并提取方法,变量和注释?[已关闭]

2022-09-02 12:47:32

有人可以提供一个详细的例子来说明我如何使用antlr4做到这一点吗?关于安装 antlr4 及其依赖项的说明将不胜感激。


答案 1

在这里。

首先,您将购买ANTLR4书;-)

其次,您将下载antlr4 jar和java语法(http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference)

然后,您可以稍微更改语法,将这些语法添加到标题中

    (...)
grammar Java;

options 
{
    language = Java;
}

// starting point for parsing a java file
compilationUnit
    (...)

我将更改语法中的一点内容,只是为了说明一些事情。

/*
methodDeclaration
    :   (type|'void') Identifier formalParameters ('[' ']')*
        ('throws' qualifiedNameList)?
        (   methodBody
        |   ';'
        )
    ;
*/
methodDeclaration
    :   (type|'void') myMethodName formalParameters ('[' ']')*
        ('throws' qualifiedNameList)?
        (   methodBody
        |   ';'
        )
    ;

myMethodName
    :   Identifier
    ;

你看,原始语法不允许您从任何其他标识符中识别方法标识符,所以我注释了原始块并添加了一个新块,只是为了向您展示如何获得所需的内容。

您必须对要检索的其他元素(如当前刚刚跳过的注释)执行相同的操作。这是给你的:-)

现在,创建一个这样的类来生成所有存根

package mypackage;

public class Gen {

    public static void main(String[] args) {
        String[] arg0 = { "-visitor", "/home/leoks/EclipseIndigo/workspace2/SO/src/mypackage/Java.g4", "-package", "mypackage" };
        org.antlr.v4.Tool.main(arg0);
    }

}

运行Gen,您将在mypackage中获得一些为您创建的java代码。

现在创建一个访问者。实际上,在此示例中,访问者将解析自身

package mypackage;

import java.io.FileInputStream;
import java.io.IOException;

import mypackage.JavaParser.MyMethodNameContext;

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;

/**
 * @author Leonardo Kenji Feb 4, 2014
 */
public class MyVisitor extends JavaBaseVisitor<Void> {

    /**
     * Main Method
     * 
     * @param args
     * @throws IOException
     */
    public static void main(String[] args) throws IOException {
        ANTLRInputStream input = new ANTLRInputStream(new FileInputStream("/home/leoks/EclipseIndigo/workspace2/SO/src/mypackage/MyVisitor.java")); // we'll
                                                                                                                                                    // parse
                                                                                                                                                    // this
                                                                                                                                                    // file
        JavaLexer lexer = new JavaLexer(input);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        JavaParser parser = new JavaParser(tokens);
        ParseTree tree = parser.compilationUnit(); // see the grammar ->
                                                    // starting point for
                                                    // parsing a java file



        MyVisitor visitor = new MyVisitor(); // extends JavaBaseVisitor<Void>
                                                // and overrides the methods
                                                // you're interested
        visitor.visit(tree);
    }

    /**
     * some attribute comment
     */
    private String  someAttribute;

    @Override
    public Void visitMyMethodName(MyMethodNameContext ctx) {
        System.out.println("Method name:" + ctx.getText());
        return super.visitMyMethodName(ctx);
    }

}

仅此而已。

你会得到类似的东西

Method name:main
Method name:visitMyMethodName

还有一件事。当我在eclipse中编写此代码时,我有一个奇怪的例外。这是由Java 7引起的,只需将这些参数添加到编译器中即可修复(多亏了此链接 http://java.dzone.com/articles/javalangverifyerror-expecting)

enter image description here


答案 2
grammar Criteria;

@parser::header {
  import java.util.regex.Pattern;
}

options
{
  superClass = ReferenceResolvingParser;
}

@parser::members {

  public CriteriaParser(TokenStream input, Object object) {
    this(input);
    setObject(object);
  }

}

/* Grammar rules */

reference returns [String value]
          : '$.' IDENTIFIER { $value = resolveReferenceValue($IDENTIFIER.text); }
          ;

operand returns [String value]
        : TRUE { $value = $TRUE.text; }
        | FALSE { $value = $FALSE.text; }
        | DECIMAL { $value = $DECIMAL.text; }
        | QUOTED_LITERAL  { $value = $QUOTED_LITERAL.text.substring(1, $QUOTED_LITERAL.text.length() - 1); }
        | reference { $value = $reference.value; }
        ;

operand_list returns [List value]
             @init{ $value = new ArrayList(); }
             : LBPAREN o=operand { $value.add($o.value); } (',' o=operand { $value.add($o.value); })* RBPAREN
             ;

comparison_expression returns [boolean value]
                      : lhs=operand NEQ rhs=operand { $value = !$lhs.value.equals($rhs.value); }
                      | lhs=operand EQ rhs=operand { $value = $lhs.value.equals($rhs.value); }
                      | lhs=operand GT rhs=operand { $value = $lhs.value.compareTo($rhs.value) > 0; }
                      | lhs=operand GE rhs=operand { $value = $lhs.value.compareTo($rhs.value) >= 0; }
                      | lhs=operand LT rhs=operand { $value = $lhs.value.compareTo($rhs.value) < 0; }
                      | lhs=operand LE rhs=operand { $value = $lhs.value.compareTo($rhs.value) <= 0; }
                      ;

in_expression returns [boolean value]
              : lhs=operand IN rhs=operand_list { $value = $rhs.value.contains($lhs.value); };

rlike_expression returns [boolean value]
                 : lhs=operand RLIKE rhs=QUOTED_LITERAL { $value = Pattern.compile($rhs.text.substring(1, $rhs.text.length() - 1)).matcher($lhs.value).matches(); }
                 ;

logical_expression returns [boolean value]
                   : c=comparison_expression { $value = $c.value; }
                   | i=in_expression { $value = $i.value; }
                   | l=rlike_expression { $value = $l.value; }
                   ;

chained_expression returns [boolean value]
                   : e=logical_expression { $value = $e.value; } (OR  c=chained_expression { $value |= $c.value; })?
                   | e=logical_expression { $value = $e.value; } (AND c=chained_expression { $value &= $c.value; })?
                   ;

grouped_expression returns [boolean value]
                   : LCPAREN c=chained_expression { $value = $c.value; } RCPAREN ;

expression returns [boolean value]
           : c=chained_expression { $value = $c.value; } (OR  e=expression { $value |= $e.value; })?
           | c=chained_expression { $value = $c.value; } (AND e=expression { $value &= $e.value; })?
           | g=grouped_expression { $value = $g.value; } (OR  e=expression { $value |= $e.value; })?
           | g=grouped_expression { $value = $g.value; } (AND e=expression { $value &= $e.value; })?
           ;

criteria returns [boolean value]
         : e=expression { $value = $e.value; }
         ;


/* Lexical rules */

AND : 'and' ;
OR  : 'or' ;

TRUE  : 'true' ;
FALSE : 'false' ;

EQ    : '=' ;
NEQ   : '<>' ;
GT    : '>' ;
GE    : '>=' ;
LT    : '<' ;
LE    : '<=' ;
IN    : 'in' ;
RLIKE : 'rlike' ;

LCPAREN : '(' ;
RCPAREN : ')' ;
LBPAREN : '[' ;
RBPAREN : ']' ;

DECIMAL : '-'?[0-9]+('.'[0-9]+)? ;

IDENTIFIER : [a-zA-Z_][a-zA-Z_.0-9]* ;

QUOTED_LITERAL :
                 (  '\''
                    ( ('\\' '\\') | ('\'' '\'') | ('\\' '\'') | ~('\'') )*
                 '\''  )
                ;

WS : [ \r\t\u000C\n]+ -> skip ;



public class CriteriaEvaluator extends CriteriaBaseListener
{

    static class CriteriaEvaluatorErrorListener extends BaseErrorListener
    {

        Optional<String> error = Optional.empty();

        @Override
        public void syntaxError(Recognizer<?, ?> recognizer, Object offendingSymbol, int line, int charPositionInLine, String msg, RecognitionException e) {
            error = Optional.of(String.format("Failed to parse at line %d:%d due to %s", line, charPositionInLine + 1, msg));
        }

    }

    public static boolean evaluate(String input, Object argument)
    {
        CriteriaLexer lexer = new CriteriaLexer(new ANTLRInputStream(input));
        CriteriaParser parser = new CriteriaParser(new CommonTokenStream(lexer), argument);
        parser.removeErrorListeners();
        CriteriaEvaluatorErrorListener errorListener = new CriteriaEvaluatorErrorListener();
        lexer.removeErrorListeners();
        lexer.addErrorListener(errorListener);
        parser.removeErrorListeners();
        parser.addErrorListener(errorListener);
        CriteriaParser.CriteriaContext criteriaCtx = parser.criteria();
        if(errorListener.error.isPresent())
        {
            throw new IllegalArgumentException(errorListener.error.get());
        }
        else
        {
            return criteriaCtx.value;
        }
    }

}

推荐