Spaces:
Running
Running
File size: 1,973 Bytes
5f923cd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | # How to Generate a Parser using ANTLR4
## Overview
[`antlr4rust`](https://crates.io/crates/antlr4rust) is a Rust library that
provides a Rust runtime for using parsers generated by
[**ANTLR4**](https://www.antlr.org/).
Parsers are generated from ANTLR `.g4` grammar files by the
[ANTLR4 tool](https://github.com/antlr4rust/antlr4/tree/rust-target/tool).
LiteRT-LM uses ANTLR parsers to parse tool call expressions generated by LLMs.
## Instructions
These are instructions for generating the Rust source files for the lexer and
parser from ANTLR `.g4` files. It's assumed you've already written `.g4` files
that define the grammar for the language you want to parse.
In the example code, the lexer is defined in `AntlrFcLexer.g4` and parser is
defined in `AntlrFcParser.g4`.
1. Build the ANTLR4 tool using Maven or download it from
https://github.com/antlr4rust/antlr4/releases/.
1. Make a directory for the generated files.
```bash
mkdir generated
```
1. Run the ANTLR4 tool to generate source files for the lexer and parser from
the `.g4` files:
```bash
java -jar /path/to/antlr4-4.13.3-SNAPSHOT-complete.jar -Dlanguage=Rust AntlrFcLexer.g4 AntlrFcParser.g4 -o generated
```
1. Write a simple Rust source file that declares the generated modules, e.g.
create and edit `generated/antlr_fc_tool_call_parser.rs`:
```rust
//! This crate contains the generated ANTLR4 Rust modules for parsing FC tool calls.
pub mod antlrfclexer;
pub mod antlrfcparser;
pub mod antlrfcparserlistener;
```
1. Write a `rust_library` target in `generated/BUILD`:
```rust
rust_library(
name = "antlr_fc_tool_call_parser",
srcs = [
"antlr_fc_tool_call_parser.rs",
"antlrfclexer.rs",
"antlrfcparser.rs",
"antlrfcparserlistener.rs",
],
deps = [
"//third_party/rust/antlr4rust/v0_5:antlr4rust",
],
)
```
|