3 min read

Json Walk

2020-02-23

github repo

On more than one occasion i’ve wanted a cli tool to extract values from json data. There are already tools available which provide pretty print, syntax highlighting, but i could not find something with equivelant of XML’s XPATH.

I came across a blog post on medium about Pico CLI, a library for creating CLI tools in Java, using annotations to define command arguments. It also mentioned an XCode tool to package the java as standalone application. So this gave me the idea to build a json CLI tool with Graalvm to produce a native binary.

The end resut is a native executable called jsonwalk. Here it is.

$ curl  "https://raw.githubusercontent.com/Biuni/PokemonGO-Pokedex/master/pokedex.json"  2>/dev/null \
  | ./jsonwalk  '$.pokemon[?( @.name == "Pikachu" )]'
[{"id":25,"num":"025","name":"Pikachu","img":"http:\/\/www.serebii.net\/pokemongo\/pokemon\/025.png","type":["Electric"],"height":"0.41 m","weight":"6.0 kg","candy":"Pikachu Candy","candy_count":50,"egg":"2 km","spawn_chance":0.21,"avg_spawns":21,"spawn_time":"04:00","multipliers":[2.34],"weaknesses":["Ground"],"next_evolution":[{"num":"026","name":"Raichu"}]}]

$ curl "https://pokeapi.co/api/v2/pokemon/25/" 2>/dev/null \
 |  ./jsonwalk '$..move.name' -o plain 
agility
attract
bide
body-slam
brick-break
captivate
charge-beam
confide
counter
covet
curse
defense
..

$ cat ../src/test/resources/sample.json | ./jsonwalk   "$..book..category" -o plain 
reference
fiction
fiction
fiction

Graalvm comes with its own command line tool to generate binaries but a Maven goal can be setup instead which typically is used in the package phase.

<plugin>
    <groupId>org.graalvm.nativeimage</groupId>
    <artifactId>native-image-maven-plugin</artifactId>
    <version>${graalvm.version}</version>
    <executions>
        <execution>
            <goals>
                <goal>native-image</goal>
            </goals>
            <phase>package</phase>
        </execution>
    </executions>
    <configuration>
        <skip>false</skip>
        <imageName>jsonwalk</imageName>
        <buildArgs>
                --no-fallback
                --report-unsupported-elements-at-runtime
        </buildArgs>
    </configuration>
</plugin>

This graalvm maven setup comes direct from the graal documentation with –report-unsupported-elements-at-runtime added to get around some compatibility problems.

To get picocli to play nice with graalvm we must add the picocli codegen module for the maven-compiler-plugin. Without this the annotations are ignored and arguments will not be passed properly at runtime.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>${maven.compiler.plugin.version}</version>
    <configuration>
        <source>${java.version}</source>
        <target>${java.version}</target>
        <showWarnings>true</showWarnings>
        <showDeprecation>true</showDeprecation>
        <annotationProcessorPaths>
        <path>
            <groupId>info.picocli</groupId>
            <artifactId>picocli-codegen</artifactId>
            <version>4.2.0</version>
        </path>
        </annotationProcessorPaths>
    </configuration>
</plugin>

In combination with Jayway JSON Path library we end up with a very simple single class implementation.

@Command(name = "jsonwalk", footer = "Copyright(c) 2020", description = "Traverse JSON with JSON Path.")
public class JsonWalk implements Runnable {

    private static final Logger logger = LoggerFactory.getLogger(JsonWalk.class);

    public static void main(String... args) {        
        new CommandLine(new JsonWalk()).execute(args);
    }

    @Parameters(index = "0", description = "JSON path to walk.")
    private String jsonPath;

    public void run() {
        String line = "";
        String submittedString = "";
        try {
            BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in));
            while ((line = stdin.readLine()) != null) {
                submittedString += line + '\n';
            }

            Object result = JsonPath.read(submittedString, jsonPath);
            if (result instanceof Collection) {
                for (Object item : (Collection)result) {
                    System.out.println(item.toString());                
                }
            } else  {
                System.out.println(result.toString());
            }

        } catch (IOException e) {
            e.printStackTrace();
        }
     }
     
}

Next steps:

  1. Add some more unit test to cover more cases.
  2. Distribute via homebrew, chocolatey, yum, aptitude…
  3. Add features.

If you want to contribute head over to the repo, check out the Issues if you are short of ideas.