jq - a really useful unix utility

Hot on the heels of the last post is a quick note on JQ - thats not oracle job queues before you ask it's a unix utility that can parse json files and pull out individual elements - think of it as sed/awk for json files.

I've only scratched the surface with it today but though it worth sharing as it looks like it could be a very handy utility

installation is simple via this command (this assumes that you have the epel repo set up i mentioned in my previous post)

 yum install jq


Once installed you can use it to read your json files and do loads of things - the help text for it is here

 jq
jq - commandline JSON processor [version 1.5]
Usage: jq [options] <jq filter> [file...]

        jq is a tool for processing JSON inputs, applying the
        given filter to its JSON text inputs and producing the
        filter's results as JSON on standard output.
        The simplest filter is ., which is the identity filter,
        copying jq's input to its output unmodified (except for
        formatting).
        For more advanced filters see the jq(1) manpage ("man jq")
        and/or https://stedolan.github.io/jq

        Some of the options include:
         -c             compact instead of pretty-printed output;
         -n             use `null` as the single input value;
         -e             set the exit status code based on the output;
         -s             read (slurp) all inputs into an array; apply filter to it;
         -r             output raw strings, not JSON texts;
         -R             read raw strings, not JSON texts;
         -C             colorize JSON;
         -M             monochrome (don't colorize JSON);
         -S             sort keys of objects on output;
         --tab  use tabs for indentation;
         --arg a v      set variable $a to value <v>;
         --argjson a v  set variable $a to JSON value <v>;
         --slurpfile a f        set variable $a to an array of JSON texts read from <f>;
        See the manpage for more options.

I just wanted to pull out some text from the json output from an azure command which i did as follows:

The normal azure list command has a --json switch to return the output in json format - so for a list of vm's you run this

azure vm list --json

The output of this is very verbose json which i won't post here.

I wanted to pull out some of the elements from this (there are loads) and just shown it in a simple csv like format

after a lot of trial and error and working out the syntax for how jq was working i came up with this command

 azure vm list --json |jq ".[] | .type +\",\"+ .powerState +\",\"+ .storageProfile.imageReference.sku"

This pulls out 3 elements from the azure command output in csv format - an extract of which is shown below


"Microsoft.Compute/virtualMachines,VM running,12-SP1"
"Microsoft.Compute/virtualMachines,VM running,2012-R2-Datacenter"
"Microsoft.Compute/virtualMachines,VM running,12-SP1"
"Microsoft.Compute/virtualMachines,VM running,OL70"
"Microsoft.Compute/virtualMachines,VM running,2012-R2-Datacenter"

I can't pretend to really have grasped the full in's and out's of the jq syntax but for the moment this gives me what i want.

jq looks like a very useful utility and one I've not seen mentioned before.