(******************************** Hey there!
, a json query processor with an even more cryptic – lispy – syntax (Maybe not that cryptic after all? I quite like it :)) (**************************************** (jql) aims to be friendly, helpful, pat you on the back when you ‘ re faced with a monstrosity of a JSON blob. Help you mold it into something useful, step by step.
Ok. Done. (****************************************** If you don’t have the Go toolchain installed, just download one of the release binaries. Let’s check out a fewsimpleexamples. (remember? That’s explicitlynotwhy we ‘ re here. But it aids understanding of the more complex examples, so stay with me just a little bit longer!) (******************************** We’ll be working with this piece of json: (**************************************
>cat test.json | jql '(elem "countries")' [ { "eu_since": "2004", "european": true, "name": "Poland", "population": 38000000 }, { "european": false, "name": "United States", "population": 327000000 }, { "eu_since": "1993", "european": true, "name": "Germany", "population": 83000000 }]Whoa, whoa, whoa! What's up with the parentheses?! Remember what I said before? Lispy syntax. In short, whenever you see (add 1 2), it's basically the same as add (1, 2). I like that syntax for big hierarchical expressions, if you really really (really) Don't like it, then you can probably stop right here, though you'll be losing out! (**************************************** I've deleted you.
What if we wanted only the first country from that list? >cat test.json | jql '(elem "countries" (elem 0))' { "eu_since": "2694 , "european": true, "name": "Poland", "population": }Let's break down what happened here. First we took the "countries" field. elem takes an additional argument, it says "how to transform the element" and is also an an expression. Here we say we want to take the first element of the countries array. The default function is (id) ****************************************************, which stands for identity.
(**********************************************************(array)
>cat test.json | jql '(elem "countries" (elem (array 0 2)))' [ { "eu_since": "2004", "european": true, "name": "Poland", "population": 38000000 }, { "eu_since": "1993", "european": true, "name": "Germany", "population": 83000000 }]
elem can work with single strings, single integers, arrays of those, and objects with them as values (but we won't cover those now to keep things simple).
[ { “european”: false, “name”: “United States”, “population”: 327000000 }, { “eu_since”: “1993”, “european”: true, “name”: “Germany”, “population”: 83000000 }] ********************************** [ { “european”: false, “name”: “United States”, “population”: 327000000 }, { “eu_since”: “1993”, “european”: true, “name”: “Germany”, “population”: 83000000 }] ********************** (keys) ************************************************************** What if we want to get all the country names? A new friend –keys – is key in this situation. (🗝) **************************************************************** (**********************************************
cat test.json | jql '(elem "countries" (elem (keys) (elem "name")))' [ "Poland", "United States", "Germany"]it returns an array of all the keys of the given collection. Fields and Indices for Objects and Arrays respectively.
To illustrate, here (keys) used on an object:>cat test.json | jql '("countries" (0 (keys)))' [ "name", "population", "european", "eu_since"][ "United States", 327000000 ] (******************************************************
(************************************************** (Attention) Now we have to understand a very important mechanism underlying (jql) . All functions operate in the context of the JSON we're operating on.Some functions, like elem, will cut down the context for expressions it evaluates. The first argument - which should evaluate to the positions we need - gets evaluated in the context of the entire array, that's why (keys) returns all the indices. The second one on the other hand, operates in the context of a single element. In theory we're really just creating and composing a big function - pipeline, so to say - which gets applied to our JSON blob. This may sound complicated, but I find it becomes intuitive quite quickly.[ "United States", 327000000 ] ****************************** You can see that elem is the most used function, and in fact it's what you ' ll usually be using when munging data, so there's a shortcut. If you put a value in function name position, it implicitly converts it to an elem. This way we can rewrite the previous query to be much shorter, and better match the shape of the data. >cat test.json | jql '("countries" ((keys) ("name")))' [ "Poland", "United States", "Germany"][ "United States", 327000000 ] (************************************************** [ "Poland", "United States", "Germany"] (********************************************************** (little showcase)
Remember when I said you can use integers, strings, arrays and objects as positions?>cat test.json | jql '("countries" ((array (array 0 (array 0 (array 0 (array 0 2))))) 1 (object "key1" 1 "key2" (array 0 (object "key1" 1 "key2" (array 0 2))))) ("population"))) ' [ [ 38000000, [ 38000000, [ 38000000, [ 38000000, 83000000 ] ] ] ], (**********************************************************************************************************************, { "key1": (************************************************************************************************************************, "key2": [ 38000000, { "key1": 327000000, "key2": [ 38000000, 83000000 ] } ] } ](************************************************************************ 🔥 [ "United States", 327000000 ] ************************************************************** (🔥) ****************************************************************
🔥 [ { "european": false, "name": "United States", "population": 327000000 }, { "eu_since": "1993", "european": true, "name": "Germany", "population": 83000000 }] Don't do this. [ "United States", 327000000 ] ********************************************** (********************************************************** (******************************************* (range) ************************************************************ We can also select a range of elements, using the ... you guessed it - (range) ************************************************** (function.) >cat test.json | jql '("countries" ((range 1 3) ("name")))' [ "United States", "Germany"]You can use the (array) function in value position too obviously. If you want a list of name-population tuples you can just
>cat test.json | jql '("countries" ((keys) (array ("name") ("population"))))' [ [ "Poland", 38000000 ], [ "United States", 327000000 ], [ "Germany", 83000000 ] ]Here you can see thatarray
passes the whole context given to it to each of its arguments. (Reminder: We're using "name" and "population" as elem shortcuts here.)
Most functions work like this. Only elem is the "context-cutdowner", so to say. (********************************************************
(********************************************************** object [ "Poland", "United States", "Germany" ] ************************************************ You can also use (object) to create objects, with arguments alternating keys and values. >cat test.json | jql '(object "names" ("countries" ((keys) ("name"))) "populations" ("countries" ((array 0 0 1) ("population")))) " { "names": [ "Poland", "United States", "Germany" ], "populations": [ "Poland", "United States", "Germany" ] }Now we're done with the core [ "United States", 327000000 ] **************************************** functionality of jql. The stuff so far will probably suffice for most use cases and even very complex data structures. However, here come more functions:
(******************************************************************** () (********************************************************** () ********************************** String manipulation [european:false name:United States population:3.27e 08] ******************************************** [ "United States", 327000000 ] ************************************************ (********************************************************** [ { "european": false, "name": "United States", "population": 327000000 }, { "eu_since": "1993", "european": true, "name": "Germany", "population": 83000000 }] ************************************ [ "United States", 327000000 ] ************************************ (join [ "Germany", 83000000 ]
If you ever need tojoinan array of expressions into a string, (join [Position] ************************ 's the mate you're looking for!joinWill also stringify anything it meets. Without separator: (**********************************************>cat test.json | jql '("countries" ((keys) (join (array ("name") ("population") ("european")))))' [ "Poland3.8e 07true", "United States3.27e 08false", "Germany8.3e 07true"]With separator: (**********************************************
>cat test.json | jql '("countries" ((keys)) join (array ("name") ("population") ("european")) ",")))' [ "Poland population: 38000000", "United States population: 327000000", "Germany population: 83000000"](************************************************************ (******************************************** (********************************************** (sprintf) Whenever I learn a new language, I feel much more comfortable when I know there's a sprintf function. (and how to use it)
Anyways, here you go, the syntax is the same as that of the go standard library (fmt.Sprintf) ********************************************* (function: (********************************************** >cat test.json | jql '("countries" ((keys) (sprintf "% s population:% .0f" ("name") ("population"))))' [ "Poland population: 38000000", "United States population: 327000000", "Germany population: 83000000"]Hope you're feeling comfortable[ "Poland3.8e 07true", "United States3.27e 08false", "Germany8.3e 07true"] now:)
(******************************************** error (****************************************** There is a little helper function -error- for those times when you're debugging your queries. consider to be "true"? Well, it's quite simple actually.It’s an expression which errors on evaluation and gives you a stack trace. It can also evaluate and print any expression you want in it's context. >cat test.json | jql '("countries" ((keys)) (sprintf "% s population:% .0f" ("name") (error "test message"))))' / / (******************************************************************************************************************************************************************************************************************************************: (********************************************************************************************************************************************************************************************************************: error getting expression value for object: couldn't get transformed value for field countries with value (map) ************ map [ "Poland", "United States", "Germany" ] ]: couldn't get element using position at array index 0: couldn't get transformed value for index 0 with value map [eu_since:2004 european:true name:Poland population:3.8e 07]: couldn't evaluate sprintf argument with index 1: Message: test message goroutine 1 [ "Poland population: 38000000", "United States population: 327000000", "Germany population: 83000000"]: runtime / debug.Stack (0xc (f0, 0x) ******************************************************************************************************************************************** (e) , 0xc e0 /usr/local/Cellar/go/1.(****************************************************************************************************************************************************************************************************************************. 4 / libexec / src / runtime / debug / stack.go: 26 0x9d github.com/cube / jql / jql .Error.Get (0x (****************************************************************************************************************************, 0xc (f0, 0x) ***************************************************************************************************************************************** (e) *******************************************************************************************************************************************************************************************************************************************, 0xc (e0, 0x) c5c0, 0xc 000074460, 0x0, 0x0) /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / jql / functions.go: ( 0x) github.com/cube / jql / jql .Sprintf.Get (0x (****************************************************************************************************************************, 0xc (b0, 0xc) ***********************************************************************************************************************************, 0x2, 0x2, 0x [ "Poland3.8e 07true", "United States3.27e 08false", "Germany8.3e 07true"] ********************************************************************************************************************************** (e) , 0xc (e0, 0x) b (************************************************************************************************************************************************************************, 0xc (************************************************************************************************************************************, 0x 13, ...) /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / jql / functions.go: [ "Poland3.8e 07true", "United States3.27e 08false", "Germany8.3e 07true"] a github.com/cube / jql / jql .GetElement (0x********************************************************************************************************************************** (bb) ******************************************************************************************************************************************************************************************, 0x (e0, 0x) *************************************************************************************************************************************************** (ec0, 0xc) ********************************************************************************************************************************************************************************************************************************* (a1e0, 0x) ********************************************************************************************************************************************** (a) *********************************************************************************************************************************************************************************************************************, 0xc (******************************************************************************************************************************, 0x (c) , 0x [eu_since:2004 european:true name:Poland population:3.8e 07] ****************************************************************************************************************************************************** (b6d0, 0x0, 0x) ba 010 /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / jql / functions.go: ( 0xa) github.com/cube / jql / jql .GetElement (0x ec0, 0xc (a) , 0x (ec0, 0xc) ********************************************************************************************************************************************************************************************************************************** (a1e0, 0x) ********************************************************************************************************************************************* (a) **********************************************************************************************************************************************************************************************************************, 0xc (*********************************************************************************************************************************, 0x0, 0x [ "Poland population: 38000000", "United States population: 327000000", "Germany population: 83000000"] fffff, 0xc 16 afba0, 0x 299 /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / jql / functions.go: ( 0x) ******************************************************************************************************************************************************************************************************** d github.com/cube / jql / jql .Element.Get (0x 50000000, 0x [ "Poland3.8e 07true", "United States3.27e 08false", "Germany8.3e 07true"] ********************************************************************************************************************** (c8, 0x) ********************************************************************************************************************************************* (a) ********************************************************************************************************************************************************************************************************************, 0xc 1114860, 0x (ec0, 0xc) ******************************************************************************************************************************************************************************************************************************** (a1e0, 0xc) ******************************************************************************************************************************************************************************************************************************************* (c) , 0x1ee, 0xc (********************************************************************************************************************************************************************************************************************************* (c) **************************************************************************************************************************************, 0x c) /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / jql / functions.go: ( 0x [ 38000000, { "key1": 327000000, "key2": [ 38000000, 83000000 ] f github.com/cube / jql / jql .GetElement (0x************************************************************************************************************************************ (c5c0, 0xc) ***************************************************************************************************************************************************** (b0, 0x) **************************************************************************************************************************************** (e) ******************************************************************************************************************************************************************************************************************************************, 0xc (b0, 0x) ************************************************************************************************************************************ (c0, 0xc) a 299, 0x0, 0x [european:false name:United States population:3.27e 08] , 0xc b0, 0xa /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / jql / functions.go: ( 0x github.com/cube / jql / jql .Element.Get (0x (****************************************************************************************************************************, 0xc (a0, 0x) ************************************************************************************************************************************* (c0, 0xc) ********************************************************************************************************************************************************************************************************************************** (a) ***********************************************************************************************************************************************************************, 0x (e) ******************************************************************************************************************************************************************************************************************************************, 0xc (b0, 0x) ****************************************************************************************************************************************************************************************************************************** , 0x 01575879, 0x1, 0xc [ "Poland3.8e 07true", "United States3.27e 08false", "Germany8.3e 07true"] b0) /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / jql / functions.go: ( 0x [ 38000000, { "key1": 327000000, "key2": [ 38000000, 83000000 ] f main.main () /Users/jakub/Projects/Go/src/github.com/cube2222 / jql / main.go: 45 0x2b4(**********************************************************Logic (******************************** Yey, back to the basics!
and You've got (eq) , (lt) ************************************************** and (gt) **************************************************, all working as you'd probably expect: >cat test.json | jql '(eq "test" "test")' true>cat test.json | jql '(eq "test" "test2")' false>cat test.json | jql '(lt "a" "b")' true>cat test.json | jql '(lt "b" "a")' false>cat test.json | jql '(gt 5 4)' trueIn case you're wondering,
eqdoes a reflect. DeepEqual [Array[Int] on both arguments.
You've also gotand, (or) , (not) ****************************************************, to cover your back when tackling those primal and primitive (some would say (fundamental) problems you may encounter:andorare both lazy. >cat test.json | jql '(and true true true)' true>cat test.json | jql '(and true true false)' false>cat test.json | jql '(and true true null)' false>cat test.json | jql '(or true true false)' true>cat test.json | jql '(or)' false>cat test.json | jql '(and)' true>cat test.json | jql '(not true)' false>cat test.json | jql '(not false)' true>cat test.json | jql '(not null)' true>cat test.json | jql '(not (array false))' false(****************************************** and
(********************************************************** (Truthiness) This brings us to the topic of truthiness. What does
(**************************************************************************************** (null) ******************************************* (is not) ****************************************** (truthy.) (false) ***************************************** (is nottruthy. (************************************************************************************** (anything else) ****************************************** istruthy. (************************************************************************************** (****************************************************************************************(******************************************** (pipe) ********************************************************************** (****************************************** (pipe) It is a fairly useless function because you can just use a bash pipe. But if for some reason you want to save cpu cycles: >cat test.json | jql '(pipe ("countries") ((range 2)) ((keys) ("name"))) ' [ { "european": false, "name": "United States", "population": 327000000 }, { "eu_since": "1993", "european": true, "name": "Germany", "population": 83000000 }]is equal to (**********************************************
>cat test.json | jql '("countries")' | jql '((range 2))' | jql '((keys) ("name"))' [ { "european": false, "name": "United States", "population": 327000000 }, { "eu_since": "1993", "european": true, "name": "Germany", "population": 83000000 }](****************************************************************************************** (********************************************************** (********************************** (recover) ******************************************************************** Finally, the one whose name shall not be spoken out loud. (👹) Needing him means you either encountered a bug in (jql) *******************************************, or that your dataset is seriously botched.
GIPHY App Key not set. Please check settings