JTC
stand for:JSON test console, but it’s a legacy name, don’t get misled.
JTC
offers a powerful way to select one or multiple elements from a source JSON and apply various actions on the selected elements at once (wrap selected elements into a new JSON, filter in / out, update elements, insert new elements, remove, copy, move, compare, transform and swap around).
Content:
-
(Short description)
(Short description)
–JTC
is simple but efficient cli utility tool to manipulate JSON data
JTC
offers following features (a short list of main features):
- simple user interface allowing applying a bulk of changes in one command
- featured walk-path interface lets extracting any combination of data from source JSON
- extracted data is representable either as it found, or encapsulated in JSON array / object
- support Regular Expressions when searching source JSON
- fast and efficient processing very large JSON files (built-in search cache)
- insert / updates operations optionally may undergo (shell cli) evaluation
- features namespaces, interpolation from namespaces and templates
- supports buffered and streamed modes of input reads
- written entirely in C 14, no dependencies (STL only, idiomatic C , no memory leaks)
- extensively debuggable
(conforms to JSON specification)json.org)
Walk path feature is easy to understand – it’s only made of 2 types of lexemes:
- subscripts – enclosed into
[
,]
: subscripts let traversing JSON tree downwards andUpwards
- search lexemes – encased into
,>
: search lexemes facilitate either full match or Regex search.
Both types of lexemes are iterable – subscript let iterating over children of currently addressed iterables node (array / object), while iterable search lexemes let iterating over all matches for a given search criteria. A walk path is made of an arbitrary number of lexemes, while the tool accepts a virtually unlimited number of walk paths. See below more detailed explanation with examples
Linux and MacOS precompiled binaries are available for download
For compiling,C 14
(or later) is required:
- to compile under MacOS, use cli:
c -o jtc -wall -std=c 14 -Ofast jtc.cpp
- To compile under Linux, use cli:
c -o jtc -Wall -std=gnu 14 -static -Ofast jtc.cpp
pass- DNDEBUG
flag if you like to compile w / o debugs, however it’s unadvisable – there’s no performance gain from doing so
or download the latest precompiled binary:
- latest(macOS)
- latest(Linux) ***************************************************************************************************************************************************************************************************** (bit)
- latestLinux (bit)
Compile and install instructions:
downloadjtc-master.zip
, unzip it, descend into unzipped folder, compile using an appropriate command, move compiled file into an install location.
here’re the example steps for (MacOS) :
- say, (jtc-master.zip) has been downloaded to a folder and the terminal app is open in that folder:
unzip jtc-master.zip
cd jtc-master
c -o jtc -wall -std=c 14 -Ofast jtc.cpp
sudo mv ./jtc / usr / local / bin /
ForLinuxyou ‘d have to compile using this line:
- (c -o jtc -Wall -std=gnu ) -static -Ofast jtc.cpp
Release Notes
See the latestRelease Notes
Quick- start guide:
runjtc -g
for walk path explanations, usage notes and additional usage examples
Consider a following JSON (a mockup of a bookmark container), stored in a fileBookmarks
:
{ "Bookmarks": [ { "children": [ { "children": [ { "name": "The New York Times", "stamp": "2017-10-03, 12:05:19", "url": "https://www.nytimes.com/" }, { "name": "HuffPost UK", "stamp": "2017-11-23, 12:05:19", "url": "https://www.huffingtonpost.co.uk/" } ], "name": "News", "stamp": "2017 - 10 - 02, 12: 05: 19 '' }, { "children": [ { "name": "Digital Photography Review", "stamp": "2017-02-27, 12:05:19", "url": "https://www.dpreview.com/" } ], "name": "Photography", "stamp": "2017 - 02 - 27, 12: 05: 19 " } ], "name": "Personal", "stamp": "2017 - 01 - 22, 12: (***********************************************************************************************************************************************************************************************************************************: 19 " }, { "children": [ { "name": "Stack Overflow", "stamp": "2018-05-01, 12:05:19", "url": "https://stackoverflow.com/" }, { "name": "C reference", "stamp": "2018-06-21, 12:05:19", "url": "https://en.cppreference.com/" } ], "name": "Work", "stamp": "2018 - 03 - 06, 12: 07: 29 '' } ] }
1. let’s start with a simple thing – list all URLs:
bash $ jtc -w 'l: 'Bookmarks "https://www.nytimes.com/" "https://www.huffingtonpost.co.uk/" "https://www.dpreview.com/" "https://stackoverflow.com/" "https://en.cppreference.com/"
The walk-path (an argument of- w
) is a combination of lexemes. There are only 2 types of lexemes:
- subscript lexemes – enclosed in
[
,]
- search lexemes – enclosed in
,>
- the walk-paths may contain any number of lexemes
let’s take a look at the walk-pathL:
:
- search lexemes are enclosed in angular brackets
,>
- that style provides arecursive search
(suffix) *************************************************** (L) instructs to search amonglabels (only) (quantifier) ***************************************************):instructs to findall occurrences, such quantifiers makes a pathiterable
2. dump all bookmark names from the (Work) folder:
bash $ jtc -w '[-1] [children] [:] [name] 'Bookmarks "Stack Overflow" "C reference"
here the walk path[-1] [children] [:] [name]
is made of the following lexemes (spaces separating lexemes are optional):
a.
: find within a JSON tree the (first) occurrence where theJSON stringvalue is matching"Work"
exactly
b.[-1]
: (step up) one tier in the JSON tree structure (ie address an immediate parent of the found JSON element)
c.[children]
:select / addressa node whose label is" children "
(it’ll be a JSON array, at the same tier with (Work) )
d.[:]
: select aneach nodein the array
e.[name]
: select / address a node whose label is"name"
- subscript offsets are enclosed into square brackets
[
,]
and may have different meaning:
- simple numerical offsets (eg:
[0]
,[5]
, etc) select / address a respective JSON immediate child in the addressed node – a.k.a. numerical subscripts
- slice / range offsets, expressed as
[N:N]
let selecting any slice / range of element in the array / object (any of (N) could be omitted in that notation)
- numerical offsets proceeded with
make a pathiterable– all children starting with the given index will be selected (e.g .: [ 2] will select / address all immediate children starting from 3rd one) – such notation is equivalent of[N:]
- numerical negative offsets (eg
[-1]
,[-2]
, etc) will select / address a parent of currently selected / found node, a parent of a parent, etc
- textual offsets (eg
[name]
,[children]
, etc) select / address nodes with corresponding labels among immediate children (ie textual subscripts)
*** there’s more on offsets and search quantifiers below
in order to understand better how a walk path works, let’s run a series of cli in a slow-motion, gradually adding lexemes to the path, perhaps with the option- L
to see also the labels (if any) of the selected elements:
bash $ jtc -w ''-l Bookmarks "name": "Work"
bash $ jtc -w '[-1] '-l Bookmarks { "children": [ { "name": "Stack Overflow", "stamp": "2018-05-01, 12:05:19", "url": "https://stackoverflow.com/" }, { "name": "C reference", "stamp": "2018-06-21, 12:05:19", "url": "https://en.cppreference.com/" } ], "name": "Work", "stamp": "2018 - 03 - 06, 12: 07: 29 '' }
bash $ jtc -w '[-1] [children] '-l Bookmarks "Children": [ { "name": "Stack Overflow", "stamp": "2018-05-01, 12:05:19", "url": "https://stackoverflow.com/" }, { "name": "C reference", "stamp": "2018-06-21, 12:05:19", "url": "https://en.cppreference.com/" }]
bash $ jtc -w '[-1] [children] [:] '-l Bookmarks { "name": "Stack Overflow", "stamp": "2018 - 05 - 01, 12: 05: 19 ", "url": "https://stackoverflow.com/" } { "name": "C reference", "stamp": "2018 - 06 - 21, 12: 05: 19 ", "url": "https://en.cppreference.com/" }
bash $ jtc -w '[-1] [children] [:] [name] '-l Bookmarks "name": "Stack Overflow" "name": "C reference"
3). dump all URL’s names:
bash $ jtc -w ' l: [-1] [name] 'Bookmarks "The New York Times" "HuffPost UK" "Digital Photography Review" "Stack Overflow" "C reference"
this walk pathl: [-1] [name]
:
-
finds recursively (encasement,>
(all):
) JSON elements with a labell
) matchingurl
-
then for an each found JSON element, select its parent ([-1]
)
-
then, select a JSON element with the label"name"
(encasement[
,]
)
4. dump all the URLs and their corresponding names, preferably wrap found pairs in JSON:
bash $ jtc -w 'l: '-w'l: [-1] [name] '-jl Bookmarks [ { "name": "The New York Times", "url": "https://www.nytimes.com/" }, { "name": "HuffPost UK", "url": "https://www.huffingtonpost.co.uk/" }, { "name": "Digital Photography Review", "url": "https://www.dpreview.com/" }, { "name": "Stack Overflow", "url": "https://stackoverflow.com/" }, { "name": "C reference", "url": "https://en.cppreference.com/" }]
- yes, multiple walks (
- w
(are allowed) (option) – Jwill wrap the walked outputs into a JSON array, but not just,
(option) – Lused together with- J
will ensure relevant walks are grouped together (try without- l
) (if multiple walks)- w
) are present, by default, walked results will be printed interleaved
5). Subscripts (offsets) and Searches explained
In short:
- Subscript lexemes (
[..]
) facilitate:
- Addressing children (by index / label) inJSON iterables(Arraysandobjects) – ie traverse JSON structure downward from the root (toward leaves), eg:
[2]
,[id]
- addressing parents (immediate and distant) – ie traverse JSON structure upwards, toward the the root (from leaves), eg:
[-1]
(tier offset from the walked element),[^2]
(tier offset from the root)
- select ranges and slices of JSON elements inJSON iterables, eg:
[ 2]
,[:]
,[:3]
,[-2:]
,[1:-1]
(Search lexemes)<..>
,>..) facilitate:
- recursive
<..>
) and non-recursive (>..) matches
- there're optional one-letter suffixes that may follow the lexemes (eg:
(Q) ) which define type of search: (REGEX) string search, (REGEX) label search, (REGEX) numerical, boolean, null, atomic, objects, arrays (or either), arbitrary JSONs, unique, duplicates, etc.
- there're also optional quantifiers to lexemes (must take the last position, after the suffix if one present) - let selecting match instance, or range of matches (eg:
(L3) - will match 4th (zero based) label"ID"
; if no quantifier present0
is assumed - first match)
- subscript lexemes could be joined with search lexemes over ':' to facilitatescoped search, eg:
[id]:
is a single lexeme which will match recursively the first occurrence of the string"value"
with the label"id"
- ie"id": "value"
- Directives: there are a few suffixes which turn a search lexeme into a directive:
- directives do not do any matching, instead they facilitate a certain action / operation with the currently walked JSON element, like: memorize it in thenamespace, or erase from it, or memorize its label, or perform ashell clievaluation
(couple directives)f
andF
) facilitate also walk branching
Refer toJTC (User Guide) for the detailed explanation of the subscripts, search lexemes and directives.
6. Debugability / JSON validation
JTC
is extensively debuggable: the more times option- D
is given the more debugs will be produced (currently debug depth may go as deep as 7:- ddddddd
). Enabling too many debugs might be overwhelming, though one specific case many would find extremely useful – when validating a failing JSON:
bash $
If JSON is big, it’s desirable to locate the parsing failure point. Specifying just one- D
let easily spotting the parsing failure point and its locus:
bash $.parsejson (), exception locus: ...], | "children": [,], | "spouse": null | } ... .location_ (), exception spot: --------------------------------->| (offset: 967) jtc json exception: expected_json_value bash $
Complete User Guide
Refer to a completeUser Guidefor further examples and guidelines.
A tiny example of class usage and its interface (c 14):
Say, we want to accomplish a following task:
- read Address Book JSON from
- sort all records by
Name
(for simplicity, assume all records have that label)
(output resulting Address Book JSON)
Below is the code sample how that could be achieved usingJson.hpp
class and the source JSON – Address Book:
# include#include#include "lib / Json.hpp" // compile with: c -o sort_ab -Wall -std=c 14 sorting_ab.cpp using namespace std; int main (int argc, char * argv []) { Json jin ({istream_iterator(cin>>noskipws), istream_iterator{}}); // read and parse json from cin vectornames (jin.walk ("[AddressBook] [ 0] [Name] "), jin.walk (). end ()); // get all the names sort (names.begin (), names.end ()); // sort the names Json srt=ARY {}; // rebuild AB with sorted records for (const auto & name: names) srt.push_back (move jin.walk ("[AddressBook] [Name]:[-1] "))); cout
Address Book JSON:
bash $ cat addressbook-sample.json { "AddressBook": [ { "Name": "John", "age": 25, "address": { "city": "New York", "street address": "599 Lafayette St", "state": "NY", "postal code": "10012" }, "phoneNumbers": [ { "type": "mobile", "number": "212 555-1234" } ], "children": [], "spouse": null }, { "Name": "Ivan", "age": 31, "address": { "city": "Seattle", "street address": "5423 Madison St ", "state": "WA", "postal code": "98104 '' }, "phoneNumbers": [ { "type": "home", "number": "3 23 12334" }, { "type": "mobile", "number": "6 54 12345" } ], "children": [], "spouse": null }, { "Name": "Jane", "age": 25, "address": { "city": "Denver", "street address": "6213 E Colfax Ave ", "state": "CO", "postal code": "80206 " }, "phoneNumbers": [ { "type": "office", "number": " 1 543 422-1231" } ], "children": [], "spouse": null } ] } bash $
Output result:
bash $ cat addressbook-sample.json | sort_ab [ [ { "Name": "Ivan", "address": { "city": "Seattle", "postal code": "98104 ", "state": "WA", "street address": "5423 Madison St " }, "age": 31, "children": [], "phoneNumbers": [ { "number": "3 23 12334", "type": "home" }, { "number": "6 54 12345", "type": "mobile" } ], "spouse": null }, { "Name": "Jane", "address": { "city": "Denver", "postal code": "80206 ", "state": "CO", "street address": " E Colfax Ave " }, "age": 25, "children": [], "phoneNumbers": [ { "number": " 1 543 422-1231", "type": "office" } ], "spouse": null }, { "Name": "John", "address": { "city": "New York", "postal code": "10012 ", "state": "NY", "street address": "599 Lafayette St " }, "age": 25, "children": [], "phoneNumbers": [ { "number": "212 555-1234", "type": "mobile" } ], "spouse": null } ] ] bash $
for the complete description of Json class interface, refer toJson.hpp
(JTC) vsJQ:
JTC
wasinspiredby the complexity ofJQinterface (and its (DSL) ), aiming to provide a user tool which would let attaining the desired result in a more feasible way
utility ideology:
- (JQ) is a stateful processor with own DSL, variables, operations, control flow logic, IO system, etc, etc
JTC
is a unix utility confining its functionality to operation types with its data model only (as per unix ideology).JTC
performs one operation at a time and if successive operations required, then (cli) to be daisy-chained over the pipe symbol|
JQis non -idiomatic in aunix way, eg, one can write a program inJQlanguage that even has nothing to do with JSON. Most of the requests (if not all) to manipulate JSONs aread hoctype of tasks, and learning (JQ) ‘s DSL forad hoctype of tasks is an overkill (that purpose is best facilitated with (GPL) ).
The number of asks on theStackoverflowto facilitate even simple queries for (JQ) is huge – that’s the proof in itself that for many people feasibility of attaining their asks withJQis a way too low, hence they default to posting their questions on the forum.
JTC
on the other hand is a utility (not a language), which employs a novel but powerful concept, which “embeds” the ask right into thewalk-path. That facilitates a much higher feasibility of attaining a desired result: building the walk-path a lexeme by a lexeme, one at a time, provides an immediate visual feedback and let coming up with the desired result quite quickly.
learning curve:
- (JQ) : before you could come up with a query to handle even a relatively simple ask, you need to become an expert inJQ‘s language, which will take some time. Coming up with the complex queries requires it seems having a “PhD” inJQ, or spending lots of time on stackoverflow and similar forums
JTC
employs only a single (but powerful) concept of the (walk-path) (which is made only of 2 types of lexemes, each type though has several variants) which is easy to grasp.
handling irregular JSONs:
- (JQ) : handling irregular JSONs for (JQ) is not a challenge, building a query is! The more irregularities you need to handle the more challenging the query ( (JQ) program) becomes
JTC
was conceived with the idea of being capable of handling complex irregular JSONs with a simplified interface – that all is fitted into the concept of thewalk-path, while daisy-chaining multiple (JTC) operations it’s possible to satisfy almost every query.
(programming model)
- (JQ) is written in (C) , which drags all intrinsic problems the language has dated its creation
JTC
is written in idiomatic (C ) (the most powerful programming language to date) using STL only. Main JSON engine / library does not have a single (new) operator, nor it has a single naked pointer acting as a resource holder / owner, thus (JTC) is guaranteed to befree of memory leaks(at least one class of the problems is off the table) –STL Guaranty.
Also,JTC
is written in a very portable way, it should not cause any problems compiling it under any unix like system.
JSON numerical fidelity:
- (JQ) is not compliant with JSON numerical definition. What jq does, it simply converts a symbolic numerical representation to an internal binary and keeps it that way. That approach:
- is not compliant with JSON definition of the numerical values
- it has problems retaining required precision
- might change original representation of numericals
JTC
validates all JSON numericals per JSON standard and keep numbers internally in their original symbolical format, so it’s free of all the above caveats:
JTC
is simple but efficient cli utility tool to manipulate JSON dataJTC
offers following features (a short list of main features):(conforms to JSON specification)json.org)
[
,]
: subscripts let traversing JSON tree downwards andUpwards,>
: search lexemes facilitate either full match or Regex search.
C 14
(or later) is required:c -o jtc -wall -std=c 14 -Ofast jtc.cpp
c -o jtc -Wall -std=gnu 14 -static -Ofast jtc.cpp
- DNDEBUG
flag if you like to compile w / o debugs, however it’s unadvisable – there’s no performance gain from doing sojtc-master.zip
, unzip it, descend into unzipped folder, compile using an appropriate command, move compiled file into an install location.unzip jtc-master.zip
cd jtc-master
c -o jtc -wall -std=c 14 -Ofast jtc.cpp
sudo mv ./jtc / usr / local / bin /
jtc -g
for walk path explanations, usage notes and additional usage examplesBookmarks
:{ "Bookmarks": [ { "children": [ { "children": [ { "name": "The New York Times", "stamp": "2017-10-03, 12:05:19", "url": "https://www.nytimes.com/" }, { "name": "HuffPost UK", "stamp": "2017-11-23, 12:05:19", "url": "https://www.huffingtonpost.co.uk/" } ], "name": "News", "stamp": "2017 - 10 - 02, 12: 05: 19 '' }, { "children": [ { "name": "Digital Photography Review", "stamp": "2017-02-27, 12:05:19", "url": "https://www.dpreview.com/" } ], "name": "Photography", "stamp": "2017 - 02 - 27, 12: 05: 19 " } ], "name": "Personal", "stamp": "2017 - 01 - 22, 12: (***********************************************************************************************************************************************************************************************************************************: 19 " }, { "children": [ { "name": "Stack Overflow", "stamp": "2018-05-01, 12:05:19", "url": "https://stackoverflow.com/" }, { "name": "C reference", "stamp": "2018-06-21, 12:05:19", "url": "https://en.cppreference.com/" } ], "name": "Work", "stamp": "2018 - 03 - 06, 12: 07: 29 '' } ] }
bash $ jtc -w 'l: 'Bookmarks "https://www.nytimes.com/" "https://www.huffingtonpost.co.uk/" "https://www.dpreview.com/" "https://stackoverflow.com/" "https://en.cppreference.com/"
- w
) is a combination of lexemes. There are only 2 types of lexemes:[
,]
,>
- the walk-paths may contain any number of lexemes
L:
:,>
- that style provides arecursive search
(suffix) *************************************************** (L) instructs to search amonglabels (only) (quantifier) ***************************************************):instructs to findall occurrences, such quantifiers makes a pathiterable
bash $ jtc -w '[-1] [children] [:] [name] 'Bookmarks "Stack Overflow" "C reference"
[-1] [children] [:] [name]
is made of the following lexemes (spaces separating lexemes are optional):
: find within a JSON tree the (first) occurrence where theJSON stringvalue is matching"Work"
exactlyb.
[-1]
: (step up) one tier in the JSON tree structure (ie address an immediate parent of the found JSON element)c.
[children]
:select / addressa node whose label is" children "
(it’ll be a JSON array, at the same tier with (Work) )d.
[:]
: select aneach nodein the arraye.
[name]
: select / address a node whose label is"name"
[
,]
and may have different meaning:
- simple numerical offsets (eg:
[0]
,[5]
, etc) select / address a respective JSON immediate child in the addressed node – a.k.a. numerical subscripts - slice / range offsets, expressed as
[N:N]
let selecting any slice / range of element in the array / object (any of (N) could be omitted in that notation) - numerical offsets proceeded with
make a pathiterable– all children starting with the given index will be selected (e.g .: [ 2] will select / address all immediate children starting from 3rd one) – such notation is equivalent of
[N:]
- numerical negative offsets (eg
[-1]
,[-2]
, etc) will select / address a parent of currently selected / found node, a parent of a parent, etc - textual offsets (eg
[name]
,[children]
, etc) select / address nodes with corresponding labels among immediate children (ie textual subscripts)
- L
to see also the labels (if any) of the selected elements:bash $ jtc -w ''-l Bookmarks "name": "Work"
bash $ jtc -w '[-1] '-l Bookmarks { "children": [ { "name": "Stack Overflow", "stamp": "2018-05-01, 12:05:19", "url": "https://stackoverflow.com/" }, { "name": "C reference", "stamp": "2018-06-21, 12:05:19", "url": "https://en.cppreference.com/" } ], "name": "Work", "stamp": "2018 - 03 - 06, 12: 07: 29 '' }
bash $ jtc -w '[-1] [children] '-l Bookmarks "Children": [ { "name": "Stack Overflow", "stamp": "2018-05-01, 12:05:19", "url": "https://stackoverflow.com/" }, { "name": "C reference", "stamp": "2018-06-21, 12:05:19", "url": "https://en.cppreference.com/" }]
bash $ jtc -w '[-1] [children] [:] '-l Bookmarks { "name": "Stack Overflow", "stamp": "2018 - 05 - 01, 12: 05: 19 ", "url": "https://stackoverflow.com/" } { "name": "C reference", "stamp": "2018 - 06 - 21, 12: 05: 19 ", "url": "https://en.cppreference.com/" }
bash $ jtc -w '[-1] [children] [:] [name] '-l Bookmarks "name": "Stack Overflow" "name": "C reference"
bash $ jtc -w ' l: [-1] [name] 'Bookmarks "The New York Times" "HuffPost UK" "Digital Photography Review" "Stack Overflow" "C reference"
l: [-1] [name]
:finds recursively (encasement,
>
(all):
) JSON elements with a labell
) matchingurl
then for an each found JSON element, select its parent ([-1]
)
then, select a JSON element with the label"name"
(encasement[
,]
)
bash $ jtc -w 'l: '-w'l: [-1] [name] '-jl Bookmarks [ { "name": "The New York Times", "url": "https://www.nytimes.com/" }, { "name": "HuffPost UK", "url": "https://www.huffingtonpost.co.uk/" }, { "name": "Digital Photography Review", "url": "https://www.dpreview.com/" }, { "name": "Stack Overflow", "url": "https://stackoverflow.com/" }, { "name": "C reference", "url": "https://en.cppreference.com/" }]
- w
(are allowed) (option) – Jwill wrap the walked outputs into a JSON array, but not just,(option) – Lused together with- J
will ensure relevant walks are grouped together (try without- l
) (if multiple walks)- w
) are present, by default, walked results will be printed interleaved
[..]
) facilitate:
- Addressing children (by index / label) inJSON iterables(Arraysandobjects) – ie traverse JSON structure downward from the root (toward leaves), eg:
[2]
,[id]
- addressing parents (immediate and distant) – ie traverse JSON structure upwards, toward the the root (from leaves), eg:
[-1]
(tier offset from the walked element),[^2]
(tier offset from the root) - select ranges and slices of JSON elements inJSON iterables, eg:
[ 2]
,[:]
,[:3]
,[-2:]
,[1:-1]
(Search lexemes)<..>
,>..) facilitate:
- recursive
<..>
) and non-recursive (>..) matches
- there're optional one-letter suffixes that may follow the lexemes (eg:
(Q) ) which define type of search: (REGEX) string search, (REGEX) label search, (REGEX) numerical, boolean, null, atomic, objects, arrays (or either), arbitrary JSONs, unique, duplicates, etc.
- there're also optional quantifiers to lexemes (must take the last position, after the suffix if one present) - let selecting match instance, or range of matches (eg:
(L3) - will match 4th (zero based) label
"ID"
; if no quantifier present0
is assumed - first match)
[id]:
is a single lexeme which will match recursively the first occurrence of the string"value"
with the label"id"
- ie"id": "value"
- directives do not do any matching, instead they facilitate a certain action / operation with the currently walked JSON element, like: memorize it in thenamespace, or erase from it, or memorize its label, or perform ashell clievaluation
(couple directives)f
andF
) facilitate also walk branching
JTC (User Guide) for the detailed explanation of the subscripts, search lexemes and directives.
JTC
is extensively debuggable: the more times option- D
is given the more debugs will be produced (currently debug depth may go as deep as 7:- ddddddd
). Enabling too many debugs might be overwhelming, though one specific case many would find extremely useful – when validating a failing JSON:bash $
- D
let easily spotting the parsing failure point and its locus:bash $.parsejson (), exception locus: ...], | "children": [,], | "spouse": null | } ... .location_ (), exception spot: --------------------------------->| (offset: 967) jtc json exception: expected_json_value bash $
Name
(for simplicity, assume all records have that label)(output resulting Address Book JSON)
Json.hpp
class and the source JSON – Address Book:# include#include#include "lib / Json.hpp" // compile with: c -o sort_ab -Wall -std=c 14 sorting_ab.cpp using namespace std; int main (int argc, char * argv []) { Json jin ({istream_iterator(cin>>noskipws), istream_iterator{}}); // read and parse json from cin vectornames (jin.walk ("[AddressBook] [ 0] [Name] "), jin.walk (). end ()); // get all the names sort (names.begin (), names.end ()); // sort the names Json srt=ARY {}; // rebuild AB with sorted records for (const auto & name: names) srt.push_back (move jin.walk ("[AddressBook] [Name]:[-1] "))); cout
bash $ cat addressbook-sample.json { "AddressBook": [ { "Name": "John", "age": 25, "address": { "city": "New York", "street address": "599 Lafayette St", "state": "NY", "postal code": "10012" }, "phoneNumbers": [ { "type": "mobile", "number": "212 555-1234" } ], "children": [], "spouse": null }, { "Name": "Ivan", "age": 31, "address": { "city": "Seattle", "street address": "5423 Madison St ", "state": "WA", "postal code": "98104 '' }, "phoneNumbers": [ { "type": "home", "number": "3 23 12334" }, { "type": "mobile", "number": "6 54 12345" } ], "children": [], "spouse": null }, { "Name": "Jane", "age": 25, "address": { "city": "Denver", "street address": "6213 E Colfax Ave ", "state": "CO", "postal code": "80206 " }, "phoneNumbers": [ { "type": "office", "number": " 1 543 422-1231" } ], "children": [], "spouse": null } ] } bash $
bash $ cat addressbook-sample.json | sort_ab [ [ { "Name": "Ivan", "address": { "city": "Seattle", "postal code": "98104 ", "state": "WA", "street address": "5423 Madison St " }, "age": 31, "children": [], "phoneNumbers": [ { "number": "3 23 12334", "type": "home" }, { "number": "6 54 12345", "type": "mobile" } ], "spouse": null }, { "Name": "Jane", "address": { "city": "Denver", "postal code": "80206 ", "state": "CO", "street address": " E Colfax Ave " }, "age": 25, "children": [], "phoneNumbers": [ { "number": " 1 543 422-1231", "type": "office" } ], "spouse": null }, { "Name": "John", "address": { "city": "New York", "postal code": "10012 ", "state": "NY", "street address": "599 Lafayette St " }, "age": 25, "children": [], "phoneNumbers": [ { "number": "212 555-1234", "type": "mobile" } ], "spouse": null } ] ] bash $
JTC
wasinspiredby the complexity ofJQinterface (and its (DSL) ), aiming to provide a user tool which would let attaining the desired result in a more feasible wayJTC
is a unix utility confining its functionality to operation types with its data model only (as per unix ideology).JTC
performs one operation at a time and if successive operations required, then (cli) to be daisy-chained over the pipe symbol|
The number of asks on theStackoverflowto facilitate even simple queries for (JQ) is huge – that’s the proof in itself that for many people feasibility of attaining their asks withJQis a way too low, hence they default to posting their questions on the forum.
JTC
on the other hand is a utility (not a language), which employs a novel but powerful concept, which “embeds” the ask right into thewalk-path. That facilitates a much higher feasibility of attaining a desired result: building the walk-path a lexeme by a lexeme, one at a time, provides an immediate visual feedback and let coming up with the desired result quite quickly.JTC
employs only a single (but powerful) concept of the (walk-path) (which is made only of 2 types of lexemes, each type though has several variants) which is easy to grasp.JTC
was conceived with the idea of being capable of handling complex irregular JSONs with a simplified interface – that all is fitted into the concept of thewalk-path, while daisy-chaining multiple (JTC) operations it’s possible to satisfy almost every query.- (JQ) is written in (C) , which drags all intrinsic problems the language has dated its creation
JTC
is written in idiomatic (C ) (the most powerful programming language to date) using STL only. Main JSON engine / library does not have a single (new) operator, nor it has a single naked pointer acting as a resource holder / owner, thus (JTC) is guaranteed to befree of memory leaks(at least one class of the problems is off the table) –STL Guaranty.
Also,JTC
is written in a very portable way, it should not cause any problems compiling it under any unix like system.
JSON numerical fidelity:
- (JQ) is not compliant with JSON numerical definition. What jq does, it simply converts a symbolic numerical representation to an internal binary and keeps it that way. That approach:
- is not compliant with JSON definition of the numerical values
- it has problems retaining required precision
- might change original representation of numericals
JTC
validates all JSON numericals per JSON standard and keep numbers internally in their original symbolical format, so it’s free of all the above caveats:
Handling | JTC |
(JQ) |
---|---|---|
Invalid Json:[ 00 ] |
|
|
Parsing result | (jtc json exception: missed_prior_enumeration) | [0] |
Precision test: |
|
|
Parsing result | [ 0.99999999999999999 ] |
[1] |
Retaining original format: |
|
|
Parsing result | [ 0.00001 ] |
[1e-05] |
performance:
Here’s a 4 million node JSONtest file:
bash $ jtc -zz standard.json 4329975
The table below comparesJTC
and jq performance for similar operations (usingTIMEFORMAT="user% U sec"
, a median value is selected from 5 attempts):
(JTC) | JQ |
---|---|
Parsing JSON: |
Parsing JSON: |
bash $ time jtc standard.json | WC-L |
bash $ time jq. standard.json | WC-L |
7091578 |
7091578 |
(user 8.) ************************************************************************************************************************************************************************************* (sec) | user 18. (sec) |
removing by key from JSON: |
removing by key from JSON: |
bash $ time jtc -pw ' |
bash $ time jq 'del (.. | .attributes?) 'standard.json | WC-L |
5573690 |
5573690 |
(user 9.) *********************************************************************************************************************************************************************************** (sec) | user 27. (sec) |
The computer’s spec used for tests:
(Model Name: MacBook Pro) 15 - inch, 2019) Model Identifier: MacBookPro 15, 1 Processor Name: Intel Core i7 Processor Speed: 2,6 GHz Number of Processors: 1 Total Number of Cores: 6 L2 Cache (per Core): 256 KB L3 Cache: 12 MB Hyper-Threading Technology: Enabled Memory: (GB) MHz DDR4
Refer to a completeUser Guidefor further examples and guidelines.
Enhancement requests are more than welcome:[email protected]
GIPHY App Key not set. Please check settings