Try the new
jc
web demo!
JC is now available as an Ansible filter plugin in the
community.general
collection! See this blog post for an example.
JSON CLI output utility
jc
JSONifies the output of many CLI tools and file-types for easier parsing in scripts. See the Parsers section for supported commands and file-types.
This allows further command-line processing of output with tools like jq
by piping commands:
ls -l /usr/bin | jc --ls | jq '.[] | select(.size > 50000000)'
{
"filename": "docker",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "root",
"size": 68677120,
"date": "Aug 14 19:41"
}
or using the alternative "magic" syntax:
jc ls -l /usr/bin | jq '.[] | select(.size > 50000000)'
{
"filename": "docker",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "root",
"size": 68677120,
"date": "Aug 14 19:41"
}
The jc
parsers can also be used as python modules. In this case the output will be a python dictionary, or list of dictionaries, instead of JSON:
>>> import jc.parsers.ls
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>>
>>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 23648,
'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'size': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 29024, 'date': 'May 3 22:26'}, {'filename': 'csh',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 375824, 'date': 'May 3
22:26'}, {'filename': 'date', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'size': 28608, 'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner':
'root', 'group': 'wheel', 'size': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags':
'-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 18128,
'date': 'May 3 22:26'}]
Two representations of the data are possible. The default representation uses a strict schema per parser and converts known numbers to int/float JSON values. Certain known values of None
are converted to JSON null
, known boolean values are converted, and, in some cases, additional semantic context fields are added.
To access the raw, pre-processed JSON, use the -r
cli option or the raw=True
function parameter in parse()
.
Schemas for each parser can be found in the docs/parsers
folder.
Release notes can be found here.
For more information on the motivations for this project, please see my blog post.
See also:
There are several ways to get jc
. You can install via pip
; other OS package repositories like apt-get
, dnf
, zypper
, pacman
, nix-env
, guix
, brew
, or portsnap
; via DEB/RPM packages; or by downloading the correct binary for your architecture and running it anywhere on your filesystem.
pip3 install jc
OS | Command |
---|---|
Debian/Ubuntu linux | apt-get install jc |
Fedora linux | dnf install jc |
openSUSE linux | zypper install jc |
Arch linux | pacman -S jc |
NixOS linux | nix-env -iA nixpkgs.jc |
Guix System linux | guix install jc |
MacOS | brew install jc |
FreeBSD | portsnap fetch update && cd /usr/ports/textproc/py-jc && make install clean |
Ansible filter plugin | ansible-galaxy collection install community.general |
For more packages and binaries, see https://kellyjonbrazil.github.io/jc-packaging/.
jc
accepts piped input from STDIN
and outputs a JSON representation of the previous command's output to STDOUT
.
COMMAND | jc PARSER [OPTIONS]
Alternatively, the "magic" syntax can be used by prepending jc
to the command to be converted. Options can be passed to jc
immediately before the command is given. (Note: command aliases and shell builtins are not supported)
jc [OPTIONS] COMMAND
The JSON output can be compact (default) or pretty formatted with the -p
option.
Note: For best results set the
LANG
locale environment variable toC
. For example, either by setting directly on the command-line:$ LANG=C date | jc --date
, or by exporting to the environment before running commands:$ export LANG=C
.
--airport
enables theairport -I
command parser (OSX)--airport-s
enables theairport -s
command parser (OSX)--arp
enables thearp
command parser--blkid
enables theblkid
command parser--cksum
enables thecksum
andsum
command parser--crontab
enables thecrontab
command and file parser--crontab-u
enables thecrontab
file parser with user support--csv
enables theCSV
file parser--date
enables thedate
command parser--df
enables thedf
command parser--dig
enables thedig
command parser--dmidecode
enables thedmidecode
command parser--du
enables thedu
command parser--env
enables theenv
andprintenv
command parser--file
enables thefile
command parser--free
enables thefree
command parser--fstab
enables the/etc/fstab
file parser--group
enables the/etc/group
file parser--gshadow
enables the/etc/gshadow
file parser--hash
enables thehash
command parser--hashsum
enables thehashsum
command parser (md5
,md5sum
,shasum
,sha1sum
,sha224sum
,sha256sum
,sha384sum
,sha512sum
)--hciconfig
enables thehciconfig
command parser--history
enables thehistory
command parser--hosts
enables the/etc/hosts
file parser--id
enables theid
command parser--ifconfig
enables theifconfig
command parser--ini
enables theINI
file parser--iptables
enables theiptables
command parser--iw-scan
enables theiw dev <device> scan
command parser (beta)--jobs
enables thejobs
command parser--kv
enables theKey/Value
file parser--last
enables thelast
andlastb
command parser--ls
enables thels
andvdir
command parser--lsblk
enables thelsblk
command parser--lsmod
enables thelsmod
command parser--lsof
enables thelsof
command parser--mount
enables themount
command parser--netstat
enables thenetstat
command parser--ntpq
enables thentpq -p
command parser--passwd
enables the/etc/passwd
file parser--ping
enables theping
andping6
command parser--pip-list
enables thepip list
command parser--pip-show
enables thepip show
command parser--ps
enables theps
command parser--route
enables theroute
command parser--shadow
enables the/etc/shadow
file parser--ss
enables thess
command parser--stat
enables thestat
command parser--sysctl
enables thesysctl -a
command parser--systemctl
enables thesystemctl
command parser--systemctl-lj
enables thesystemctl list-jobs
command parser--systemctl-ls
enables thesystemctl list-sockets
command parser--systemctl-luf
enables thesystemctl list-unit-files
command parser--timedatectl
enables thetimedatectl status
command parser--tracepath
enables thetracepath
andtracepath6
command parser--traceroute
enables thetraceroute
andtraceroute6
command parser--uname
enables theuname -a
command parser--uptime
enables theuptime
command parser--w
enables thew
command parser--wc
enables thewc
command parser--who
enables thewho
command parser--xml
enables theXML
file parser--yaml
enables theYAML
file parser
-a
aboutjc
. Prints information aboutjc
and the parsers (in JSON, of course!)-d
debug mode. Prints trace messages if parsing issues are encountered (use-dd
for verbose debugging)-m
monochrome JSON output-p
pretty format the JSON output-q
quiet mode. Suppresses parser warning messages-r
raw output. Provides a more literal JSON output with all values as strings and no additional semantic processing
You can specify custom colors via the JC_COLORS
environment variable. The JC_COLORS
environment variable takes four comma separated string values in the following format:
JC_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>
Where colors are: black
, red
, green
, yellow
, blue
, magenta
, cyan
, gray
, brightblack
, brightred
, brightgreen
, brightyellow
, brightblue
, brightmagenta
, brightcyan
, white
, or default
For example, to set to the default colors:
JC_COLORS=blue,brightblack,magenta,green
or
JC_COLORS=default,default,default,default
Custom local parser plugins may be placed in a jc/jcparsers
folder in your local "App data directory":
- Linux/unix:
$HOME/.local/share/jc/jcparsers
- macOS:
$HOME/Library/Application Support/jc/jcparsers
- Windows:
$LOCALAPPDATA\jc\jc\jcparsers
Local parser plugins are standard python module files. Use the jc/parsers/foo.py
parser as a template and simply place a .py
file in the jcparsers
subfolder.
Local plugin filenames must be valid python module names, therefore must consist entirely of alphanumerics and start with a letter. Local plugins may override default plugins.
Note: The application data directory follows the XDG Base Directory Specification
Some parsers like ls
, ps
, dig
, etc. will work on any platform. Other parsers that are platform-specific will generate a warning message if they are used on an unsupported platform. To see all parser information, including compatibility, run jc -ap
.
You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux lsof
output on an OSX laptop. In that case you can suppress the warning message with the -q
cli option or the quiet=True
function parameter in parse()
:
cat lsof.out | jc --lsof -q
Tested on:
- Centos 7.7
- Ubuntu 18.04
- Ubuntu 20.04
- Fedora32
- OSX 10.11.6
- OSX 10.14.6
- NixOS
- FreeBSD12
Feel free to add/improve code or parsers! You can use the jc/parsers/foo.py
parser as a template and submit your parser with a pull request.
- Local parser plugin feature contributed by Dean Serenevy
- CI automation and code optimizations by philippeitis
ifconfig-parser
module by KnightWhoSayNixmltodict
module by Martín Blechruamel.yaml
module by Anthon van der Neuttrparse
module by Luis Benitez- Parsing code from Conor Heine adapted for some parsers
- Excellent constructive feedback from Ilya Sher
Here are some examples of jc
output. For more examples, see EXAMPLES.md or the parser documentation.
arp | jc --arp -p # or: jc -p arp
[
{
"address": "gateway",
"hwtype": "ether",
"hwaddress": "00:50:56:f7:4a:fc",
"flags_mask": "C",
"iface": "ens33"
},
{
"address": "192.168.71.1",
"hwtype": "ether",
"hwaddress": "00:50:56:c0:00:08",
"flags_mask": "C",
"iface": "ens33"
},
{
"address": "192.168.71.254",
"hwtype": "ether",
"hwaddress": "00:50:56:fe:7a:b4",
"flags_mask": "C",
"iface": "ens33"
}
]
cat homes.csv
"Sell", "List", "Living", "Rooms", "Beds", "Baths", "Age", "Acres", "Taxes"
142, 160, 28, 10, 5, 3, 60, 0.28, 3167
175, 180, 18, 8, 4, 1, 12, 0.43, 4033
129, 132, 13, 6, 3, 1, 41, 0.33, 1471
...
cat homes.csv | jc --csv -p
[
{
"Sell": "142",
"List": "160",
"Living": "28",
"Rooms": "10",
"Beds": "5",
"Baths": "3",
"Age": "60",
"Acres": "0.28",
"Taxes": "3167"
},
{
"Sell": "175",
"List": "180",
"Living": "18",
"Rooms": "8",
"Beds": "4",
"Baths": "1",
"Age": "12",
"Acres": "0.43",
"Taxes": "4033"
},
{
"Sell": "129",
"List": "132",
"Living": "13",
"Rooms": "6",
"Beds": "3",
"Baths": "1",
"Age": "41",
"Acres": "0.33",
"Taxes": "1471"
}
]
dig cnn.com @205.251.194.64 | jc --dig -p # or: jc -p dig cnn.com @205.251.194.64
[
{
"id": 5509,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
"qr",
"rd",
"ra"
],
"query_num": 1,
"answer_num": 1,
"authority_num": 0,
"additional_num": 1,
"question": {
"name": "cnn.com.",
"class": "IN",
"type": "A"
},
"answer": [
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.129.67"
}
],
"query_time": 28,
"server": "2600",
"when": "Tue Nov 12 07:13:03 PST 2019",
"rcvd": 100
}
]
cat /etc/hosts | jc --hosts -p
[
{
"ip": "127.0.0.1",
"hostname": [
"localhost"
]
},
{
"ip": "::1",
"hostname": [
"ip6-localhost",
"ip6-loopback"
]
},
{
"ip": "fe00::0",
"hostname": [
"ip6-localnet"
]
}
]
ifconfig | jc --ifconfig -p # or: jc -p ifconfig
[
{
"name": "ens33",
"flags": 4163,
"state": [
"UP",
"BROADCAST",
"RUNNING",
"MULTICAST"
],
"mtu": 1500,
"ipv4_addr": "192.168.71.137",
"ipv4_mask": "255.255.255.0",
"ipv4_bcast": "192.168.71.255",
"ipv6_addr": "fe80::c1cb:715d:bc3e:b8a0",
"ipv6_mask": 64,
"ipv6_scope": "0x20",
"mac_addr": "00:0c:29:3b:58:0e",
"type": "Ethernet",
"rx_packets": 8061,
"rx_bytes": 1514413,
"rx_errors": 0,
"rx_dropped": 0,
"rx_overruns": 0,
"rx_frame": 0,
"tx_packets": 4502,
"tx_bytes": 866622,
"tx_errors": 0,
"tx_dropped": 0,
"tx_overruns": 0,
"tx_carrier": 0,
"tx_collisions": 0,
"metric": null
}
]
cat example.ini
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes
[bitbucket.org]
User = hg
[topsecret.server.com]
Port = 50022
ForwardX11 = no
cat example.ini | jc --ini -p
{
"bitbucket.org": {
"serveraliveinterval": "45",
"compression": "yes",
"compressionlevel": "9",
"forwardx11": "yes",
"user": "hg"
},
"topsecret.server.com": {
"serveraliveinterval": "45",
"compression": "yes",
"compressionlevel": "9",
"forwardx11": "no",
"port": "50022"
}
}
$ ls -l /usr/bin | jc --ls -p # or: jc -p ls -l /usr/bin
[
{
"filename": "apropos",
"link_to": "whatis",
"flags": "lrwxrwxrwx.",
"links": 1,
"owner": "root",
"group": "root",
"size": 6,
"date": "Aug 15 10:53"
},
{
"filename": "ar",
"flags": "-rwxr-xr-x.",
"links": 1,
"owner": "root",
"group": "root",
"size": 62744,
"date": "Aug 8 16:14"
},
{
"filename": "arch",
"flags": "-rwxr-xr-x.",
"links": 1,
"owner": "root",
"group": "root",
"size": 33080,
"date": "Aug 19 23:25"
}
]
netstat -apee | jc --netstat -p # or: jc -p netstat -apee
[
{
"proto": "tcp",
"recv_q": 0,
"send_q": 0,
"local_address": "localhost",
"foreign_address": "0.0.0.0",
"state": "LISTEN",
"user": "systemd-resolve",
"inode": 26958,
"program_name": "systemd-resolve",
"kind": "network",
"pid": 887,
"local_port": "domain",
"foreign_port": "*",
"transport_protocol": "tcp",
"network_protocol": "ipv4"
},
{
"proto": "tcp6",
"recv_q": 0,
"send_q": 0,
"local_address": "[::]",
"foreign_address": "[::]",
"state": "LISTEN",
"user": "root",
"inode": 30510,
"program_name": "sshd",
"kind": "network",
"pid": 1186,
"local_port": "ssh",
"foreign_port": "*",
"transport_protocol": "tcp",
"network_protocol": "ipv6"
},
{
"proto": "udp",
"recv_q": 0,
"send_q": 0,
"local_address": "localhost",
"foreign_address": "0.0.0.0",
"state": null,
"user": "systemd-resolve",
"inode": 26957,
"program_name": "systemd-resolve",
"kind": "network",
"pid": 887,
"local_port": "domain",
"foreign_port": "*",
"transport_protocol": "udp",
"network_protocol": "ipv4"
},
{
"proto": "raw6",
"recv_q": 0,
"send_q": 0,
"local_address": "[::]",
"foreign_address": "[::]",
"state": "7",
"user": "systemd-network",
"inode": 27001,
"program_name": "systemd-network",
"kind": "network",
"pid": 867,
"local_port": "ipv6-icmp",
"foreign_port": "*",
"transport_protocol": null,
"network_protocol": "ipv6"
},
{
"proto": "unix",
"refcnt": 2,
"flags": null,
"type": "DGRAM",
"state": null,
"inode": 33322,
"program_name": "systemd",
"path": "/run/user/1000/systemd/notify",
"kind": "socket",
"pid": 1607
}
]
cat /etc/passwd | jc --passwd -p
[
{
"username": "root",
"password": "*",
"uid": 0,
"gid": 0,
"comment": "System Administrator",
"home": "/var/root",
"shell": "/bin/sh"
},
{
"username": "daemon",
"password": "*",
"uid": 1,
"gid": 1,
"comment": "System Services",
"home": "/var/root",
"shell": "/usr/bin/false"
}
]
ping 8.8.8.8 -c 3 | jc --ping -p # or: jc -p ping 8.8.8.8 -c 3
{
"destination_ip": "8.8.8.8",
"data_bytes": 56,
"pattern": null,
"destination": "8.8.8.8",
"packets_transmitted": 3,
"packets_received": 3,
"packet_loss_percent": 0.0,
"duplicates": 0,
"time_ms": 2005.0,
"round_trip_ms_min": 23.835,
"round_trip_ms_avg": 30.46,
"round_trip_ms_max": 34.838,
"round_trip_ms_stddev": 4.766,
"responses": [
{
"type": "reply",
"timestamp": null,
"bytes": 64,
"response_ip": "8.8.8.8",
"icmp_seq": 1,
"ttl": 118,
"time_ms": 23.8,
"duplicate": false
},
{
"type": "reply",
"timestamp": null,
"bytes": 64,
"response_ip": "8.8.8.8",
"icmp_seq": 2,
"ttl": 118,
"time_ms": 34.8,
"duplicate": false
},
{
"type": "reply",
"timestamp": null,
"bytes": 64,
"response_ip": "8.8.8.8",
"icmp_seq": 3,
"ttl": 118,
"time_ms": 32.7,
"duplicate": false
}
]
}
ps axu | jc --ps -p # or: jc -p ps axu
[
{
"user": "root",
"pid": 1,
"cpu_percent": 0.0,
"mem_percent": 0.1,
"vsz": 128072,
"rss": 6784,
"tty": null,
"stat": "Ss",
"start": "Nov09",
"time": "0:08",
"command": "/usr/lib/systemd/systemd --switched-root --system --deserialize 22"
},
{
"user": "root",
"pid": 2,
"cpu_percent": 0.0,
"mem_percent": 0.0,
"vsz": 0,
"rss": 0,
"tty": null,
"stat": "S",
"start": "Nov09",
"time": "0:00",
"command": "[kthreadd]"
},
{
"user": "root",
"pid": 4,
"cpu_percent": 0.0,
"mem_percent": 0.0,
"vsz": 0,
"rss": 0,
"tty": null,
"stat": "S<",
"start": "Nov09",
"time": "0:00",
"command": "[kworker/0:0H]"
}
]
traceroute -m 2 8.8.8.8 | jc --traceroute -p # or: jc -p traceroute -m 2 8.8.8.8
{
"destination_ip": "8.8.8.8",
"destination_name": "8.8.8.8",
"hops": [
{
"hop": 1,
"probes": [
{
"annotation": null,
"asn": null,
"ip": "192.168.1.254",
"name": "dsldevice.local.net",
"rtt": 6.616
},
{
"annotation": null,
"asn": null,
"ip": "192.168.1.254",
"name": "dsldevice.local.net",
"rtt": 6.413
},
{
"annotation": null,
"asn": null,
"ip": "192.168.1.254",
"name": "dsldevice.local.net",
"rtt": 6.308
}
]
},
{
"hop": 2,
"probes": [
{
"annotation": null,
"asn": null,
"ip": "76.220.24.1",
"name": "76-220-24-1.lightspeed.sntcca.sbcglobal.net",
"rtt": 29.367
},
{
"annotation": null,
"asn": null,
"ip": "76.220.24.1",
"name": "76-220-24-1.lightspeed.sntcca.sbcglobal.net",
"rtt": 40.197
},
{
"annotation": null,
"asn": null,
"ip": "76.220.24.1",
"name": "76-220-24-1.lightspeed.sntcca.sbcglobal.net",
"rtt": 29.162
}
]
}
]
}
uptime | jc --uptime -p # or: jc -p uptime
{
"time": "11:30:44",
"uptime": "1 day, 21:17",
"users": 1,
"load_1m": 0.01,
"load_5m": 0.04,
"load_15m": 0.05
}
cat cd_catalog.xml
<?xml version="1.0" encoding="UTF-8"?>
<CATALOG>
<CD>
<TITLE>Empire Burlesque</TITLE>
<ARTIST>Bob Dylan</ARTIST>
<COUNTRY>USA</COUNTRY>
<COMPANY>Columbia</COMPANY>
<PRICE>10.90</PRICE>
<YEAR>1985</YEAR>
</CD>
<CD>
<TITLE>Hide your heart</TITLE>
<ARTIST>Bonnie Tyler</ARTIST>
<COUNTRY>UK</COUNTRY>
<COMPANY>CBS Records</COMPANY>
<PRICE>9.90</PRICE>
<YEAR>1988</YEAR>
</CD>
...
cat cd_catalog.xml | jc --xml -p
{
"CATALOG": {
"CD": [
{
"TITLE": "Empire Burlesque",
"ARTIST": "Bob Dylan",
"COUNTRY": "USA",
"COMPANY": "Columbia",
"PRICE": "10.90",
"YEAR": "1985"
},
{
"TITLE": "Hide your heart",
"ARTIST": "Bonnie Tyler",
"COUNTRY": "UK",
"COMPANY": "CBS Records",
"PRICE": "9.90",
"YEAR": "1988"
}
]
}
}
cat istio.yaml
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
name: "default"
namespace: "default"
spec:
peers:
- mtls: {}
---
apiVersion: "networking.istio.io/v1alpha3"
kind: "DestinationRule"
metadata:
name: "default"
namespace: "default"
spec:
host: "*.default.svc.cluster.local"
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
cat istio.yaml | jc --yaml -p
[
{
"apiVersion": "authentication.istio.io/v1alpha1",
"kind": "Policy",
"metadata": {
"name": "default",
"namespace": "default"
},
"spec": {
"peers": [
{
"mtls": {}
}
]
}
},
{
"apiVersion": "networking.istio.io/v1alpha3",
"kind": "DestinationRule",
"metadata": {
"name": "default",
"namespace": "default"
},
"spec": {
"host": "*.default.svc.cluster.local",
"trafficPolicy": {
"tls": {
"mode": "ISTIO_MUTUAL"
}
}
}
}
]