Three little tools: mmsum, mmwatch, mmhistogram

In a recent blog post, my colleague Marek talked about some SSDP-based DDoS activity we'd been seeing recently. In that blog post he used a tool called mmhistogram to output an ASCII histogram.

That tool is part of a small suite of command-line tools that can be handy when messing with data. Since a reader asked for them to be open sourced... here they are.

Suppose you have the following CSV of the ages of major Star Wars characters at the time of Episode IV:

Anakin Skywalker (Darth Vader),42
Boba Fett,32
C-3PO,32
Chewbacca,200
Count Dooku,102
Darth Maul,54
Han Solo,29
Jabba the Hutt,600
Jango Fett,66
Jar Jar Binks,52
Lando Calrissian,31
Leia Organa (Princess Leia),19
Luke Skywalker,19
Mace Windu,72
Obi-Wan Kenobi,57
Palpatine,82
Qui-Gon Jinn,92
R2-D2,32
Shmi Skywalker,72
Wedge Antilles,21
Yoda,896

You can get an ASCII histogram of the ages as follows using the mmhistogram tool.

$ cut -d, -f2 epiv | mmhistogram -t "Age"
Age min:19.00 avg:123.90 med=54.00 max:896.00 dev:211.28 count:21
Age:
 value |-------------------------------------------------- count
     0 |                                                   0
     1 |                                                   0
     2 |                                                   0
     4 |                                                   0
     8 |                                                   0
    16 |************************************************** 8
    32 |                         ************************* 4
    64 |             ************************************* 6
   128 |                                            ****** 1
   256 |                                                   0
   512 |                                      ************ 2

Handy for getting a quick sense of the data. (These charts are inspired by the ASCII output from systemtap).

The mmwatch tool is handy if you want to look at output from a command-line tool that provides some snapshot of values, but need to have a rate.

For example, here's df -H on my machine:

$ df -H
Filesystem             Size   Used  Avail Capacity  iused   ifree %iused  Mounted on
/dev/disk1             250G   222G    28G    89% 54231161 6750085   89%   /
devfs                  384k   384k     0B   100%     1298       0  100%   /dev
map -hosts             0B     0B     0B   100%        0       0  100%   /net
map auto_home          0B     0B     0B   100%        0       0  100%   /home
/dev/disk4             7.3G    50M   7.2G     1%    12105 1761461    1%   
/Volumes/LANGDON

Now imagine you were interested in understanding the rate of change in iused and ifree. You can with mmwatch. It's just like watch but looks for changing numbers and interprets them as rates:

$ mmwatch 'df -H'

Here's a short GIF showing it working:

And the final tool is mmsum that simply sums a list of floating point numbers (one per line).

Suppose you are downloading real-time rainfall data from the UK's Environment Agency and would like to know the total current rainfall. mmsum can help:

$ curl -s 'https://environment.data.gov.uk/flood-monitoring/id/measures?parameter=rainfall' | jq -e '.items[].latestReading.value+0' | ./mmsum
40.2

All these tools can be found on the Cloudflare Github.