Log Formatting

Current is a flexible logging system that prefers structured data. To that end, it supports a number of ways to format the logs to be able to extract maximum data from each entry.

It’s important to understand the data model Current uses before considering formatting. For documentation on the data model, check out Log Entries.

JSON

It should come as no surprise that current.sh loves JSON formatted logs. Toplevel JSON objects are automatically detected, parsed and stored as internal structured data.

Additionally, the system supports a number of special keys to provide flexibility:

  • @timestamp: This toplevel key, if present, will be parsed as a RFC3339 formatted timestamp with nanosecond precision and used as the timestamp for the log entry.
  • @tags: A toplevel key with an object as a value. The keys and values of the sub object are pulled out and stored as tags for the current log entry.
  • Additional @ prefixed keys: These have the @ stripped and are stored as tags if the value is also a string. This overlaps with the @tags mechanism in case some emitters have issue with creating the sub object.

Observant readers might notice a similarity in the naming here with logstash. That’s not by accident. Logstash’s simple scheme and existing uses provide an excellent template to build from.

All other key/value pairs are normal attributes of a log entry. Values which are objects are traversed as expected and their values stored.

Value types

When JSON is parsed, its values are converted into current values. Here is how those types are mapped:

  1. String: JSON strings are stored as is
  2. Integer: Any JSON number that can be parsed as an integer (i.e., no float point syntax used)
  3. Float: Any JSON number that is not an integer
  4. Boolean: JSON’s true and false
  5. Interval: Not yet supported in translation (coming soon)

The one unrepresented JSON value here is null. If current sees a value is null, it simply moves on and doesn’t store anything.

Freeform

A more freeform style is also supported. This style is very similar to logfmt but has support for quoted values and can store integers and floats as well as strings for the values.

An example would be request=/data elapse=18. Two attributes, request and elapse, would be created with their respective values.

Generally we recommend people use JSON, but it can be easier to emit this more terse format sometimes.

If/when there is data that we are not able to parse as a key=value sequence, the system will automatically store the entire text of the log entry under the key message.

Tag Prefix

The system allows a special prefix at the beginning of a log entry to specify any tags that are to be applied. The syntax for this is: #tags{key:val|key2:val2}. This is used by the current tool to inject tags and can be used by any third party tools.