Loki4j Logback

Loki4j Logback

  • Docs
  • Contributing
  • Help
  • GitHub

›Logback

Logback

  • Configuration
  • Labels and structured metadata
  • JSON Message Layout
  • Protobuf Support
  • Apache HttpClient
  • Grafana Cloud
  • Monitoring
  • Tracing
  • Migration Guide
  • Compatibility Matrix
  • Old Docs

Loki4j configuration

Reference

Loki4j appender XML configuration consists of 3 sections: general settings, HTTP settings, and format settings.

<appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
    <!-- general settings -->
    <http>
        <!-- HTTP settings -->
    </http>
    <format>
        <!-- format settings -->
    </format>
</appender>

Most Loki4j appender settings are optional. These few that are required are marked explicitly.

General settings

SettingDefaultDescription
batchMaxItems1000Max number of events to put into a single batch before sending it to Loki
batchMaxBytes4194304Max number of bytes a single batch can contain (as counted by Loki). This value should not be greater than server.grpc_server_max_recv_msg_size in your Loki config
batchTimeoutMs60000Max time in milliseconds to keep a batch before sending it to Loki, even if max items/bytes limits for this batch are not reached
sendQueueMaxBytes41943040Max number of bytes to keep in the send queue. When the queue is full, incoming log events are dropped
maxRetries2Max number of attempts to send a batch to Loki before it will be dropped. A failed batch send could be retried in case of ConnectException, or receiving statuses 429, 503 from Loki. All other exceptions and 4xx-5xx statuses do not cause a retry in order to avoid duplicates
minRetryBackoffMs500Initial backoff delay before the next attempt to re-send a failed batch. Batches are retried with an exponential backoff (e.g. 0.5s, 1s, 2s, 4s, etc.) and jitter
maxRetryBackoffMs60000Maximum backoff delay before the next attempt to re-send a failed batch
maxRetryJitterMs500Upper bound for a jitter added to the retry delays
dropRateLimitedBatchesfalseIf true, batches that Loki responds to with a 429 status code (TooManyRequests) will be dropped rather than retried
internalQueuesCheckTimeoutMs25A timeout for Loki4j threads to sleep if encode or send queues are empty. Decreasing this value means lower latency at the cost of higher CPU usage
useDirectBufferstrueUse off-heap memory for storing intermediate data
drainOnStoptrueIf true, the appender will try to send all the remaining events on shutdown, so the proper shutdown procedure might take longer. Otherwise, the appender will drop the unsent events
metricsEnabledfalseIf true, the appender will report its metrics using Micrometer
verbosefalseIf true, the appender will print its own debug logs to stderr

HTTP settings

Loki4j uses Java HTTP client by default. If you want to switch to Apache HTTP client, please see this page. HTTP settings are the same for both Java And Apache clients:

SettingDefaultDescription
http.urlRequired. Loki endpoint to be used for sending batches
http.connectionTimeoutMs30000Time in milliseconds to wait for HTTP connection to Loki to be established before reporting an error
http.requestTimeoutMs5000Time in milliseconds to wait for HTTP request to Loki to be responded to before reporting an error
http.auth.usernameUsername to use for basic auth
http.auth.passwordPassword to use for basic auth
http.tenantIdTenant identifier. It is required only for sending logs directly to Loki operating in multi-tenant mode. Otherwise, this setting has no effect

Format settings

By default, Loki4j encodes log record batches to JSON before sending them to Loki API. If you want to use Protobuf encoding instead, please follow this guide. Format settings do not depend on the encoding you use.

SettingDefaultDescription
format.label.patternRequired. Logback pattern to use for log record's label
format.label.structuredMetadataPatternLogback pattern to use for log record's structured metadata
format.label.pairSeparator,Character sequence to use as a separator between labels
format.label.keyValueSeparator=Character to use as a separator between label's name and its value
format.label.readMarkersfalseIf true, Loki4j scans each log record for the attached LabelMarker to add its values to the record's labels
format.label.streamCacheBoundAtomicMapCacheAn implementation of a stream cache to use. By default, caches up to 1000 unique label sets
format.staticLabelsfalseIf you use only a constant label set (e.g., same keys and values) for all log records, you can set this flag to true and save some CPU and RAM

Plain text message layout

Plain text log message layout (backed by Logback's PatternLayout class) is used by default. It supports the following settings:

SettingDefaultDescription
format.message.patternRequired. Logback pattern to use for log record's message

JSON message layout

This layout converts log messages to JSON. Please check the instruction on how to enable and use it. This layout has the following settings:

SettingDefaultDescription
format.message.loggerName.enabledtrueEnable loggerName provider
format.message.loggerName.fieldNamelogger_nameA JSON field name to use for loggerName
format.message.loggerName.targetLength-1The desired target length of logger name: -1 to disable abbreviation, 0 to print class name only, >0 to abbreviate to the target length
format.message.logLevel.enabledtrueEnable logLevel provider
format.message.logLevel.fieldNamelevelA JSON field name to use for logLevel
format.message.kvp.enabledtrueEnable keyValuePair provider
format.message.kvp.prefixkvp_A prefix added to each JSON field name written by this provider
format.message.kvp.noPrefixfalseWhether to omit prefix for this provider
format.message.kvp.fieldSerializerAn implementation of field JSON serializer. By default, writeObjectField() is used
format.message.kvp.includeA set of keys to include in JSON payload. If not specified, all keys are included
format.message.kvp.excludeA set of keys to exclude from JSON payload. The exclude list has precedence over the include list. If not specified, all keys are included
format.message.mdc.enabledtrueEnable MDC provider
format.message.mdc.prefixmdc_A prefix added to each JSON field name written by this provider
format.message.mdc.noPrefixfalseWhether to omit prefix for this provider
format.message.mdc.includeA set of MDC keys to include in JSON payload. If not specified, all keys are included
format.message.mdc.excludeA set of MDC keys to exclude from JSON payload. The exclude list has precedence over the include list. If not specified, all keys are included
format.message.message.enabledtrueEnable message provider
format.message.message.fieldNamemessageA JSON field name to use for message
format.message.stackTrace.enabledtrueEnable stackTrace provider
format.message.stackTrace.fieldNamestack_traceA JSON field name to use for stackTrace
format.message.stackTrace.throwableConverterAn optional custom stack trace formatter
format.message.threadName.enabledtrueEnable threadName provider
format.message.threadName.fieldNamethread_nameA JSON field name to use for this threadName
format.message.timestamp.enabledtrueEnable timestamp provider
format.message.timestamp.fieldNametimestamp_msA JSON field name to use for timestamp
format.message.customProviderAn optional list of custom JSON providers

Examples

Minimalistic zero-dependency configuration

In Java 11 and later, we can use standard HTTP client and Loki JSON API. This setup is supported natively by Loki4j and does not require any extra dependencies. We need to define only required settings, leaving optional settings with their default values.

<appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
    <http>
        <url>http://localhost:3100/loki/api/v1/push</url>
    </http>
    <format>
        <label>
            <pattern>app=my-app,host=${HOSTNAME},level=%level</pattern>
        </label>
        <message>
            <pattern>l=%level h=${HOSTNAME} c=%logger{20} t=%thread | %msg %ex</pattern>
        </message>
    </format>
</appender>

Overriding the default settings

In this example, we would like to change max batch size to 100 records, batch timeout to 10s, label key-value separator to :, and sort log records by time before sending them to Loki. Messages should be in JSON format, without timestamp field, and with logger name abbreviated to 20 characters. Also, we would like to use Apache HTTP sender with request timeout 10s and Protobuf API. Finally, we want to see Loki4j debug output.

<appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
    <batchMaxItems>100</batchMaxItems>
    <batchTimeoutMs>10000</batchTimeoutMs>
    <verbose>true</verbose>
    <http class="com.github.loki4j.logback.ApacheHttpSender">
        <url>http://localhost:3100/loki/api/v1/push</url>
        <requestTimeoutMs>10000</requestTimeoutMs>
    </http>
    <format class="com.github.loki4j.logback.ProtobufEncoder">
        <label>
            <pattern>app:my-app,host:${HOSTNAME}</pattern>
            <keyValueSeparator>:</keyValueSeparator>
        </label>
        <message class="com.github.loki4j.logback.JsonLayout">
            <timestamp>
                <enabled>false</enabled>
            </timestamp>
            <loggerName>
                <targetLength>20</targetLength>
            </loggerName>
        </message>
    </format>
</appender>
Labels and structured metadata →
  • Reference
    • General settings
    • HTTP settings
    • Format settings
  • Examples
    • Minimalistic zero-dependency configuration
    • Overriding the default settings
Loki4j Logback
Docs
Reference
Community
GitHubLinkedIn
More
Powered by Docusaurus
Copyright © 2020-2025 Anton Nekhaev and Contributors