Full Configuration โ€” Collect Everything

This document is the single reference for enabling every telemetry signal Sentrinel can collect. It covers the backend plugin (Elysia, Express, Next.js, Bun) and the Flutter SDK side by side, with every option explained and every data type mapped to the dashboard page that displays it.

If you want "just works, show me everything" โ€” copy the first config block for each side and you are done. The rest of the document explains what each option does and why it exists.


Table of contents

  1. Backend config: Elysia
  2. Backend config: Express
  3. Backend config: Next.js
  4. Backend config: Bun native
  5. Flutter config
  6. Browser config
  7. OpenTelemetry without a plugin
  8. Backend environment variables
  9. What each signal captures
  10. Data masking
  11. Sampling safety
  12. Distributed tracing
  13. Structured logging deep dive
  14. What you see in the dashboard
  15. Troubleshooting

Backend config: Elysia

This enables every signal the backend plugin supports: request metrics, request logs with payloads, console log capture, structured logging, error tracking, distributed tracing, resource monitoring, consumer tracking, and deploy markers.

import { Elysia } from "elysia";
import { sentrinelPlugin } from "@sentrinel/plugin";

const app = new Elysia()
  .use(sentrinelPlugin({
    // โ”€โ”€ Core โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    serverUrl: "http://localhost:3001",            // your Sentrinel API URL
    appName: "my-api",                             // shown in dashboard
    env: "prod",                                   // "dev" | "staging" | "prod"
    apiKey: process.env.SENTRINEL_API_KEY,         // per-app ingest key
    version: process.env.GIT_SHA,                  // deploy markers on charts
    debug: false,                                  // verbose plugin logging

    // โ”€โ”€ Flush control โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    flushInterval: 30000,                          // ms between batch sends

    // โ”€โ”€ Consumer tracking โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    consumerIdentifier: (ctx) =>
      ctx.request.headers.get("x-consumer-id") ?? "unknown",

    // โ”€โ”€ Path exclusion โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    excludePaths: ["/health", "/metrics", /^\/internal\//],

    // โ”€โ”€ Request logging (raw request/response rows) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    requestLogging: {
      enabled: true,                               // turn on request log rows
      sampleRate: 1.0,                             // 100% of fast successes
      slowRequestThresholdMs: 500,                 // always capture slow ones
      logRequestHeaders: true,                     // include request headers
      logRequestBody: true,                        // include request body
      logResponseBody: true,                       // include response body
      maxBodySize: 65536,                          // 64KB max body capture
      maskHeaders: [/^authorization$/i, /^cookie$/i, /^x-api-key$/i],
      maskQueryParams: ["token", "secret", "key"],
      maskBodyFields: [/^password$/i, /^token$/i, /^credit_card$/i, /^cvv$/i],
    },

    // โ”€โ”€ Console log capture โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    logCapture: {
      enabled: true,                               // capture console.* output
      minLevel: "debug",                           // "debug" = capture everything
      maxPerRequest: 100,                          // cap per request
      maxMessageLength: 2000,                      // truncate long messages
    },

    // โ”€โ”€ Structured logging โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    logging: {
      minLevel: "debug",                           // drop nothing before buffering
      echo: true,                                  // mirror to stdout in dev
    },
  }))
  .get("/", () => "hello")
  .listen(3000);

What each option does

Option What it controls Why it matters
serverUrl Where telemetry is sent Must point to the API, not the dashboard
appName Groups all data for this service Must match the API key's app
env Scopes to an environment Staging key cannot write to prod
apiKey Authenticates ingestion Required when SENTRINEL_REQUIRE_INGEST_KEY=true
version Deploy markers Charts get vertical lines at deploy time
flushInterval How often batches are sent Lower = more realtime, higher = less network
consumerIdentifier Identifies the API client Powers the Consumers dashboard page
excludePaths Ignores noisy endpoints Health checks, metrics scrapes, internal routes
requestLogging.enabled Writes request log rows Required for Request Logs, Apdex, payloads
requestLogging.sampleRate Fraction of fast successes kept 0.1 = 10%, errors/slow always kept
requestLogging.slowRequestThresholdMs Always-capture threshold Requests above this are never sampled out
requestLogging.logRequestHeaders Includes headers in logs Shows in the Headers tab
requestLogging.logRequestBody Includes request body Shows in the Payloads tab. Read before your handler, from a clone โ€” a route that verifies a raw signature still sees its own bytes
requestLogging.logResponseBody Includes response body Shows in the Payloads tab
requestLogging.maxBodySize Truncates large bodies Prevents memory bloat
requestLogging.maskHeaders Redacts sensitive headers Runs in-process before anything leaves
requestLogging.maskQueryParams Redacts sensitive query params Prevents tokens/keys from leaking
requestLogging.maskBodyFields Redacts sensitive body fields Recursively masks nested JSON
logCapture.enabled Captures console.* output Correlates logs to the request that produced them
logCapture.minLevel Minimum level to capture "debug" = everything, "warn" = warnings+
logCapture.maxPerRequest Per-request log cap Prevents hot loops from flooding the buffer
logCapture.maxMessageLength Truncates long messages Keeps storage bounded
logging.minLevel Drops records before buffering "debug" = nothing dropped
logging.echo Mirrors to stdout Useful in dev, off in prod

When a request shows no payload

An empty Payloads tab on a write request is usually correct. Two cases produce one, and they are worth telling apart before assuming capture is broken:

Anything else with a non-zero requestSize and no recorded body is a bug worth reporting.


Backend config: Express

import express from "express";
import { sentrinelExpressMiddleware } from "@sentrinel/plugin/express";

const app = express();

app.use(sentrinelExpressMiddleware({
  serverUrl: "http://localhost:3001",
  appName: "my-express-api",
  env: "prod",
  apiKey: process.env.SENTRINEL_API_KEY,
  version: process.env.GIT_SHA,

  consumerIdentifier: "x-consumer-id",           // header name (string)

  excludePaths: ["/health", "/metrics"],

  requestLogging: {
    enabled: true,
    sampleRate: 1.0,
    slowRequestThresholdMs: 500,
    logRequestHeaders: true,
    logRequestBody: true,
    logResponseBody: true,
    maxBodySize: 65536,
    maskHeaders: [/^authorization$/i, /^cookie$/i],
    maskBodyFields: [/^password$/i, /^token$/i],
  },

  logCapture: {
    enabled: true,
    minLevel: "debug",
    maxPerRequest: 100,
  },

  logging: {
    minLevel: "debug",
    echo: false,
  },
}));

app.get("/users", (req, res) => {
  res.json(getUsers());
});

app.listen(3000);

Important: Mount the middleware before your routes so it sees every request.


Backend config: Next.js

// middleware.ts (App Router)
import { sentrinelNextMiddleware } from "@sentrinel/plugin/next";

const sentrinel = sentrinelNextMiddleware({
  serverUrl: process.env.SENTRINEL_URL!,
  appName: "my-next-app",
  env: process.env.VERCEL_ENV ?? "prod",
  apiKey: process.env.SENTRINEL_API_KEY,
  version: process.env.VERCEL_GIT_COMMIT_SHA,

  requestLogging: {
    enabled: true,
    sampleRate: 1.0,
    slowRequestThresholdMs: 500,
    logRequestHeaders: true,
    logRequestBody: true,
    logResponseBody: true,
    maxBodySize: 65536,
    maskHeaders: [/^authorization$/i, /^cookie$/i],
    maskBodyFields: [/^password$/i, /^token$/i],
  },

  logCapture: {
    enabled: true,
    minLevel: "info",
    maxPerRequest: 50,
  },

  logging: {
    minLevel: "debug",
    echo: false,
  },
});

export function middleware(request: Request) {
  return sentrinel.middleware(request);
}

export const config = {
  matcher: "/api/:path*",
};

For Pages Router API routes, wrap each handler:

import { withSentrinel } from "@sentrinel/plugin/next";

export default withSentrinel({
  serverUrl: process.env.SENTRINEL_URL!,
  appName: "my-next-app",
  env: process.env.VERCEL_ENV ?? "prod",
}, async function handler(req, res) {
  res.json({ data: "hello" });
});

Backend config: Bun native

For raw Bun.serve without Elysia:

import { sentrinelBunMiddleware } from "@sentrinel/plugin/bun";

Bun.serve({
  port: 3000,
  fetch: sentrinelBunMiddleware({
    serverUrl: "http://localhost:3001",
    appName: "my-bun-app",
    env: "prod",
    apiKey: process.env.SENTRINEL_API_KEY,
    version: process.env.GIT_SHA,

    consumerIdentifier: "x-consumer-id",

    requestLogging: {
      enabled: true,
      sampleRate: 1.0,
      slowRequestThresholdMs: 500,
      logRequestHeaders: true,
      logRequestBody: true,
      logResponseBody: true,
    },

    logCapture: {
      enabled: true,
      minLevel: "debug",
    },

    logging: {
      minLevel: "debug",
      echo: false,
    },
  }, async (req) => {
    return new Response("Hello");
  }),
});

Flutter config

The Flutter SDK captures HTTP requests, errors, crashes, release health sessions, performance frames, and structured logs. It sends traceparent headers so backend traces connect automatically.

import 'package:flutter/material.dart';
import 'package:sentrinel_flutter/sentrinel_flutter.dart';

void main() => SentrinelFlutter.run(
  options: SentrinelOptions(
    serverUrl: 'https://api.sentrinel.dev',
    appName: 'mobile-app',
    env: 'prod',
    apiKey: const String.fromEnvironment('SENTRINEL_API_KEY'),
    release: '1.4.2',           // critical for release health
    flushInterval: Duration(seconds: 30),
  ),
  app: () => runApp(const MyApp()),
);

// In your root widget:
class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      // Navigation tracking + breadcrumbs
      navigatorObservers: [SentrinelNavigatorObserver()],
      home: HomeScreen(),
    );
  }
}

Key rule: runApp goes inside the app callback. Async errors are only caught within the zone โ€” calling runApp outside it silently misses most of them.

Using the core Dart package (non-Flutter)

import 'package:sentrinel/sentrinel.dart';

void main() {
  Sentrinel.init(
    serverUrl: 'https://api.sentrinel.dev',
    appName: 'my-dart-app',
    env: 'prod',
    apiKey: const String.fromEnvironment('SENTRINEL_API_KEY'),
    release: '1.4.2',
    consumerIdentifier: 'ios_app',  // or 'android_app', user id, etc.
  );
  runApp(const MyApp());
}

// HTTP client โ€” every request is recorded + traceparent is sent
final client = Sentrinel.httpClient();

// Dio adapter
final dio = Dio();
dio.httpClientAdapter = SentrinelHttpClientAdapter();

What gets captured automatically

Signal How What you see
HTTP requests Sentrinel.httpClient() Method, URL, status, latency, size
Crashes runZonedGuarded + FlutterError.onError Stack trace, device info, breadcrumbs
Release health Session per app launch Crash-free rate per release
Performance Frame timing + app start Slow/frozen frames, start-to-first-frame
Distributed tracing traceparent header Connects Flutter โ†’ backend in one trace
Breadcrumbs HTTP + navigation auto, manual add Last 25 events before crash/error

Capturing errors manually

// Uncaught errors (in your zone handler)
runZonedGuarded(() {
  // ...
}, (error, stack) {
  Sentrinel.captureError(error, stack, path: 'main');
});

// Caught errors
try {
  await api.submitPost(post);
} catch (e, stack) {
  Sentrinel.captureError(e, stack, path: 'submit_post');
  rethrow;
}

Adding context

// After login โ€” attaches to every later record
Sentrinel.setContext({
  'userId': user.id,
  'tier': user.subscriptionTier,
  'locale': Localizations.localeOf(context).languageCode,
});

// On logout
Sentrinel.clearContext();

Adding breadcrumbs

Sentrinel.addBreadcrumb('tapped pay', category: 'ui', data: {'amount': 42});

Structured logs

Sentrinel.info('User signed in', category: 'auth', attributes: {
  'method': 'google',
});

Sentrinel.warn('Cache miss', category: 'cache', attributes: {
  'key': 'feed_v2',
});

Sentrinel.error('Payment failed', category: 'billing', attributes: {
  'orderId': order.id,
  'reason': 'insufficient_funds',
});

Flush on background

When the app goes to the background, flush buffered telemetry so nothing is lost (especially crash reports):

class _MyAppState extends State<MyApp> with WidgetsBindingObserver {
  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance.addObserver(this);
  }

  @override
  void didChangeAppLifecycleState(AppLifecycleState state) {
    if (state == AppLifecycleState.paused ||
        state == AppLifecycleState.detached) {
      Sentrinel.flush();
    }
  }

  @override
  void dispose() {
    WidgetsBinding.instance.removeObserver(this);
    super.dispose();
  }
}

Full Flutter example

import 'dart:async';
import 'package:flutter/material.dart';
import 'package:sentrinel_flutter/sentrinel_flutter.dart';

void main() => SentrinelFlutter.run(
  options: SentrinelOptions(
    serverUrl: 'https://api.sentrinel.dev',
    appName: 'my-app',
    env: 'prod',
    apiKey: const String.fromEnvironment('SENTRINEL_API_KEY'),
    release: '1.4.2',
  ),
  app: () => runApp(const MyApp()),
);

class MyApp extends StatefulWidget {
  const MyApp({super.key});

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> with WidgetsBindingObserver {
  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance.addObserver(this);
  }

  @override
  void didChangeAppLifecycleState(AppLifecycleState state) {
    if (state == AppLifecycleState.paused ||
        state == AppLifecycleState.detached) {
      Sentrinel.flush();
    }
  }

  @override
  void dispose() {
    WidgetsBinding.instance.removeObserver(this);
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      navigatorObservers: [SentrinelNavigatorObserver()],
      home: Scaffold(
        body: Center(
          child: ElevatedButton(
            onPressed: () async {
              final client = Sentrinel.httpClient();
              final res = await client.get(
                Uri.parse('https://api.example.com/posts'),
              );
              // Process response...
            },
            child: const Text('Load Posts'),
          ),
        ),
      ),
    );
  }
}

Browser config

A web app is the third surface, and the only one where the API key cannot go in the config. Everything in a JavaScript bundle is served to every visitor, and an ingest key cannot be scoped down after the fact. So the setup is two halves: the page holds no secret and posts to your own origin, and a server route forwards with the key.

Client โ€” everything on

import { initSentrinelBrowser } from '@sentrinel/plugin/browser';

// Before hydration or first render. An error thrown *during* hydration is one
// of the most common real failures, and a handler installed after misses it.
export const sentrinel = initSentrinelBrowser({
  // Same origin, so no key ships in the bundle.
  endpoint: '/api/sentrinel',

  // Crash-free rate is per release. Without this every build reads `unknown`.
  release: '2026.8.2',

  // โ”€โ”€ What to collect โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  captureErrors: true,       // window errors + unhandled rejections
  captureRequests: true,     // patch fetch: timing, status, trace propagation
  captureBreadcrumbs: true,  // clicks, SPA navigation, requests
  trackSessions: true,       // one per page load โ€” release health's denominator

  // โ”€โ”€ Volume โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  sampleRate: 1,                 // successful requests kept
  slowRequestThresholdMs: 2_000, // above this, kept regardless of sampleRate
  flushInterval: 10_000,
  maxBatch: 100,                 // flush early once this many records queue

  // โ”€โ”€ Tracing โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  // Same-origin only by default: sending trace ids to a third party leaks them
  // and turns a simple request into one needing CORS preflight.
  tracePropagationTargets: [/^\//, 'https://api.myapp.com'],

  // โ”€โ”€ Noise โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  // On top of the built-in list (`Script error.`, ResizeObserver loops,
  // browser-extension frames).
  ignoreErrors: ['AbortError', 'The user aborted a request'],

  // โ”€โ”€ Redaction โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  // Runs before the value is ever on the wire โ€” the only place that helps for
  // data that should not leave the device at all. Return null to drop.
  beforeSend: (record) => {
    if (record.kind === 'request' && record.queryParams?.token) {
      record.queryParams.token = '[redacted]';
    }
    return record;
  },

  debug: import.meta.env.DEV,
});

Server โ€” the tunnel

Any runtime that speaks Request/Response: Bun, Elysia, Hono, Next route handlers, TanStack Start, Remix, Deno, Workers.

import { createSentrinelTunnel } from '@sentrinel/plugin/tunnel';

export const POST = createSentrinelTunnel({
  serverUrl: 'https://api.sentrinel.dev',

  // Pinned here, never read from the batch. A browser-supplied app name is a
  // claim from an untrusted source โ€” without pinning, anyone who found the
  // endpoint could write into a different app in your account.
  appName: 'admin',
  env: 'prod',

  apiKey: process.env.SENTRINEL_API_KEY!,   // never reaches the client
  release: process.env.GIT_SHA,

  // An open endpoint that will buffer anything is a memory-exhaustion target.
  maxBodyBytes: 256 * 1024,

  // Read server-side and overrides whatever the browser claimed. An identity a
  // page asserts about itself is a claim, not a fact.
  consumerIdentifier: (request) => userIdFromSession(request),

  beforeForward: (batch, request) => batch,  // null drops the batch

  debug: process.env.NODE_ENV !== 'production',
});

What gets captured automatically

Signal Source
Uncaught errors โ†’ session crashed window error event
Unhandled rejections โ†’ session crashed unhandledrejection
Handled errors โ†’ session errored captureError()
Request timing, status, query params patched fetch
Network failures (status 0) patched fetch
Breadcrumbs: clicks, navigation, requests DOM + history patch
One session per page load pagehide / visibilitychange
Language, user agent, cores, connection, screen navigator, screen

Manual API

sentrinel.captureError(err, { orderId: 'o_1' });
sentrinel.addBreadcrumb('tapped pay', { category: 'ui', data: { amount: 42 } });
sentrinel.setUser('user_123');       // null on sign-out
sentrinel.setContext({ tier: 'pro' });
await sentrinel.flush();
await sentrinel.close();             // unpatch, end the session, send

Not covered

Source maps (a minified stack stays minified), web vitals, XMLHttpRequest (only fetch is patched), and session replay.

Full detail: BROWSER.md.


OpenTelemetry without a plugin

Any OpenTelemetry SDK or Collector can export straight to Sentrinel. This works for any language โ€” Go, Rust, Java, Python, Ruby, .NET โ€” not just JS/Dart.

OTEL_EXPORTER_OTLP_ENDPOINT=https://api.sentrinel.dev
OTEL_EXPORTER_OTLP_PROTOCOL=http/json
OTEL_EXPORTER_OTLP_HEADERS=x-api-key=snt_live_your_key_here

GenAI spans (gen_ai.request.model + gen_ai.usage.*_tokens) are priced automatically and appear on the LLM cost dashboard page.

What gets collected

OTel signal Dashboard page
Traces / Spans Traces waterfall
GenAI span attributes LLM cost
Resource attributes App auto-provisioning

Backend environment variables

These control the Sentrinel API server itself. Set them in .env or your deployment platform (Dokploy, Docker, etc.).

# โ”€โ”€ Storage โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
DATABASE_URL=postgresql://sentrinel:<password>@postgres:5432/sentrinel
TELEMETRY_STORE=clickhouse                # "postgres" or "clickhouse"
CLICKHOUSE_DB=sentrinel
CLICKHOUSE_USER=sentrinel
CLICKHOUSE_PASSWORD=<password>
CLICKHOUSE_URL=http://clickhouse:8123

# โ”€โ”€ Network โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
SENTRINEL_API_PORT=3001
PUBLIC_API_URL=https://api.sentrinel.dev   # compiled into dashboard at build time
PUBLIC_DASHBOARD_URL=https://app.sentrinel.dev

# โ”€โ”€ Auth & security โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
SENTRINEL_REQUIRE_AUTH=true               # enforce session auth on dashboard routes
SENTRINEL_REQUIRE_INGEST_KEY=true         # require API keys for ingestion
SENTRINEL_DISABLE_SIGNUP=false            # lock signup after first user
SENTRINEL_ALLOWED_ORIGINS=https://app.sentrinel.dev,http://localhost:3000
SENTRINEL_COOKIE_SAMESITE=lax
SENTRINEL_COOKIE_DOMAIN=

# โ”€โ”€ Billing, trials & the operator console โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
SENTRINEL_ADMIN_EMAILS=                   # comma-separated; empty = nobody is an admin
SENTRINEL_TRIAL_DAYS=14                   # 0 disables trials (self-hosted wants this)
SENTRINEL_INGEST_GRACE_DAYS=7             # keep accepting telemetry this long after a block

# โ”€โ”€ Retention โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
RETENTION_LOGS_DAYS=30                    # raw request/app log retention
RETENTION_METRICS_DAYS=400                # aggregated metrics (~13 months)
RETENTION_REPLAY_DAYS=14                  # session replay recordings

# โ”€โ”€ Connection pools โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
DATABASE_POOL_SIZE=10
DATABASE_READ_POOL_SIZE=10
DATABASE_REPLICA_URL=                     # optional read replica

Key settings explained

Variable Why it matters
TELEMETRY_STORE=clickhouse Production recommended. Postgres works for dev/small deployments.
SENTRINEL_REQUIRE_INGEST_KEY=true Don't turn this off. Prevents unauthenticated data writes.
PUBLIC_API_URL Dashboard builds need this at compile time. Changing it requires a rebuild.
RETENTION_LOGS_DAYS Controls disk usage. 30 days is a good default.
RETENTION_METRICS_DAYS 400 days gives ~13 months of history.
RETENTION_REPLAY_DAYS Session recordings are the largest thing stored per unit of value, so they get their own, shorter clock. Nobody watches a replay from six weeks ago.
SENTRINEL_ADMIN_EMAILS Who may open Platform admin and change any org's billing. Deliberately an env var, not a database column: a column is one compromised signup away from being set, an env var needs redeploy access. Fails closed โ€” leave it empty and nobody is an admin, including you.
SENTRINEL_TRIAL_DAYS How long a newly created org may use Sentrinel before paying. Existing orgs are never backfilled onto a trial clock โ€” they stay active. Set to 0 for self-hosted, where there is nobody to bill.
SENTRINEL_INGEST_GRACE_DAYS How long telemetry keeps being accepted after the dashboard blocks. Cutting ingest instantly means that by the time someone pays, the window they needed is gone. After this, ingest answers 402.

What each signal captures

Request metrics (always collected)

In-memory aggregation, flushed every 30s. Exact regardless of sampling.

Field Description
Request count Total requests per endpoint per minute
Success count Requests with status < 400
Error count Requests with status >= 400
Response time avg, min, max, p50, p95, p99
Request/response size Bytes transferred
Status codes Distribution (200, 404, 500, etc.)

Dashboard pages: Traffic, Performance, Errors, Consumers

Request logs (when requestLogging.enabled: true)

Individual request/response records with full detail.

Field Description
Method, path, status HTTP method, URL path, status code
Response time Latency in milliseconds
Request/response size Bytes
Consumer identifier Which client made the call
Request headers Full headers (masked)
Request body Full body (masked, truncated)
Response body Full body (masked, truncated)
Query parameters URL params (masked)
Error message If status >= 400
Trace ID Links to distributed trace
Business context Fields from addRequestContext()
Sample rate How this record was sampled

Dashboard pages: Request Logs, Traffic (endpoint detail), Performance (Apdex)

Application logs (when logging.enabled or logCapture.enabled)

Structured log records correlated to requests.

Field Description
Level debug, info, warn, error
Message Stable, groupable string
Category Hierarchical (e.g., "api.checkout")
Attributes Structured fields (user_id, amount, etc.)
Request ID Links to the request
Trace ID Links to the distributed trace
Span ID Links to the specific span
Sequence Order within request

Dashboard pages: Logs, Request Logs (Logs tab)

Errors (auto-collected for status >= 400)

Fingerprinted and grouped into issues with occurrence counting.

Field Description
Error type Exception class or HTTP status
Error message Human-readable description
Stack trace Full call stack (when available)
Fingerprint 16-char hash for grouping
Title Auto-generated from type + message
Culprit Top frame in your code
Occurrences How many times this bug fired
First/last seen Time range
Regression Auto-detected if resolved bug reappears

Dashboard pages: Issues, Errors

Distributed traces (auto-collected for every request)

Field Description
Trace ID Unique per request
Root span The HTTP request itself
Child spans DB queries, external calls, custom spans
Span kind SERVER, CLIENT, DB, INTERNAL, PRODUCER, CONSUMER
Duration Per-span timing
Attributes Custom key-value pairs
Status OK or ERROR

Dashboard pages: Traces (waterfall), Request Logs (Trace tab)

Resource metrics (auto-collected)

Field Description
CPU usage Percentage
Memory RSS Resident Set Size
Memory heap total Total heap allocation
Memory heap used Currently used heap

Dashboard pages: Resources

LLM cost (auto-detected from GenAI spans)

Field Description
Provider openai, anthropic, google, etc.
Model gpt-4o, claude-3, etc.
Input/output tokens Per-call token counts
Cost (USD) Auto-computed from price table
Consumer Which client made the call
Endpoint Which API route triggered it

Dashboard pages: LLM cost

Release health (Flutter sessions)

Field Description
Session ID Unique per app launch
Status ok, crashed, abnormal, errored
Release version Which build
Environment prod, staging, dev
Device OS/version iOS/Android version
Duration Session length

Dashboard pages: Release Health


Data masking

Masking runs in your process, before anything leaves it. A masked field never reaches the network, let alone Sentrinel's disk.

Headers to mask

maskHeaders: [
  /^authorization$/i,    // auth tokens
  /^cookie$/i,           // session cookies
  /^x-api-key$/i,        // API keys
  /^x-auth-token$/i,     // custom auth
],

Query parameters to mask

maskQueryParams: [
  "token",               // exact match
  "secret",              // exact match
  /^key$/i,              // regex match
],

Body fields to mask

maskBodyFields: [
  /^password$/i,         // password fields
  /^token$/i,            // token fields
  /^credit_card$/i,      // payment fields
  /^cvv$/i,              // security codes
  /^ssn$/i,              // social security
  /^secret$/i,           // any secret
],

maskBodyFields descends into nested JSON objects. A field named password at any depth is masked.


Sampling safety

Three properties make sampling safe:

  1. Errors and slow requests are never sampled out. sampleRate applies only to successful, fast requests โ€” the ones you were never going to read.

  2. Metrics stay exact. Counters are computed in-process before sampling, so a 1% sample rate still yields correct request counts and error rates. Only the individual log rows are thinned.

  3. Every row records the rate it was captured at, so counts derived from raw logs can be extrapolated honestly rather than silently under-reporting.

Traffic level sampleRate Why
Development 1.0 You want every request
Low (< 100 req/s) 0.5 Keep half, still plenty of detail
Medium (100-1k req/s) 0.1 10% is enough for debugging
High (> 1k req/s) 0.01 1% for volume, errors always kept

Distributed tracing

Traces connect the full request path: Flutter tap โ†’ API โ†’ database โ†’ external service. The root span is always the HTTP request. Child spans are created with traceSpan().

Manual spans

import { traceSpan, traced } from "@sentrinel/plugin";

// Wrap a block
const user = await traceSpan("db.findUser", async (span) => {
  span.setAttribute("db.system", "postgresql");
  span.setAttribute("db.operation", "SELECT");
  return db.users.findById(id);
});

// Wrap a function at definition
const getProducts = traced("db.getProducts", () => db.products.find());

Auto-trace an entire service

import { traceObject } from "@sentrinel/plugin";

const db = traceObject(new DatabaseClient(), "db");
// Every method on db is now auto-traced as "db.*"

Cross-service calls

import { sentrinelFetch } from "@sentrinel/plugin";

// Automatically propagates traceparent header
const res = await sentrinelFetch("https://payments.internal/v1/charges", {
  method: "POST",
  body: JSON.stringify({ amount }),
});

Flutter โ†’ backend correlation

The Flutter SDK automatically sends traceparent on every HTTP request. The backend plugin picks it up and continues the same trace. No extra configuration needed โ€” just make sure both sides use the same Sentrinel API.

Span kinds

Use the right kind for accurate waterfall coloring:

Kind Use for
SERVER Incoming HTTP request (auto-created)
CLIENT Outgoing HTTP call
DB Database query
INTERNAL Business logic
PRODUCER Message queue publish
CONSUMER Message queue consume

Structured logging deep dive

Why structured logging

console.log(\user ${id} failed checkout`)` is easy to write and impossible to query. You cannot filter by user, group by failure reason, or tie it to the request that produced it.

Structured logging fixes this by separating the message (stable, groupable) from the attributes (filterable fields).

Context is inherited, not repeated

withContext({ userId: user.id, tenantId: org.id, tier: user.tier }, async () => {
  await processOrder();
  // every log inside carries those three fields
  // even three levels deep, in code that never heard of a user
});

Categories are hierarchical

const log = getLogger(["api", "checkout"]);
// logs under "api.checkout"

const sub = log.getChild("db");
// logs under "api.checkout.db"

// Filter by "api" to see everything in the API subsystem

Canonical wide events

Instead of scattered log lines, attach rich context to the request itself:

addRequestContext({
  tier: user.tier,
  cartTotal,
  experiment: "new-pricing",
  outcome: "success",
});

"Slow checkouts for enterprise customers in the new pricing experiment" becomes one query rather than a correlation exercise.

Everything is linked


What you see in the dashboard

Dashboard page Requires Shows
Apps Plugin installed Overview cards per service
Traffic Metrics (auto) Request volume, endpoints, response times
Errors Errors (auto for >= 400) Error rates, status breakdown
Issues Errors with stack traces Fingerprinted error groups, occurrence counts
Performance Metrics (auto) + Request logs p50/p95/p99, Apdex, slowest endpoints
Consumers Consumer identifier set Per-client metrics, usage breakdown
Request logs requestLogging.enabled: true Individual request entries with full detail
Logs logging.enabled or logCapture.enabled Structured logs with filtering
Traces traceSpan() or OTLP Distributed trace waterfalls
Resources Metrics (auto) CPU and memory over time
Uptime Health checks configured HTTP checks, uptime percentage
Alerts Alert rules configured Threshold alerts with notifications
SLOs SLOs configured Error budget, burn-down
Cron monitors Monitors configured Job check-in status
LLM cost GenAI spans Token usage, cost by model
Release health Flutter release set Crash-free rate per release
Dashboards Custom widgets Arranged views
Status pages Public config Customer-facing status
SQL console Any telemetry data Read-only SQL queries

Troubleshooting

Nothing shows up

  1. Check API health: curl http://localhost:3001/health
  2. Check serverUrl: Must point to the API, not the dashboard
  3. Check API key: Must match appName + env
  4. Check requestLogging.enabled: true: Metrics work without it, request logs don't
  5. Check excludePaths: Your endpoint might be excluded

Logs not appearing

Traces missing

Flutter crashes not showing

403 errors in logs

The API key doesn't match appName + env. Create a new key in the dashboard that matches both values.


Reference: all plugin options

Option Type Default Backend Flutter
serverUrl string โ€” required required
appName string โ€” required required
env string "dev" required required
apiKey string โ€” recommended recommended
version / release string โ€” deploy markers crash-free rate
flushInterval number / Duration 30000 / 30s batch interval batch interval
consumerIdentifier string / (ctx) => string โ€” client ID client ID
excludePaths (string|RegExp)[] [] paths to ignore โ€”
debug boolean false verbose logging โ€”
requestLogging.enabled boolean false request log rows โ€”
requestLogging.sampleRate number 1 fraction kept โ€”
requestLogging.slowRequestThresholdMs number 2000 always-capture ms โ€”
requestLogging.logRequestHeaders boolean false include headers โ€”
requestLogging.logRequestBody boolean false include req body โ€”
requestLogging.logResponseBody boolean false include res body โ€”
requestLogging.maxBodySize number 65536 body truncation โ€”
requestLogging.maskHeaders (string|RegExp)[] [] redact headers โ€”
requestLogging.maskQueryParams (string|RegExp)[] [] redact params โ€”
requestLogging.maskBodyFields (string|RegExp)[] [] redact body fields โ€”
logCapture.enabled boolean false capture console.* โ€”
logCapture.minLevel string "info" min level โ€”
logCapture.maxPerRequest number 50 per-request cap โ€”
logCapture.maxMessageLength number 2000 truncation โ€”
logging.minLevel string "debug" min buffered level โ€”
logging.echo boolean false mirror to stdout โ€”