diff --git a/documentation/concept/root-directory-structure.md b/documentation/concept/root-directory-structure.md index 2ea388fb..c150e2ca 100644 --- a/documentation/concept/root-directory-structure.md +++ b/documentation/concept/root-directory-structure.md @@ -200,7 +200,7 @@ for this effect. ## `log` directory -Contains the [log files](/docs/troubleshooting/log/) for QuestDB: +Contains the [log files](/docs/operations/logging-metrics/) for QuestDB: ```filestructure ├── log diff --git a/documentation/operations/logging-metrics.md b/documentation/operations/logging-metrics.md index fed12cef..04687f75 100644 --- a/documentation/operations/logging-metrics.md +++ b/documentation/operations/logging-metrics.md @@ -1,20 +1,87 @@ --- -title: Logging & Metrics -description: - Create various logs and customize a log.conf within QuestDB. Apply the - /metrics endpoint to interact with Prometheus. +title: Logging and metrics +description: Configure and understand QuestDB logging and metrics, including log levels, configuration options, and Prometheus integration. --- import { ConfigTable } from "@theme/ConfigTable" import httpMinimalConfig from "./_http-minimal.config.json" -This page outlines logging, as configured in QuestDB's `log.conf` and metrics, -accessible via Prometheus. Together, create robust outbound pipelines reporting -what is happening with QuestDB. +This page outlines logging in QuestDB. It covers how to configure logs via `log.conf` and expose metrics via Prometheus. - [Logging](/docs/operations/logging-metrics/#logging) - [Metrics](/docs/operations/logging-metrics/#metrics) +## Log location + +QuestDB creates the following file structure in its +[root_directory](/docs/concept/root-directory-structure/): + +```filestructure +questdb +├── conf +├── db +├── log +├── public +└── snapshot (optional) +``` + +Log files are stored in the `log` folder: + +```filestructure +├── log +│   ├── stdout-2020-04-15T11-59-59.txt +│   └── stdout-2020-04-12T13-31-22.txt +``` + +## Understanding log levels + +QuestDB provides the following types of log information: + +| Type | Marker | Details | Default | +| -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | +| Advisory | A | Startup information such as hosts, listening ports, etc. Rarely used after startup | Enabled | +| Critical | C | Internal database errors. Serious issues. Things that should not happen in general operation. | Enabled | +| Error | E | An error, usually (but not always) caused by a user action such as inserting a `symbol` into a `timestamp` column. For context on how this error happened, check for Info-level messages logged before the error. | Enabled | +| Info | I | Logs for activities. Info-level messages often provide context for an error if one is logged later. | Enabled | +| Debug | D | Finer details on what is happening. Useful to debug issues. | Disabled | + +For more information, see the +[QuestDB source code](https://github.com/questdb/questdb/blob/master/core/src/main/java/io/questdb/log/LogLevel.java). + + +### Example log messages + +Advisory: +``` +2023-02-24T14:59:45.076113Z A server-main Config: +2023-02-24T14:59:45.076130Z A server-main - http.enabled : true +2023-02-24T14:59:45.076144Z A server-main - tcp.enabled : true +2023-02-24T14:59:45.076159Z A server-main - pg.enabled : true +``` + +Critical: +``` +2022-08-08T11:15:13.040767Z C i.q.c.p.WriterPool could not open [table=`sys.text_import_log`, thread=1, ex=could not open read-write [file=/opt/homebrew/var/questdb/db/sys.text_import_log/_todo_], errno=13] +``` + +Error: +``` +2023-02-24T14:59:45.059012Z I i.q.c.t.t.InputFormatConfiguration loading input format config [resource=/text_loader.json] +2023-03-20T08:38:17.076744Z E i.q.c.l.u.AbstractLineProtoUdpReceiver could not set receive buffer size [fd=140, size=8388608, errno=55] +``` + +Info: +``` +2020-04-15T16:42:32.879970Z I i.q.c.TableReader new transaction [txn=2, transientRowCount=1, fixedRowCount=1, maxTimestamp=1585755801000000, attempts=0] +2020-04-15T16:42:32.880051Z I i.q.g.FunctionParser call to_timestamp('2020-05-01:15:43:21','yyyy-MM-dd:HH:mm:ss') -> to_timestamp(Ss) +``` + +Debug: +``` +2023-03-31T11:47:05.723715Z D i.q.g.FunctionParser call cast(investmentMill,INT) -> cast(Li) +2023-03-31T11:47:05.723729Z D i.q.g.FunctionParser call rnd_symbol(4,4,4,2) -> rnd_symbol(iiii) +``` + ## Logging The logging behavior of QuestDB may be set in dedicated configuration files or diff --git a/documentation/operations/tls.md b/documentation/operations/tls.md index 5b119851..461129d1 100644 --- a/documentation/operations/tls.md +++ b/documentation/operations/tls.md @@ -113,7 +113,7 @@ SELECT reload_tls(); The function returns `true` if the reload is successful; otherwise, it returns `false`. In case of unsuccessful reload, you should check error messages in the -[server logs](/docs/troubleshooting/log/) for more details. +[server logs](/docs/operations/logging-metrics/) for more details. When Role-based Access Control (RBAC) is enabled, the `reload_tls()` SQL function is only available to the admin user, i.e. to the built-in user account diff --git a/documentation/sidebars.js b/documentation/sidebars.js index c59937a8..01c6dfbb 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -478,7 +478,6 @@ module.exports = { type: "category", items: [ "troubleshooting/faq", - "troubleshooting/log", "troubleshooting/os-error-codes", ], }, diff --git a/documentation/troubleshooting/faq.md b/documentation/troubleshooting/faq.md index c3263b66..d1270864 100644 --- a/documentation/troubleshooting/faq.md +++ b/documentation/troubleshooting/faq.md @@ -14,7 +14,7 @@ Log files are stored in the `log` folder under the [root_directory](/docs/concept/root-directory-structure/). The log has the following levels to assist filtering: -Check the [log](/docs/troubleshooting/log/) page for the available log levels. +Check the [log](/docs/operations/logging-metrics/) page for the available log levels. ## How do I delete a row? diff --git a/documentation/troubleshooting/log.md b/documentation/troubleshooting/log.md deleted file mode 100644 index 440e0bae..00000000 --- a/documentation/troubleshooting/log.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -title: Log -description: Details about QuestDB log ---- - -QuestDB log provides database information. This page presents log level, their -details, and where to configure and find the log. - -## Log location - -### QuestDB open source - -QuestDB creates the following file structure in its -[root_directory](/docs/concept/root-directory-structure/): - -```filestructure -questdb -├── conf -├── db -├── log -├── public -└── snapshot (optional) -``` - -Log files are stored in the `log` folder: - -```filestructure -├── log -│   ├── stdout-2020-04-15T11-59-59.txt -│   └── stdout-2020-04-12T13-31-22.txt -``` - -## Log levels - -QuestDB log provides the following types of log information: - -| Type | Marker | Details | Default | -| -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -| Advisory | A | Startup information such as hosts, listening ports, etc. Rarely used after startup | Enabled | -| Critical | C | Internal database errors. Serious issues. Things that should not happen in general operation. | Enabled | -| Error | E | An error, usually (but not always) caused by a user action such as inserting a `symbol` into a `timestamp` column. For context on how this error happened, check for Info-level messages logged before the error. | Enabled | -| Info | I | Logs for activities. Info-level messages often provide context for an error if one is logged later. | Enabled | -| Debug | D | Finer details on what is happening. Useful to debug issues. | Disabled | - -For more information, see the -[source code](https://github.com/questdb/questdb/blob/master/core/src/main/java/io/questdb/log/LogLevel.java) -on GH. - -### Log examples - -The below is some examples of log messages by type. - -Advisory: - -``` -2023-02-24T14:59:45.076113Z A server-main Config: -2023-02-24T14:59:45.076130Z A server-main - http.enabled : true -2023-02-24T14:59:45.076144Z A server-main - tcp.enabled : true -2023-02-24T14:59:45.076159Z A server-main - pg.enabled : true -``` - -Critical: - -``` - -2022-08-08T11:15:13.040767Z C i.q.c.p.WriterPool could not open [table=`sys.text_import_log`, thread=1, ex=could not open read-write [file=/opt/homebrew/var/questdb/db/sys.text_import_log/_todo_], errno=13] -... -2022-08-23T07:55:15.490045Z C i.q.c.h.p.JsonQueryProcessorState [77] internal error [q=`REINDEX TABLE 'weather' COLUMN timestamp LOCK EXCLUSIVE;`, ex= -... -2022-11-17T10:28:00.464140Z C i.q.c.h.p.JsonQueryProcessorState [7] Uh-oh. Error! -``` - -Error: - -``` -2023-02-24T14:59:45.059012Z I i.q.c.t.t.InputFormatConfiguration loading input format config [resource=/text_loader.json] -... -2023-03-20T08:38:17.076744Z E i.q.c.l.u.AbstractLineProtoUdpReceiver could not set receive buffer size [fd=140, size=8388608, errno=55] -... -2022-08-03T11:13:41.947760Z E server-main [errno=48] could not bind socket [who=http-server, bindTo=0.0.0.0:9000] -``` - -Info: - -``` -2020-04-15T16:42:32.879970Z I i.q.c.TableReader new transaction [txn=2, transientRowCount=1, fixedRowCount=1, maxTimestamp=1585755801000000, attempts=0] -2020-04-15T16:42:32.880051Z I i.q.g.FunctionParser call to_timestamp('2020-05-01:15:43:21','yyyy-MM-dd:HH:mm:ss') -> to_timestamp(Ss) -2020-04-15T16:42:32.880657Z I i.q.c.p.WriterPool >> [table=`table_1`, thread=12] -2020-04-15T16:42:32.881330Z I i.q.c.AppendMemory truncated and closed [fd=32] -2020-04-15T16:42:32.881448Z I i.q.c.AppendMemory open /usr/local/var/questdb/db/table_1/2020-05/timestamp.d [fd=32, pageSize=16777216] -2020-04-15T16:42:32.881708Z I i.q.c.AppendMemory truncated and closed [fd=33] -``` - -Debug: - -``` -2023-03-31T11:47:05.723715Z D i.q.g.FunctionParser call cast(investmentMill,INT) -> cast(Li) -2023-03-31T11:47:05.723729Z D i.q.g.FunctionParser call rnd_symbol(4,4,4,2) -> rnd_symbol(iiii) -2023-03-31T11:47:05.723749Z D i.q.g.FunctionParser call investmentMill * 950399 -> *(LL) -2023-03-31T11:47:05.723794Z D i.q.g.FunctionParser call cast(1672531200000000L,TIMESTAMP) -> cast(Ln) -2023-03-31T11:47:05.723803Z D i.q.g.FunctionParser call cast(1672531200000000L,TIMESTAMP) + investmentMill * 950399 -> +(NL) -``` - -## Log configuration - -QuestDB logging can be configured globally to `DEBUG` via either providing the -java option `-Debug` or setting the environment variable `QDB_DEBUG=true`. See -[reference manual](/docs/operations/logging-metrics/#logging) for details on -logging configuration.