Refactor logging pages (#125)

This pull request includes significant updates to the QuestDB documentation, specifically focusing on logging and metrics. The changes aim to consolidate information, improve clarity, and provide a more comprehensive guide on configuring and understanding QuestDB logging and metrics. Documentation improvements: * [`documentation/operations/logging-metrics.md`](diffhunk://#diff-ff4c8d9bf93aa93f2cdfd38f82c4199b44552bb3fad9d40ae5ff2d5d91e7e8f8L2-R84): Updated the title and description to better reflect the content, and expanded the section to include detailed explanations of log levels, log file locations, and example log messages. * [`documentation/troubleshooting/log.md`](diffhunk://#diff-656c46d49688ae6fb9f756e42d63f805d32547815ac35ec58a6f661ca6ef4df4L1-L109): Removed redundant information that has now been consolidated into the `logging-metrics.md` file to avoid duplication and ensure consistency.
questdb · Jan 17, 2025 · 2e0e9a0 · 2e0e9a0
1 parent 6e4dd1f
commit 2e0e9a0
Show file tree

Hide file tree

Showing 6 changed files with 77 additions and 120 deletions.
diff --git a/documentation/concept/root-directory-structure.md b/documentation/concept/root-directory-structure.md
@@ -200,7 +200,7 @@ for this effect.
 
 ## `log` directory
 
-Contains the [log files](/docs/troubleshooting/log/) for QuestDB:
+Contains the [log files](/docs/operations/logging-metrics/) for QuestDB:
 
 ```filestructure
 ├── log

diff --git a/documentation/operations/logging-metrics.md b/documentation/operations/logging-metrics.md
@@ -1,20 +1,87 @@
 ---
-title: Logging & Metrics
-description:
-  Create various logs and customize a log.conf within QuestDB. Apply the
-  /metrics endpoint to interact with Prometheus.
+title: Logging and metrics
+description: Configure and understand QuestDB logging and metrics, including log levels, configuration options, and Prometheus integration.
 ---
 
 import { ConfigTable } from "@theme/ConfigTable"
 import httpMinimalConfig from "./_http-minimal.config.json"
 
-This page outlines logging, as configured in QuestDB's `log.conf` and metrics,
-accessible via Prometheus. Together, create robust outbound pipelines reporting
-what is happening with QuestDB.
+This page outlines logging in QuestDB. It covers how to configure logs via `log.conf` and expose metrics via Prometheus. 
 
 - [Logging](/docs/operations/logging-metrics/#logging)
 - [Metrics](/docs/operations/logging-metrics/#metrics)
 
+## Log location
+
+QuestDB creates the following file structure in its
+[root_directory](/docs/concept/root-directory-structure/):
+
+```filestructure
+questdb
+├── conf
+├── db
+├── log
+├── public
+└── snapshot (optional)
+```
+
+Log files are stored in the `log` folder:
+
+```filestructure
+├── log
+│   ├── stdout-2020-04-15T11-59-59.txt
+│   └── stdout-2020-04-12T13-31-22.txt
+```
+
+## Understanding log levels
+
+QuestDB provides the following types of log information:
+
+| Type     | Marker | Details                                                                                                                                                                                                           | Default  |
+| -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
+| Advisory | A      | Startup information such as hosts, listening ports, etc. Rarely used after startup                                                                                                                                | Enabled  |
+| Critical | C      | Internal database errors. Serious issues. Things that should not happen in general operation.                                                                                                                     | Enabled  |
+| Error    | E      | An error, usually (but not always) caused by a user action such as inserting a `symbol` into a `timestamp` column. For context on how this error happened, check for Info-level messages logged before the error. | Enabled  |
+| Info     | I      | Logs for activities. Info-level messages often provide context for an error if one is logged later.                                                                                                               | Enabled  |
+| Debug    | D      | Finer details on what is happening. Useful to debug issues.                                                                                                                                                       | Disabled |
+
+For more information, see the
+[QuestDB source code](https://github.com/questdb/questdb/blob/master/core/src/main/java/io/questdb/log/LogLevel.java).
+
+
+### Example log messages
+
+Advisory:
+```
+2023-02-24T14:59:45.076113Z A server-main Config:
+2023-02-24T14:59:45.076130Z A server-main  - http.enabled : true
+2023-02-24T14:59:45.076144Z A server-main  - tcp.enabled  : true
+2023-02-24T14:59:45.076159Z A server-main  - pg.enabled   : true
+```
+
+Critical:
+```
+2022-08-08T11:15:13.040767Z C i.q.c.p.WriterPool could not open [table=`sys.text_import_log`, thread=1, ex=could not open read-write [file=/opt/homebrew/var/questdb/db/sys.text_import_log/_todo_], errno=13]
+```
+
+Error:
+```
+2023-02-24T14:59:45.059012Z I i.q.c.t.t.InputFormatConfiguration loading input format config [resource=/text_loader.json]
+2023-03-20T08:38:17.076744Z E i.q.c.l.u.AbstractLineProtoUdpReceiver could not set receive buffer size [fd=140, size=8388608, errno=55]
+```
+
+Info:
+```
+2020-04-15T16:42:32.879970Z I i.q.c.TableReader new transaction [txn=2, transientRowCount=1, fixedRowCount=1, maxTimestamp=1585755801000000, attempts=0]
+2020-04-15T16:42:32.880051Z I i.q.g.FunctionParser call to_timestamp('2020-05-01:15:43:21','yyyy-MM-dd:HH:mm:ss') -> to_timestamp(Ss)
+```
+
+Debug:
+```
+2023-03-31T11:47:05.723715Z D i.q.g.FunctionParser call cast(investmentMill,INT) -> cast(Li)
+2023-03-31T11:47:05.723729Z D i.q.g.FunctionParser call rnd_symbol(4,4,4,2) -> rnd_symbol(iiii)
+```
+
 ## Logging
 
 The logging behavior of QuestDB may be set in dedicated configuration files or

diff --git a/documentation/operations/tls.md b/documentation/operations/tls.md
@@ -113,7 +113,7 @@ SELECT reload_tls();
 
 The function returns `true` if the reload is successful; otherwise, it returns
 `false`. In case of unsuccessful reload, you should check error messages in the
-[server logs](/docs/troubleshooting/log/) for more details.
+[server logs](/docs/operations/logging-metrics/) for more details.
 
 When Role-based Access Control (RBAC) is enabled, the `reload_tls()` SQL
 function is only available to the admin user, i.e. to the built-in user account

diff --git a/documentation/sidebars.js b/documentation/sidebars.js
@@ -478,7 +478,6 @@ module.exports = {
       type: "category",
       items: [
         "troubleshooting/faq",
-        "troubleshooting/log",
         "troubleshooting/os-error-codes",
       ],
     },

diff --git a/documentation/troubleshooting/faq.md b/documentation/troubleshooting/faq.md
@@ -14,7 +14,7 @@ Log files are stored in the `log` folder under the
 [root_directory](/docs/concept/root-directory-structure/). The log has the
 following levels to assist filtering:
 
-Check the [log](/docs/troubleshooting/log/) page for the available log levels.
+Check the [log](/docs/operations/logging-metrics/) page for the available log levels.
 
 ## How do I delete a row?
 

diff --git a/documentation/troubleshooting/log.md b/documentation/troubleshooting/log.md