Skip to content

Commit

Permalink
[skip ci] profreading
Browse files Browse the repository at this point in the history
  • Loading branch information
variar committed Jun 1, 2021
1 parent eae7781 commit 6cff6e9
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 9 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ title: "Version 20.4 released"
date: 2020-04-26T01:46+03:00
anchor: "v20_4"
weight: 45
bookhidden: true
aliases:
- /docs/news/release_20.4
---

## Version 20.4
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ title: "Version 20.9 preview"
date: 2020-04-26T01:46+03:00
anchor: "v20_9_preview"
weight: 40
bookhidden: true
aliases:
- /docs/news/release_20.9_preview
---

## Version 20.9
Expand Down
16 changes: 8 additions & 8 deletions website/content/docs/news/allocation.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Allocation matters"
date: 2021-04-16T01:46+03:00
anchor: "alloacation"
date: 2021-05-16T01:46+03:00
anchor: "allocation"
weight: 27
---

Expand Down Expand Up @@ -40,13 +40,13 @@ internal TBB malloc code. There is also some impact from the __TBB_malloc_safer_
Perf report shows that a lot of time is wasted destroying vectors of `QString`.
When search is executing there is one thread that reads raw data from the file,
and several threads that do searching through the blocks of lines. Block size is
configurable in settings. Typically it is 5000 or 10000 lines. When a thread
configurable in settings. Typically, it is 5000 or 10000 lines. When a thread
gets new raw data from file, it first transforms raw bytes to a vector of
`QString` objects converting each line from file text encoding to UTF16 (internal
Qt string representation). Finally each line is converted to UTF8 so it can be
Qt string representation). Finally, each line is converted to UTF8, so it can be
passed to the Hyperscan regex matching engine.

So during search operation klogg creates and destroys `QString` and `QByteArray`
So during search operation _Klogg_ creates and destroys `QString` and `QByteArray`
objects for each line in the file. `QString` is used to do conversion from
the file encoding to Qt internal representation and `QByteArray` holds
UTF8 data for the Hyperscan engine.
Expand All @@ -68,7 +68,7 @@ improvement.

### Switching application-wide memory allocator

Klogg has been using the scalable memory allocator provided by the Intel
_Klogg_ has been using the scalable memory allocator provided by the Intel
TBB library for several years. It is designed to work well for multi-threaded
applications. Using it instead of the default system memory allocator resulted in
5-10% performance improvement. It is very easy to use the TBB malloc to override
Expand Down Expand Up @@ -96,8 +96,8 @@ use of the LD_PRELOAD environment variable. So I used static override option. To
klogg executable is linked with the mimalloc-override.o file. This file must be provided first so
the linker chooses symbols from it for all global memory allocation functions.

After some more performance testing it looks like the mimalloc allocator is indeed faster in typical
klogg use cases. Both indexing and search are around 5% faster.
After doing some more performance testing it looks like the mimalloc allocator is indeed faster in typical
klogg use cases. Both indexing and search are around 10% faster.

Final perf report now looks like this:
```
Expand Down
2 changes: 1 addition & 1 deletion website/content/docs/news/hyperscan.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Overall search is now about 2 times faster. And there is still room for improvem
However, there is a downside. Hyperscan library needs boost and ragel to compile.
And compilation takes a lot of time. CI builds now run 2 times slower.

One more drawback of Hyprescan library is that does not support full PCRE2 syntax.
One more drawback of Hyperscan library is that does not support full PCRE2 syntax.
In particular these constructs are not supported:

- Backreferences and capturing sub-expressions.
Expand Down

0 comments on commit 6cff6e9

Please sign in to comment.