Skip to content

Commit

Permalink
Setup asdf and direnv managed development environment
Browse files Browse the repository at this point in the history
Now `cd` into the project directory would activate correct versions of build
dependencies.
  • Loading branch information
iilyak committed Nov 5, 2021
1 parent 057e101 commit dbd29a0
Show file tree
Hide file tree
Showing 5 changed files with 147 additions and 4 deletions.
3 changes: 3 additions & 0 deletions .envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
use asdf
watch_file ".asdf"
dotenv_if_exists .envrc.local
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,7 @@ build.log
build.log
.cache
clouseau.iml

# direnv & asdf
.asdf
.envrc.local
4 changes: 4 additions & 0 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
maven 3.8.2
java zulu-7.48.0.11
scala 2.9.1.final

29 changes: 29 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
.PHONY: build
# target: build - Build package, run tests and create distribution
build: .asdf
@mvn

# target: tools - Bootstrap all build tools
PHONY: tools
tools: .asdf
@touch .envrc

.PHONY: run
# target: run - Start local inistance of clouseau
run: .asdf
@mvn scala:run -Dlauncher=clouseau1

.PHONY: help
# target: help - Print this help
help:
@egrep "^# target: " Makefile \
| sed -e 's/^# target: //g' \
| sort \
| awk '{printf(" %-20s", $$1); $$1=$$2=""; print "-" $$0}'

.asdf: .tool-versions
@cat $< | cut -d' ' -f 1 | xargs -I {} sh -c '\
asdf list {} > /dev/null 2>&1 \
|| asdf plugin-add {}'
@asdf install
@touch $@
111 changes: 107 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,111 @@

Expose Lucene features to erlang RPC.

## Setting up the development environment

We use a combination of [`direnv`](https://github.com/direnv/direnv) and [`asdf`](https://github.com/asdf-vm/asdf) to manage development environment.
The minimal set of tools you need to install manually is:

- xcode
- brew
- asdf
- git
- coreutils

1. install asdf itself
```
brew install asdf
```
2. integrate asdf with your shell
- fish
```
echo -e "\nsource $(brew --prefix asdf)/libexec/asdf.fish" >> ~/.config/fish/config.fish
```
- bash
```
echo -e "\n. $(brew --prefix asdf)/libexec/asdf.sh" >> ~/.bash_profile
```
- zsh
```
echo -e "\n. $(brew --prefix asdf)/libexec/asdf.sh" >> ${ZDOTDIR:-~}/.zshrc
```
3. install direnv plugin
```
asdf plugin-add direnv
asdf install direnv latest
asdf global direnv latest
```
4. integrate direnv plugin with your shell
- fish
```
printf "\
# Hook direnv into your shell.
asdf exec direnv hook fish | source
# A shortcut for asdf managed direnv.
function direnv
asdf exec direnv \"\$argv\"
end
" >> ~/.config/fish/config.fish
```
- bash
```
cat << EOF >> ~/.bashrc
# Hook direnv into your shell.
eval "$(asdf exec direnv hook bash)"
# A shortcut for asdf managed direnv.
direnv() { asdf exec direnv "$@"; }
EOF
```
- zsh
```
cat << EOF >> ~/.zshrc
# Hook direnv into your shell.
eval "$(asdf exec direnv hook zsh)"
# A shortcut for asdf managed direnv.
direnv() { asdf exec direnv "$@"; }
EOF
```
5. enable `use asdf` feature
```
echo 'source "$(asdf direnv hook asdf)"' >> ~/.config/direnv/direnvrc
```
6. exit from the current shell or start a new one
7. `cd` into the directory where you checked out `clouseau` and do `direnv allow`
8. do `make .asdf` (we need it only when we add new tools into `.tool-versions`)

Now every time you `cd` into the project directory the correct versions of tools
will be activated.

### Custom Java

In some cases version of Java is not available in asdf-java.
There is a trick which makes globally available version recognized by
asdf.

```
ln -s /Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk/Contents/Home ~/.asdf/installs/java/oracle-1.7.0
asdf reshim java
```

## Building distribution

Just run `make` to get `target/clouseau-{version}-SNAPSHOT.zip` file.

## Running locally

`make run`

## Changing tools version

The list of tools are configured in a `.tool-versions` file. Make sure you
run `make tools` every time you add new tools. You don't need to do it when
there is just a version change.

## Configuration options
This guide explains the various clouseau configuration options available, and how to use them to tune clouseau performance and scalability. There are two categories of clouseau options, first category is about tuning the JVM (ex: Xmx) and other category of options that go into clouseau.ini.
This guide explains the various clouseau configuration options available, and how to use them to tune clouseau performance and scalability. There are two categories of clouseau options, first category is about tuning the JVM (ex: Xmx) and other category of options that go into clouseau.ini.

Clouseau configuration options (as determined by the relevant role in chef-repo) are stored in `/opt/clouseau/etc/clouseau.ini` and some options (about JVM tuning) go into the command used to start and stop clouseau.

Expand Down Expand Up @@ -52,7 +155,7 @@ This option allows to specify the maximum number indices opened at a given point
These options allows us to close the search indices if there is no activity within a specified interval. As mentioned in the above section about the `max_indexes_open`, when number of indices opened reaches the `max_indexes_open` limit then clouseau will close the index that was opened first even if there is an activity on that index. This was leading to errors and hence added this option to close the idle indices first.
Basically we will close the idle indices so that we can avoid reaching the limit specified in `max_indexes_open`. So that we will close the idle indices first and if we still reach the limit then clouseau will close the oldest index (based on the open time).

If `close_if_idle` is set to true, then it will start monitoring the activity of the indices and will close the index if there is no activity in two consecutive idle check intervals. By default the `idle_check_interval_secs` is 300 seconds (5 minutes) and can be overridden by specifying the `idle_check_interval_secs`. In the example configuration specified in the above section, this was set to 600 seconds, which will close the index if there is no activity for anywhere between 601 to 1200 seconds.
If `close_if_idle` is set to true, then it will start monitoring the activity of the indices and will close the index if there is no activity in two consecutive idle check intervals. By default the `idle_check_interval_secs` is 300 seconds (5 minutes) and can be overridden by specifying the `idle_check_interval_secs`. In the example configuration specified in the above section, this was set to 600 seconds, which will close the index if there is no activity for anywhere between 601 to 1200 seconds.

This variation (instead of fixed) in the index closing due to inactivity is a result of the implementation approach, where during the first index idle check interval we mark the index as idle and close that index if the state is still idle during the next check. And if there is any activity between the two `index idle check` intervals then it will not be closed.

Expand All @@ -64,7 +167,7 @@ This variation (instead of fixed) in the index closing due to inactivity is a re
s:maintenance_mode("heap dump of clouseau").
```
2. Find the clouseau pid (process: "java [...] com.cloudant.clouseau.Main /opt/cloudant/etc/clouseau.ini" ):
```
ps aux | grep clouseau
```
Expand All @@ -84,5 +187,5 @@ This variation (instead of fixed) in the index closing due to inactivity is a re
```
rsync -avz db<X>.<CLUSTER>.cloudant.com:/srv/heap.bin .
```
7. Analyze the heap dump using `visualVM` tool or Eclipse memory analyzer tool(http://www.eclipse.org/mat/downloads.php).

0 comments on commit dbd29a0

Please sign in to comment.