Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beeper 1.114.0 brad #44

Open
wants to merge 34 commits into
base: upstream-1.114.0
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
873dade
Add prometheus histograms for room shutdown/delete/purge
Fizzadar Mar 12, 2024
b74698c
Install shared secret authenticator in Docker image
Fizzadar Mar 8, 2024
06a5eb3
Update Beeper base rules
Fizzadar Mar 8, 2024
f767ff9
Add JWT UI auth flow
Fizzadar Mar 8, 2024
4aaaa8c
Update email copy for Beeper
Fizzadar Mar 8, 2024
00f1432
Add Beeper log formatter
Fizzadar Mar 8, 2024
db600a7
Add Beeper previews
Fizzadar Mar 8, 2024
39cf8f8
Add Beeper inbox endpoints
Fizzadar Mar 8, 2024
8ebff95
Remove any bridge bot users from ignored user account data
Fizzadar Mar 8, 2024
c8040b3
Add Beeper notification counts
Fizzadar Aug 5, 2024
f2fe26f
Include stream ordering in the unsigned event dict
Fizzadar Mar 8, 2024
feb6420
Allow clients to add extra content to receipts
Fizzadar Mar 8, 2024
2531973
Include empty identity server in well known
Fizzadar Oct 25, 2024
2a20f37
Allow checking username availability when registration disabled
Fizzadar Mar 8, 2024
54bbbc6
Change default DM room redaction power level to 100
Fizzadar Mar 8, 2024
5dcb148
Disable transaction timeout when purging rooms
Fizzadar Mar 8, 2024
b78225f
Send all notifications as high priority
Fizzadar Mar 8, 2024
b2bca1f
Add Beeper specific test cases
Fizzadar Mar 8, 2024
64fe3e9
Add CI
Fizzadar Mar 8, 2024
3f7490f
Remove dependabot config
Fizzadar Mar 8, 2024
0e729ff
Add Beeper readme
Fizzadar Mar 8, 2024
400d0c5
Add Beeper release flow scripts
Fizzadar Mar 25, 2024
ef1b8c1
Add env var to disable writing to the `user_ips` table
Fizzadar Mar 28, 2024
e963de2
Increase last seen granularity to 1h
Fizzadar Mar 28, 2024
934fae1
Bump sytest/complement commits
Fizzadar May 21, 2024
14f8fff
Add at_order property to inbox_state endpoint
bradtgmurray Aug 13, 2024
d5eb076
Look for done.at_order in the correct place in inbox_state endpoint
bradtgmurray Aug 13, 2024
de26ee5
Bump sytest + complement commits
Fizzadar Sep 9, 2024
6e8db23
Add note to readme about real fork
Fizzadar Sep 9, 2024
e9da5db
Enable admin APIs on synapse workers
Fizzadar Oct 4, 2024
311bc0e
Bump CI action
Fizzadar Oct 4, 2024
6509f0a
Register the admin user servelet in generic worker
Fizzadar Oct 4, 2024
84604d5
Enable correct admin servelet
Fizzadar Oct 4, 2024
f7bd701
Replace isort and black with ruff in Beeper CI
bradtgmurray Dec 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions .github/dependabot.yml

This file was deleted.

137 changes: 137 additions & 0 deletions .github/workflows/beeper-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
name: Beep

on:
push:
branches: ["beeper", "beeper-*"]
pull_request:


jobs:
lint-style:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install poetry
- run: poetry install
- run: poetry run ruff check --output-format=github .

lint-types:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install poetry
- run: poetry install --extras all
- run: poetry run mypy

# Tests

test-trial:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- run: pip install poetry
- run: poetry install --extras all
- run: poetry run trial -j4 tests

test-sytest:
runs-on: ubuntu-latest
container:
image: matrixdotorg/sytest-synapse:focal
volumes:
- ${{ github.workspace }}:/src
env:
SYTEST_BRANCH: 91a3ab32fcb3ada91423d661125c428e6b2b7abe
TOP: ${{ github.workspace }}
POSTGRES: 1
MULTI_POSTGRES: 1
WOKRERS: 1
steps:
- uses: actions/checkout@v2
- name: Run SyTest
run: /bootstrap.sh synapse
working-directory: /src
- name: Summarise results.tap
if: ${{ always() }}
run: /sytest/scripts/tap_to_gha.pl /logs/results.tap
- name: Upload SyTest logs
uses: actions/upload-artifact@v4
if: ${{ always() }}
with:
name: Sytest Logs - ${{ job.status }} - (${{ join(matrix.*, ', ') }})
path: |
/logs/results.tap
/logs/**/*.log*

test-complement:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v2
with:
repository: matrix-org/complement
path: complement
ref: 39733c1b2f8314800776748cc7164f9a34650686
- name: Install complement dependencies
run: |-
sudo apt-get -qq update
sudo apt-get install -qqy libolm3 libolm-dev
go install -v github.com/gotesttools/gotestfmt/v2/cmd/gotestfmt@latest
- name: Run Complement
run: ./scripts-dev/complement.sh
env:
COMPLEMENT_DIR: complement

# Builds

build-python:
runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: 1
steps:
- uses: actions/checkout@v2
- uses: docker/setup-qemu-action@v2
- uses: docker/setup-buildx-action@v2
with:
# TEMPORARY, see: https://github.com/docker/build-push-action/issues/761
driver-opts: |
image=moby/buildkit:v0.10.6
- uses: docker/login-action@v2
with:
registry: ${{ secrets.CI_REGISTRY }}
username: ${{ secrets.CI_REGISTRY_USER }}
password: ${{ secrets.CI_REGISTRY_PASSWORD }}
- run: |-
if [ "${{ github.ref_name }}" = "beeper" ]; then
tag=$(cat pyproject.toml | grep -E "^version =" | sed -E 's/^version = "(.+)"$/\1/')
else
tag="${{ github.head_ref || github.ref_name }}"
fi

docker buildx build \
--push \
--platform linux/amd64 \
--tag ${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }} \
-f docker/Dockerfile \
.

if [ "${{ github.ref_name }}" = "beeper" ]; then
docker pull ${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }}
docker tag \
${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }} \
${{ secrets.CI_REGISTRY }}/synapse:latest
docker push ${{ secrets.CI_REGISTRY }}/synapse:latest
fi

# Ensure the image works properly
docker run \
--entrypoint '' \
${{ secrets.CI_REGISTRY }}/synapse:$tag-${{ github.sha }} \
python -m synapse.app.homeserver --help

echo "Pushed image: synapse:$tag-${{ github.sha }}"
if [ "${{ github.ref_name }}" = "beeper" ]; then
echo "Pushed image: synapse:latest"
fi
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Synapse: Beeper Edition

This is Beeper's custom version of synapse, we rebase roughly 25 commits on top of each upstream release with a few Beeper specific modifications. We also have an actual Synapse fork here: [**beeper/synapse-fork**](https://github.com/beeper/synapse-fork) which is where we make changes we expect to merge into upstream.

## Rebase flow

### Create PR

Here we're upgrading to `v1.96.1`:

```
# Make a new branch from the upstream release, we do this so we can create a PR
# of Beeper -> upstream to run tests/confirm we're happy.
git checkout -f v1.96.1
git checkout -b upstream-1.96.1
git push -u beeper upstream-1.96.1

# Check out the base branch, pull any changes
git checkout beeper
git pull

# Now create a new branch to rebase
git checkout -b beeper-1.96.1
# And do the rebase
git rebase v1.96.1
# fix any conflicts...

# Push and make a PR from this branch to the upstream one created above
git push -u beeper beeper-1.96.1
```

### Make release

Once it's ready we just overwrite the `beeper` branch with the new one:

```
git checkout beeper-1.96.1
git push --force beeper beeper
```
31 changes: 31 additions & 0 deletions beeper/complete_release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env bash

set -euo pipefail
source $(realpath $(dirname $0))/utils.sh

BEEPER_REMOTE=$(get_beeper_remote)

VERSION=${1:-}

if [ -z "$VERSION" ]; then
echo >&2 "Must specify version!"
exit 1
fi

echo "Completing Synapse: Beeper Edition version $VERSION"
echo "WARNING: this script will DELETE the branch called: beeper"
read -p "Press enter to continue"

UPSTREAM_BRANCH=upstream-$VERSION
BEEPER_BRANCH=beeper-$VERSION

git checkout $BEEPER_BRANCH
git branch -D beeper
git checkout -b beeper
git push --force $BEEPER_REMOTE beeper

# Cleanup
git branch -D $BEEPER_BRANCH
git push $BEEPER_REMOTE --delete $BEEPER_BRANCH
git branch -D $UPSTREAM_BRANCH
git push $BEEPER_REMOTE --delete $UPSTREAM_BRANCH
43 changes: 43 additions & 0 deletions beeper/prepare_release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/usr/bin/env bash

set -euo pipefail
source $(realpath $(dirname $0))/utils.sh

BEEPER_REMOTE=$(get_beeper_remote)

VERSION=${1:-}

if [ -z "$VERSION" ]; then
echo >&2 "Must specify version!"
exit 1
fi

STARTING_BRANCH=$(git branch --show-current)

echo "Preparing Synapse: Beeper Edition version $VERSION"
echo "WARNING: this script will rebase on top of the CURRENT BRANCH: $STARTING_BRANCH"
read -p "Press enter to continue"

TAG=v$VERSION
UPSTREAM_BRANCH=upstream-$VERSION
BEEPER_BRANCH=beeper-$VERSION

# Checkout the tag, create upstream branch, push it
echo "Setup branch $UPSTREAM_BRANCH"
git checkout -f $TAG
git checkout -b $UPSTREAM_BRANCH
git push -u $BEEPER_REMOTE $UPSTREAM_BRANCH

# Switch back to our starting branch, create new version branch from it
echo "Setup branch $BEEPER_BRANCH"
git checkout $STARTING_BRANCH
git checkout -b $BEEPER_BRANCH

# And rebase against upstream, applying only our Beeper commits
echo "Initiate rebase..."
git rebase $UPSTREAM_BRANCH || read -p "Rebase was a mess, press enter once you fix it"

git push -u $BEEPER_REMOTE $BEEPER_BRANCH

echo "OK we done!"
echo "Go HERE and make the PR: https://github.com/beeper/synapse/compare/upstream-$VERSION...beeper-$VERSION?expand=1"
23 changes: 23 additions & 0 deletions beeper/utils.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
function get_upstream_remote() {
for remote in $(git remote); do
url=$(git remote get-url $remote)
if [ "$url" = "git@github.com:element-hq/synapse.git" ]; then
echo $remote
return 0
fi
done
echo >&2 "No upstream remote found (looking for URL: git@github.com:element-hq/synapse.git)"
return 1
}

function get_beeper_remote() {
for remote in $(git remote); do
url=$(git remote get-url $remote)
if [ "$url" = "git@github.com:beeper/synapse.git" ]; then
echo $remote
return 0
fi
done
echo >&2 "No upstream remote found (looking for URL: git@github.com:beeper/synapse.git)"
return 1
}
4 changes: 4 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,10 @@ COPY --from=requirements /synapse/requirements.txt /synapse/
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --prefix="/install" --no-deps --no-warn-script-location -r /synapse/requirements.txt

# Beeper: install shared secret authenticator
RUN pip install --prefix="/install" --no-deps --no-warn-script-location \
'git+https://github.com/devture/matrix-synapse-shared-secret-auth@e178353ec87c56e0169dd04466d4769da5ed9c46#egg=shared_secret_authenticator'

# Copy over the rest of the synapse source code.
COPY synapse /synapse/synapse/
COPY rust /synapse/rust/
Expand Down
Loading
Loading