Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature aofv4 #4

Open
wants to merge 26 commits into
base: v4
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
51ae40f
put on aof airs
bug-superman Jun 18, 2023
389c602
add the aof mainifest
bug-superman Jun 24, 2023
3a869c5
fix the file name
bug-superman Jun 24, 2023
c1f9387
add the aof check
bug-superman Jun 24, 2023
a405cd5
7/22
hwy1314 Jul 22, 2023
2e10736
Implemented the basic framework of aof.go.
hwy1314 Jul 25, 2023
7b7a6b3
Implemented the basic framework of aofcheck.go.
hwy1314 Jul 29, 2023
1c6e5fd
Further improved the functions of aof.go and checkaof.go.
hwy1314 Aug 8, 2023
b1f8511
Implemented the reading function of most AOF files. But the cluster c…
hwy1314 Aug 21, 2023
3d321a1
Fixed some bugs in reading AOF and changed the format of some code.
hwy1314 Aug 23, 2023
f68eaa8
Fixed some errors related to reading AOF files, removed some unnecess…
hwy1314 Aug 26, 2023
28bee42
Fixed a bug where line breaks cross the bufio. Reader cache area caus…
hwy1314 Aug 29, 2023
8f79194
Fixed a bug regarding aof offset and changed the way timestamps are c…
hwy1314 Sep 4, 2023
50031b9
Removed redundant checkaof functionality and combined it with v4 func…
hwy1314 Sep 16, 2023
7b2d7d1
Fixed some AOF related bugs in V4 versions, including timestamp and A…
hwy1314 Sep 18, 2023
72e1881
Adapted to the v4 version and created a new. go file prasing_ AOF ,fu…
hwy1314 Sep 19, 2023
3480a72
Improved the code format a bit.
hwy1314 Sep 20, 2023
d7a76aa
Added test files for AOF.
hwy1314 Sep 21, 2023
19f4878
Update ci.yml
hwy1314 Sep 21, 2023
cb70f44
Update ci.yml
hwy1314 Sep 21, 2023
c9db618
Update ci.yml
hwy1314 Sep 21, 2023
696b93f
Added a simple feature to filter dangerous commands and fixed some er…
hwy1314 Sep 29, 2023
fb2176b
Added Python testing, removed some redundant code, optimized code str…
hwy1314 Oct 7, 2023
a1fa728
Simplified the original list structure, removed a large amount of red…
hwy1314 Oct 8, 2023
fbe2b13
Reduced some redundant code and modified some malformed code.
hwy1314 Oct 17, 2023
0da1584
Added the function of directly reading incrAOF files and historyAOF f…
hwy1314 Oct 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
name: Feature request
about: 功能性建议(Suggest an idea for this project)
title: ''
labels: 'type: feature request'
assignees: suxb201

---

请描述你的功能性建议。清晰地说明你希望添加或改进的功能,并尽可能提供详细的信息。

Please describe your feature request. Clearly state the functionality you would like to add or improve and provide as much detail as possible.
48 changes: 21 additions & 27 deletions .github/ISSUE_TEMPLATE/issue.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,34 @@
---
name: issue
about: 大多数问题请使用这个模板
name: Issue
about: 使用中遇到问题(Encountering problems while using)
title: ''
labels: ''
labels: 'type: question'
assignees: ''

---

- [ ] 请确保已经看过 wiki:https://github.com/alibaba/RedisShake/wiki
- [ ] 请确保已经学习过 Markdown 语法,良好的排版有助于维护人员了解你的问题
- [ ] 请在此提供足够的信息供社区维护人员排查问题
- [ ] 请在提交 issue 前删除此模板中多余的文字,包括这几句话
### 问题描述(Issue Description)

**问题描述**
请在这里简要描述你遇到的问题。

这里描述你的问题。
Please provide a brief description of the issue you encountered.

redis-shake 的日志:
```
shake 的日志贴在这里
```
### 环境信息(Environment)

---
- RedisShake 版本(RedisShake Version):
- Redis 源端版本(Redis Source Version):
- Redis 目的端版本(Redis Destination Version):
- Redis 部署方式(standalone/cluster/sentinel):
- 是否在云服务商实例上部署(Deployed on Cloud Provider):

**源端 Redis**
版本:版本号,自建还是云厂商?
日志:
```
日志贴在这里
```
### 日志信息(Logs)

---
如果有错误日志或其他相关日志,请在这里提供。

If there are any error logs or other relevant logs, please provide them here.

### 其他信息(Additional Information)

请提供任何其他相关的信息,如配置文件、错误信息或截图等。

**目的端 Redis**
版本:版本号,是否是集群?自建还是云厂商?
日志:
```
日志贴在这里
```
Please provide any additional information, such as configuration files, error messages, or screenshots.
1 change: 1 addition & 0 deletions .github/release-drafter.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# https://github.com/release-drafter/release-drafter#example
name-template: 'redis-shake-v$RESOLVED_VERSION'
tag-template: 'v$RESOLVED_VERSION'
template: |
Expand Down
63 changes: 55 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,21 @@
name: CI

on: [ pull_request ]
on:
push:
branches: [ v4 ]
pull_request:
branches: [ v4 ]
workflow_dispatch:

jobs:
black-box-test:
runs-on: ubuntu-latest
strategy:
max-parallel: 1
matrix:
redis-version: [ 5, 6, 7 ]
redis-version: [ "2.8", "3.0", "4.0", "5.0", "6.0", "7.0" ]
fail-fast: false
container: ubuntu:latest
steps:
- name: Git checkout
uses: actions/checkout@v2
Expand All @@ -19,26 +27,65 @@ jobs:

- name: clone and make redis
run: |
sudo apt-get install git
apt-get update
apt-get install -y --no-install-recommends git build-essential ca-certificates
git clone https://github.com/redis/redis
cd redis
git checkout ${{ matrix.redis-version }}.0
git checkout ${{ matrix.redis-version }}
make -j
mkdir bin
cp src/redis-server bin/redis-server
echo "$GITHUB_WORKSPACE/redis/bin" >> $GITHUB_PATH

- name: clone and make TairString Module
if: contains( '5.0, 6.0, 7.0', matrix.redis-version)
run: |
cd $GITHUB_WORKSPACE
apt-get install -y cmake
git clone https://github.com/tair-opensource/TairString.git
cd TairString
mkdir build
cd build
cmake ../ && make -j
cp $GITHUB_WORKSPACE/TairString/lib/tairstring_module.so /lib



- name: clone and make TairHash Module
if: contains( '5.0, 6.0, 7.0', matrix.redis-version)
run: |
cd $GITHUB_WORKSPACE
git clone https://github.com/tair-opensource/TairHash.git
cd TairHash
mkdir build
cd build
cmake ../ && make -j
cp $GITHUB_WORKSPACE/TairHash/lib/tairhash_module.so /lib



- name: clone and make TairZset Module
if: contains( '5.0, 6.0, 7.0', matrix.redis-version)
run: |
cd $GITHUB_WORKSPACE
git clone https://github.com/tair-opensource/TairZset.git
cd TairZset
mkdir build
cd build
cmake ../ && make -j
cp $GITHUB_WORKSPACE/TairZset/lib/tairzset_module.so /lib


- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
python-version: '3.11'

- name: make redis-shake
run: |
sh build.sh

- name: test
run: |
cd test
pip3 install -r requirements.txt
python3 main.py
python -m pip install -r tests/requirements.txt
sh test.sh
63 changes: 63 additions & 0 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Sample workflow for building and deploying a VitePress site to GitHub Pages
#
name: Pages

on:
# Runs on pushes targeting the `main` branch. Change this to `master` if you're
# using the `master` branch as the default branch.
push:
branches: [ v4 ]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: pages
cancel-in-progress: false

jobs:
# Build job
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: 18
cache: npm
cache-dependency-path: docs/package-lock.json
- name: Setup Pages
uses: actions/configure-pages@v3
- name: Install dependencies
run: npm ci # or pnpm install / yarn install
working-directory: docs
- name: Build with VitePress
run: npm run docs:build # or pnpm docs:build / yarn docs:build
working-directory: docs
- name: Upload artifact
uses: actions/upload-pages-artifact@v2
with:
path: docs/.vitepress/dist

# Deployment job
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
needs: build
runs-on: ubuntu-latest
name: Deploy
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
21 changes: 21 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ on:

jobs:
build:
permissions: write-all
runs-on: ubuntu-latest
steps:
- name: Checkout
Expand Down Expand Up @@ -65,3 +66,23 @@ jobs:
asset_content_type: application/gzip
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: "Upload release windows-amd64"
uses: actions/upload-release-asset@v1
with:
upload_url: ${{ steps.release.outputs.upload_url }}
asset_path: ./bin/redis-shake-windows-amd64.tar.gz
asset_name: redis-shake-windows-amd64.tar.gz
asset_content_type: application/gzip
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: "Upload release windows-arm64"
uses: actions/upload-release-asset@v1
with:
upload_url: ${{ steps.release.outputs.upload_url }}
asset_path: ./bin/redis-shake-windows-arm64.tar.gz
asset_name: redis-shake-windows-arm64.tar.gz
asset_content_type: application/gzip
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
17 changes: 10 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
.idea
data
__pycache__
bin
.DS_Store
# system
.idea/
__pycache__/
.DS_Store/

# compiled output or test output
bin/
dist/
tmp/
data/
*.log
*.rdb
*.aof
Loading