Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

redis allow from anywhere + bigger nodes #4349

Merged
merged 84 commits into from
Jun 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
8eb49d3
redis allow from anywhere + bigger nodes
carlomazzaferro May 26, 2023
e36d847
ci: redis allow from anywhere + bigger nodes
carlomazzaferro May 26, 2023
49cca16
Merge branch 'main' into feat/redis-improvements
rhlsthrm Jun 1, 2023
978b50e
Merge branch 'main' into feat/redis-improvements
rhlsthrm Jun 1, 2023
df095ac
Merge branch 'main' into feat/redis-improvements
rhlsthrm Jun 2, 2023
7ba1f94
Merge branch 'feat/redis-improvements' of https://github.com/connext/…
carlomazzaferro Jun 2, 2023
8a010f5
fix vpc id
carlomazzaferro Jun 2, 2023
1aa632c
chore: improve logging and refactor (#4337)
wanglonghong May 25, 2023
8f67dd2
fix: paginate reading pending txs from the redis cache (#4342)
wanglonghong May 26, 2023
624c3fe
fix: commitlint
rhlsthrm May 26, 2023
3d8e400
fix: commitlint
rhlsthrm May 25, 2023
1ae7c65
Testnet prod (#4258)
rhlsthrm Jun 5, 2023
71707ad
fix: remove gelato temporarily
rhlsthrm May 23, 2023
242107a
fix: reenable gelato
rhlsthrm May 23, 2023
d68fde4
fix: merge
rhlsthrm May 25, 2023
d78e84c
fix: build
rhlsthrm May 25, 2023
617f21a
fix: lint
rhlsthrm May 25, 2023
b5dc49b
fix: commitlint
rhlsthrm May 26, 2023
af06f21
ci: fix ci
rhlsthrm May 26, 2023
099b969
ci: testnet prod sync (#4343)
May 26, 2023
4cef77f
ci: fix tf var name
rhlsthrm May 26, 2023
5deb466
ci: add linea rpc
rhlsthrm May 26, 2023
17061b5
ci: bump sequencer CPU
rhlsthrm May 26, 2023
80c452c
Prod Sync (#4339)
liu-zhipeng May 26, 2023
1983da3
ci: fix commitlint
rhlsthrm May 26, 2023
f135ab7
ci: fix ci
rhlsthrm May 26, 2023
93ebd18
ci: redis allow from anywhere + bigger nodes
carlomazzaferro May 26, 2023
4c58a0c
chore: remove console logs
rhlsthrm May 26, 2023
dbcbb73
ci: fix batch size
rhlsthrm May 26, 2023
06ecf85
feat: refactor sequencer subscriber (#4352)
wanglonghong May 29, 2023
16e84ab
feat: dont scale with load (#4354)
wanglonghong May 29, 2023
833aec4
fix: resolve testnet prod sync conflicts (#4356)
wanglonghong May 29, 2023
52a91d0
ci: commitlint
rhlsthrm May 29, 2023
39f76a5
fix: expire data check (#4357)
wanglonghong May 29, 2023
adf0905
feat: should not handle transfer status here
wanglonghong May 29, 2023
361378d
ci: fix commitlint
preethamr May 30, 2023
c0366e7
fix: commitlint
preethamr May 30, 2023
f610a1b
ci: commitlint
rhlsthrm May 30, 2023
d4f658f
feat: query if processed root message exists in db
liu-zhipeng May 29, 2023
caaea67
fix: increase unprocessed root message limit to 100
liu-zhipeng May 29, 2023
8aec7a8
feat: check root message status on db
liu-zhipeng May 29, 2023
f533bb4
fix: message status query
liu-zhipeng May 30, 2023
d8c0492
test: add more unit tests for poller
liu-zhipeng May 30, 2023
aa47e49
test: add unit tests for database
liu-zhipeng May 30, 2023
ee5f96b
fix: 4362 save transferid in the messages table (#4364)
liu-zhipeng May 30, 2023
6a46e33
chore: update graph packages
rhlsthrm May 31, 2023
ff4cf7f
ci: lint
rhlsthrm May 29, 2023
6668c27
ci: testnet prod sync (#4360)
preethamr May 30, 2023
fa5fc10
ci: commitlint
rhlsthrm May 31, 2023
1551df6
ci: fix db
rhlsthrm May 31, 2023
c5f7a6c
ci: set proverBatchSize per chain (#4375)
wanglonghong May 31, 2023
b71670f
chore: update (#4378)
wanglonghong May 31, 2023
5a6920b
feat: 4326 scale sequencer http server (#4363)
preethamr May 31, 2023
ec7ae27
fix: update default ports
preethamr May 31, 2023
12c3055
fix: update sequencer url in smoke tests
preethamr May 31, 2023
a828629
fix: sequencer exec data not found in the cache (#4383)
preethamr Jun 1, 2023
babb107
fix: remove unecessary prefetch config setup
preethamr Jun 1, 2023
0f316f8
ci: better rpcs
rhlsthrm May 31, 2023
8bb8772
ci: bigger prover
rhlsthrm May 31, 2023
a0dc39c
fix: sequencer exec data not found in the cache (#4383)
preethamr Jun 1, 2023
262bdab
fix: wait until none (#4386)
wanglonghong Jun 1, 2023
df1fa5d
ci: fix tests
rhlsthrm Jun 1, 2023
105b1f3
ci: fix smoke test config
rhlsthrm Jun 1, 2023
39b5d3b
fix: update tf config for sequncer cname
preethamr Jun 1, 2023
9e6f3ab
chore: bump chain abstraction
just-a-node Jun 1, 2023
eab6687
fix: sequencer http queue config update
preethamr Jun 1, 2023
53b2c08
fix: add auto scaling to seq publisher
preethamr Jun 1, 2023
a4cc278
fix: typo and seq health port config change
preethamr Jun 1, 2023
4edba3e
fix vpc id
carlomazzaferro Jun 2, 2023
9cb6c02
feat: prod
carlomazzaferro May 16, 2023
7e6474a
feat: pass full images
carlomazzaferro May 16, 2023
e962900
feat: enable alarms in testnet staging
carlomazzaferro May 17, 2023
89c4d78
feat: testnet prod alarms
carlomazzaferro May 17, 2023
61cdb5b
fix: typos
just-a-node May 31, 2023
e87f4f0
chore: update docker build command for mac
just-a-node Jun 1, 2023
802c40e
chore: update docker settings for mac
just-a-node Jun 1, 2023
f56266d
fix: use the correct server package
just-a-node Jun 1, 2023
3983b98
ci: remove deprecated commands (#4385)
Jun 2, 2023
94ae372
ci: fix job
rhlsthrm Jun 2, 2023
9367c2f
fix: add proper types (#4401)
Jun 5, 2023
6b2002e
chore: update from other branch
rhlsthrm Jun 5, 2023
b46e692
Merge branch 'main' into feat/redis-improvements
rhlsthrm Jun 5, 2023
4e7bc26
Merge branch 'feat/redis-improvements' of https://github.com/connext/…
carlomazzaferro Jun 6, 2023
b5c605b
fix sg name
carlomazzaferro Jun 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ops/modules/redis/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
resource "aws_elasticache_cluster" "redis" {
cluster_id = "redis-cluster-${var.environment}-${var.stage}-${var.family}"
engine = "redis"
node_type = "cache.t2.micro"
node_type = var.node_type
num_cache_nodes = 1
# parameter_group_name = "default.redis6.x"
# engine_version = "6.x"
Expand Down
30 changes: 30 additions & 0 deletions ops/modules/redis/sgs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,33 @@ resource "aws_security_group_rule" "allow-res-to-redis" {
to_port = 6379
type = "ingress"
}



resource "aws_security_group" "allow_from_anywhere" {
count = var.public_redis ? 1 : 0
description = "Allow all inbound traffic"
name = "redis-cluster-${var.environment}-${var.family}-allow-all"
vpc_id = var.vpc_id

ingress {
from_port = 6379
to_port = 6379
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}



resource "aws_security_group_rule" "allow-all-to-redis" {
count = var.public_redis ? 1 : 0
description = "Allow worker nodes to communicate with cache"
from_port = 6379
protocol = "tcp"
security_group_id = aws_security_group.redis.id
source_security_group_id = aws_security_group.allow_from_anywhere[0].id
to_port = 6379
type = "ingress"
}

14 changes: 13 additions & 1 deletion ops/modules/redis/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,17 @@ variable "family" {

variable "cache_subnet_group_subnet_ids" {
description = "subnet group ids"
type = list(string)
type = list(string)
}

variable "node_type" {
description = "node type of redis cluster"
default = "cache.t2.small"
type = string
}

variable "public_redis" {
description = "whether to make redis public"
default = false
type = bool
}
6 changes: 6 additions & 0 deletions ops/testnet/prod/core/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,8 @@ module "sequencer_cache" {
sg_id = module.network.ecs_task_sg
vpc_id = module.network.vpc_id
cache_subnet_group_subnet_ids = module.network.public_subnets
node_type = "cache.t2.medium"
public_redis = true
}

module "router_cache" {
Expand All @@ -456,6 +458,8 @@ module "router_cache" {
sg_id = module.network.ecs_task_sg
vpc_id = module.network.vpc_id
cache_subnet_group_subnet_ids = module.network.public_subnets
node_type = "cache.t2.medium"
public_redis = true
}

module "relayer_cache" {
Expand All @@ -466,4 +470,6 @@ module "relayer_cache" {
sg_id = module.network.ecs_task_sg
vpc_id = module.network.vpc_id
cache_subnet_group_subnet_ids = module.network.public_subnets
node_type = "cache.t2.medium"
public_redis = true
}
5 changes: 4 additions & 1 deletion ops/testnet/staging/core/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -499,6 +499,7 @@ module "sequencer_cache" {
sg_id = module.network.ecs_task_sg
vpc_id = module.network.vpc_id
cache_subnet_group_subnet_ids = module.network.public_subnets
public_redis = true
}

module "router_cache" {
Expand All @@ -508,7 +509,8 @@ module "router_cache" {
family = "router"
sg_id = module.network.ecs_task_sg
vpc_id = module.network.vpc_id
cache_subnet_group_subnet_ids = module.network.public_subnets
cache_subnet_group_subnet_ids = module.network.public_sub
public_redis = true
}

module "relayer_cache" {
Expand All @@ -519,4 +521,5 @@ module "relayer_cache" {
sg_id = module.network.ecs_task_sg
vpc_id = module.network.vpc_id
cache_subnet_group_subnet_ids = module.network.public_subnets
public_redis = true
}