Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EVM Rollup + DAL Nodes support #579

Merged
merged 70 commits into from
Dec 10, 2023
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
fced193
introduce smart rollups statefulsets
nicolasochem Apr 15, 2023
bbf5487
ensure it starts
nicolasochem Apr 15, 2023
a538c02
fix rpc endpoint
nicolasochem Apr 15, 2023
ac8e372
add bootstrap param injection
nicolasochem May 3, 2023
b936ecb
add an empty boot sector for now
nicolasochem May 4, 2023
267c75d
put bootstrap rollup params that work in values.yaml example
nicolasochem May 17, 2023
6b2e5b0
fix example
nicolasochem May 28, 2023
a73a341
remove leftover signer code
nicolasochem May 28, 2023
41ebc13
add rollup ingress
nicolasochem May 28, 2023
f018884
set path type properly
nicolasochem May 29, 2023
f17a9d2
make rollup node listen on 0.0.0.0
nicolasochem May 31, 2023
7950c49
use debug container, pass boot sector of evm rollup
nicolasochem May 31, 2023
d0f63b1
use debug image in chain initiator to inject wasm
nicolasochem May 31, 2023
8c33ac7
make debug image work
nicolasochem May 31, 2023
63bca97
put hex of kernel in activation
nicolasochem May 31, 2023
cd21109
add debug statements
nicolasochem May 31, 2023
ae62943
fix typo
nicolasochem May 31, 2023
a0974ee
fix enumerate
nicolasochem May 31, 2023
5f98486
revert config gen changes as the rollup is not in utils container
nicolasochem May 31, 2023
6893519
add a function to replace file with its content in hex
nicolasochem Jun 1, 2023
be07479
remove sleep
nicolasochem Jun 1, 2023
9a769db
add evm proxy
nicolasochem Jun 1, 2023
ecc6c27
add evm proxy script
nicolasochem Jun 1, 2023
6c1a232
add "run" to evm-proxy cmd
nicolasochem Jun 1, 2023
c14f56d
add evm proxy service
nicolasochem Jun 1, 2023
eecc09b
add ingress for evm proxy
nicolasochem Jun 1, 2023
1fe5e8a
add comment to chain-initiator
nicolasochem Jun 7, 2023
23d3d25
add persistent data dir for rollup node
nicolasochem Jun 7, 2023
b48dd97
support several evm rollups
nicolasochem Jun 8, 2023
2c84253
fix path type for nginx
nicolasochem Jun 9, 2023
cfbaa30
DAL initial
nicolasochem Jun 9, 2023
d5f8ebd
add service ports
nicolasochem Jun 10, 2023
8a2d0ab
new syntax for evm proxy
nicolasochem Jun 12, 2023
aba1a5a
Merge remote-tracking branch 'origin/evm_rollup' into evm_rollup
nicolasochem Jun 12, 2023
28f8187
fix ports, remove wrong DAL vars
nicolasochem Jun 12, 2023
3d2e62e
DAL permissions fix
nicolasochem Jun 12, 2023
aa5e27d
DAL: use unsafe SRS in config-init
nicolasochem Jun 12, 2023
ffda22e
dal node new syntax
nicolasochem Jun 15, 2023
edb23b4
Merge remote-tracking branch 'origin/master' into evm_rollup
nicolasochem Jun 20, 2023
fa9299c
add ability for the baker to pick a DAL node
nicolasochem Jun 20, 2023
811d6c6
remove debug container workaround - evm rollup is now in main container
nicolasochem Jun 20, 2023
892d955
new EVM rollup model: installer kernel + small files
nicolasochem Jun 22, 2023
9839dc7
remove usafe srs param
nicolasochem Jun 22, 2023
a5a20c5
fix xxd command
nicolasochem Jun 23, 2023
7308c14
fix xxd in activation script as well
nicolasochem Jun 26, 2023
3b6d07b
set -e: fail activation when a command fails
nicolasochem Jun 26, 2023
bf877fb
support for bootstrap profile in DAL
nicolasochem Jul 28, 2023
cecb905
fix newlines in dal script
nicolasochem Jul 28, 2023
3e749f3
switch to --mode dev on evm proxy
nicolasochem Sep 14, 2023
945e698
switch to universal smart rollup node (instead of proto-dependent)
nicolasochem Sep 18, 2023
5db2e15
ensure you can add annotations to your evm proxy and evm node
nicolasochem Sep 28, 2023
a79d589
proper match for annotation
nicolasochem Sep 29, 2023
abcef9f
add option to pass public IP to dal nodes
nicolasochem Oct 10, 2023
4c8766e
add ability to pass attester profiles to DAL node
nicolasochem Oct 10, 2023
bd10f33
dal: add ability to specify peer
nicolasochem Oct 10, 2023
ea864f0
new evm proxy CLI
nicolasochem Oct 31, 2023
191040e
actually mondaynet doesn't have evm rollup
nicolasochem Oct 31, 2023
197c8c9
remove --mode dev for evm proxy
nicolasochem Nov 1, 2023
f69fc07
put evm proxy in dev mode again
nicolasochem Nov 2, 2023
80af9d2
Merge branch 'master' into evm_rollup
nicolasochem Nov 15, 2023
4f7f5e6
replace `--version dev` with `--devmode`
nicolasochem Nov 15, 2023
59a1fc9
fix disappearing ingress issue
nicolasochem Nov 23, 2023
20b7564
DAL update to 50Gi
nicolasochem Dec 3, 2023
e1a6dba
Merge branch 'master' into evm_rollup
nicolasochem Dec 8, 2023
2f5ac91
Merge branch 'master' into evm_rollup
nicolasochem Dec 8, 2023
230e93d
DAL comments
nicolasochem Dec 8, 2023
8377099
optional hardcoded identity for DAL nodes
nicolasochem Dec 9, 2023
4ea8e52
add ability to set storage size of dal nodes in values.yaml
nicolasochem Dec 9, 2023
5d18e9d
remove tmp changes
nicolasochem Dec 9, 2023
b414882
fix tests
nicolasochem Dec 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 77 additions & 2 deletions charts/tezos/scripts/chain-initiator.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,92 @@ until $CLIENT rpc get /chains/main/blocks/head/header | grep '"level":'; do
sleep 2
done

set -x
set -o pipefail
if ! $CLIENT rpc get /chains/main/blocks/head/header | grep '"level": 0,'; then
echo "Chain already activated, considering activation successful and exiting"
exit 0
fi

# Substitute #fromfile with the hex encoded files in question.
# This is for bootstrapped smart rollups.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this all for?


# Note that this is low-level string substitution with `read`
# Due to the size of the hex-encoded kernel, using `sed` was not possible.

PARAMETERS_FILE='/etc/tezos/parameters.json'
TMP_PARAMETERS_FILE='/etc/tezos/tmp_parameters.json'

# Pattern to search for
pattern='fromfile#'

# Buffer for characters
buffer=''

# Whether 'fromfile#' was detected
detected_fromfile=false

# Process each character
while IFS= read -r -n1 char
do
# Add the character to the buffer
buffer=$(printf "%s%s" "$buffer" "$char")

# If the buffer ends with the pattern
if [ "${buffer%"$pattern"}" != "$buffer" ]
then
detected_fromfile=true

# Clear the buffer
buffer=''

# Read the filename
filename=''
while IFS= read -r -n1 char && [ "$char" != '"' ]
do
filename=$(printf "%s%s" "$filename" "$char")
done

echo "Found kernel file: $filename"

# Check if file exists
if [ ! -f "$filename" ]; then
echo "Kernel file $filename not found!"
exit 1
fi

# Convert the file content to hex and append to the temp file
xxd -ps -c 256 "$filename" | tr -d '\n' >> $TMP_PARAMETERS_FILE

# Add a closing double quote
printf '"' >> $TMP_PARAMETERS_FILE
elif [ ${#buffer} -ge ${#pattern} ]
then
# Write the oldest character in the buffer to the temporary file
printf "%s" "${buffer%"${buffer#?}"}" >> $TMP_PARAMETERS_FILE

# Remove the oldest character from the buffer
buffer=${buffer#?}
fi
done < "$PARAMETERS_FILE"

# If there's anything left in the buffer, write it to the file
if [ -n "$buffer" ]
then
printf "%s" "$buffer" >> $TMP_PARAMETERS_FILE
fi

# Replace the original parameters.json file with the modified one only if 'fromfile#' was detected
if $detected_fromfile; then
mv $TMP_PARAMETERS_FILE $PARAMETERS_FILE
echo "Updated JSON saved in '$PARAMETERS_FILE'"
else
rm -f $TMP_PARAMETERS_FILE
echo "No 'fromfile#' detected in '$PARAMETERS_FILE', no changes made."
fi
echo Activating chain:
$CLIENT -d /var/tezos/client --block \
genesis activate protocol \
{{ .Values.activation.protocol_hash }} \
with fitness 1 and key \
$( cat /etc/tezos/activation_account_name ) \
and parameters /etc/tezos/parameters.json 2>&1 | head -200
and parameters $PARAMETERS_FILE 2>&1 | head -200
9 changes: 9 additions & 0 deletions charts/tezos/scripts/evm-proxy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
set -ex

TEZ_BIN=/usr/local/bin

CMD="$TEZ_BIN/octez-evm-proxy-server run \
--rpc-addr 0.0.0.0 \
--rollup-node-endpoint http://rollup-evm:8932"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the url need to be a variable bec there can be multiple sts's with their own services?


exec $CMD
17 changes: 17 additions & 0 deletions charts/tezos/scripts/smart-rollup-node.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
set -ex

TEZ_VAR=/var/tezos
TEZ_BIN=/usr/local/bin
CLIENT_DIR="$TEZ_VAR/client"
ROLLUP_DATA_DIR="$TEZ_VAR/rollup"

xxd -ps -c 0 /usr/local/share/tezos/evm_kernel.wasm | tr -d '\n' > /var/tezos/smart-rollup-boot-sector
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment here whats going on?

CMD="$TEZ_BIN/octez-smart-rollup-node-alpha \
--endpoint http://tezos-node-rpc:8732 \
-d $CLIENT_DIR \
run operator for ${ROLLUP_ADDRESS} with operators ${OPERATOR_ACCOUNT} \
--data-dir ${ROLLUP_DATA_DIR} \
--boot-sector-file /var/tezos/smart-rollup-boot-sector \
--rpc-addr 0.0.0.0"

exec $CMD
6 changes: 4 additions & 2 deletions charts/tezos/templates/_containers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
{{- $_ := set . "localvars" (eq .image "utils") }}
{{- end }}
{{- if not (hasKey . "run_script") }}
{{- $_ := set . "run_script" (eq .image "octez") }}
{{- $_ := set . "run_script" (or (eq .image "octez") (eq .image "octez_debug") ) }}
{{- end }}
{{- if not (hasKey . "script_command") }}
{{- $_ := set . "script_command" .type }}
Expand All @@ -88,6 +88,8 @@
{{- $node_vals_images := $.node_vals.images | default dict }}
{{- if eq .image "octez" }}
image: "{{ or $node_vals_images.octez $.Values.images.octez }}"
{{- else if eq .image "octez_debug" }}
image: "{{ or $node_vals_images.octez_debug $.Values.images.octez_debug }}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smartrollup spec in value.y should show ability to specify image on the sts level.

{{- else }}
image: "{{ $.Values.tezos_k8s_images.utils }}"
{{- end }}
Expand Down Expand Up @@ -211,7 +213,7 @@
{{- define "tezos.init_container.chain_initiator" }}
{{- include "tezos.generic_container" (dict "root" $
"type" "chain-initiator"
"image" "octez"
"image" "octez_debug"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know much about this new image. Just curious why we are using it here

) | nindent 0 }}
{{- end }}

Expand Down
2 changes: 1 addition & 1 deletion charts/tezos/templates/activate-job.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ spec:
{{- include "tezos.init_container.chain_initiator" $ | indent 8 }}
initContainers:
- image: {{ .Values.tezos_k8s_images.utils }}
imagePullPolicy: IfNotPresent
imagePullPolicy: Always
name: config-generator
args:
- config-generator
Expand Down
2 changes: 2 additions & 0 deletions charts/tezos/templates/configs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ data:
{{- $_ := set $tacoinfraSigners $signerName (pick $signerConfig "accounts") }}
{{- end }}
{{ $tacoinfraSigners | default dict | mustToPrettyJson | indent 4 }}
OCTEZ_ROLLUP_NODES: |
{{ $.Values.smartRollupNodes | default dict | mustToPrettyJson | indent 4 }}

---

Expand Down
218 changes: 218 additions & 0 deletions charts/tezos/templates/octez-rollup-node.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
{{- range $k, $v := .Values.smartRollupNodes }}

apiVersion: v1
kind: Service
metadata:
name: evm-proxy-{{ $k }}
namespace: {{ $.Release.Namespace }}
spec:
type: NodePort
ports:
- port: 8545
name: evm-proxy
selector:
app: evm-proxy-{{ $k }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: evm-proxy-{{ $k }}
namespace: {{ $.Release.Namespace }}
spec:
podManagementPolicy: Parallel
replicas: 1
serviceName: evm-proxy-{{ $k }}
selector:
matchLabels:
app: evm-proxy-{{ $k }}
template:
metadata:
labels:
app: evm-proxy-{{ $k }}
spec:
containers:
- name: octez-evm-proxy
image: "{{ $.Values.images.octez }}"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8545
name: evm-proxy
command:
- /bin/sh
args:
- "-c"
- |
{{ tpl ($.Files.Get "scripts/evm-proxy.sh") $ | indent 12 }}
securityContext:
fsGroup: 1000
---
{{- if $v.evm_proxy_ingress | default false }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is missing in value.y spec

{{- if $v.evm_proxy_ingress.enabled | default false }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: evm-proxy-{{ $k }}
namespace: {{ $.Release.Namespace }}
{{- with $v.evm_proxy_ingress.labels }}
labels:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with $v.evm_proxy_ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ $v.evm_proxy_ingress.className }}
{{- if $v.evm_proxy_ingress.tls }}
tls:
{{- range $v.evm_proxy_ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
- host: {{ $v.evm_proxy_ingress.host }}
http:
paths:
- pathType: ImplementationSpecific
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should use the pathType prop from values.yaml and default to Prefix. I think our ingresses everywhere default to Prefix

path: /*
backend:
service:
name: evm-proxy-{{ $k }}
port:
name: evm-proxy
{{- end }}
{{- end }}
---
apiVersion: v1
kind: Service
metadata:
name: rollup-{{ $k }}
namespace: {{ $.Release.Namespace }}
spec:
type: NodePort
ports:
- port: 8932
name: rollup
selector:
app: rollup-{{ $k }}
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: rollup-{{ $k }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The values.y example rollup is called rollup-node-0. This will turn into rollup-rollup-node-0. Why not let the name be exactly what the value is in values.y?

namespace: {{ $.Release.Namespace }}
spec:
podManagementPolicy: Parallel
replicas: 1
serviceName: rollup-{{ $k }}
selector:
matchLabels:
app: rollup-{{ $k }}
template:
metadata:
labels:
app: rollup-{{ $k }}
spec:
containers:
- name: octez-smart-rollup-node
image: "{{ $.Values.images.octez_debug }}"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8932
name: rollup
command:
- /bin/sh
volumeMounts:
- mountPath: /var/tezos
name: var-volume
args:
- "-c"
- |
{{ tpl ($.Files.Get "scripts/smart-rollup-node.sh") $ | indent 12 }}
env:
- name: ROLLUP_ADDRESS
value: {{ $v.rollup_address }}
- name: OPERATOR_ACCOUNT
value: {{ $v.operator_account }}
initContainers:
- image: {{ $.Values.tezos_k8s_images.utils }}
imagePullPolicy: IfNotPresent
name: config-generator
args:
- "config-generator"
envFrom:
- configMapRef:
name: tezos-config
env:
- name: MY_POD_NAME
value: {{ $k }}
- name: MY_POD_TYPE
value: {{ $.Values.smart_rollup_node_statefulset.pod_type }}
volumeMounts:
- mountPath: /var/tezos
name: var-volume
- mountPath: /etc/secret-volume
name: tezos-accounts
securityContext:
fsGroup: 1000
volumes:
- volume: var-volume
name: var-volume
- name: tezos-accounts
secret:
secretName: tezos-secret
volumeClaimTemplates:
- metadata:
name: var-volume
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: "15Gi"
---
{{- if $v.ingress | default false }}
{{- if $v.ingress.enabled | default false }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rollup-{{ $k }}
namespace: {{ $.Release.Namespace }}
{{- with $v.ingress.labels }}
labels:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with $v.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ $v.ingress.className }}
{{- if $v.ingress.tls }}
tls:
{{- range $v.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
- host: {{ $v.ingress.host }}
http:
paths:
- pathType: ImplementationSpecific
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use Prefix field from values.y

path: /*
backend:
service:
name: rollup-{{ $k }}
port:
name: rollup
{{- end }}
{{- end }}
{{- end }}
Loading