Support to create ubuntu OS #6

Jianlin-lv · 2022-03-26T14:48:11Z

Running following command to create ubuntu ostree:
'./deb-ostree-builder -a amd64 -p ubuntu focal --no-checkout -f'

Signed-off-by: Jianlin Lv lvlin@mail.ustc.edu.cn

Running following command to create ubuntu ostree: './deb-ostree-builder -a amd64 -p ubuntu focal --no-checkout -f' Signed-off-by: Jianlin Lv <lvlin@mail.ustc.edu.cn>

KevyVo · 2022-09-29T22:30:22Z

Hey Jianliv,

First of all, I think the work you have done here to make the image builder create an image with Ubuntu and ostree is exactly what I need. I tried to run your repository that supports Ubuntu with the command /deb-ostree-builder -a amd64 -p ubuntu focal --no-checkout -f. I have run into multiple issues while trying to create this image. This is the latest issue I ran into and I am not sure how to fix this. I appreciate any help you are willing to provide.

OUTPUT:
`I: Base system installed successfully.

21:48:32 eob_ostree: trap cleanup EXIT
21:48:32 eob_ostree: EOB_OSTREE_CHROOT=/var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/ostree/ubuntu
21:48:32 eob_ostree: EOB_SYSCONF_CHROOT=/var/cache/deb-ostree-builder/tmp/ostree-bin/etc/deb-ostree-builder
21:48:32 eob_ostree: EOB_ROOTDIR_CHROOT=/var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/chroot
21:48:32 eob_ostree: mkdir -p /var/cache/deb-ostree-builder/ostree/ubuntu /var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/ostree/ubuntu /var/cache/deb-ostree-builder/tmp/ostree-bin/etc/deb-ostree-builder /var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/chroot
21:48:32 eob_ostree: eob_mount --bind /var/cache/deb-ostree-builder/ostree/ubuntu /var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/ostree/ubuntu
21:48:32 eob.sh: local target
21:48:32 eob.sh: '[' 3 -lt 2 ']'
21:48:32 eob.sh: mount --bind /var/cache/deb-ostree-builder/ostree/ubuntu /var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/ostree/ubuntu
21:48:32 eob.sh: eval 'target=${3}'
++ 21:48:32 eob.sh: target=/var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/ostree/ubuntu
21:48:32 eob.sh: EOB_MOUNTS+=("${target}")
21:48:32 eob_ostree: eob_mount --bind /etc/deb-ostree-builder /var/cache/deb-ostree-builder/tmp/ostree-bin/etc/deb-ostree-builder
21:48:32 eob.sh: local target
21:48:32 eob.sh: '[' 3 -lt 2 ']'
21:48:32 eob.sh: mount --bind /etc/deb-ostree-builder /var/cache/deb-ostree-builder/tmp/ostree-bin/etc/deb-ostree-builder
mount: /var/cache/deb-ostree-builder/tmp/ostree-bin/etc/deb-ostree-builder: special device /etc/deb-ostree-builder does not exist.
21:48:32 eob.sh: cleanup
21:48:32 eob_ostree: cleanup_ostree_config
21:48:32 eob_ostree: tmp_ostree --repo=/var/cache/deb-ostree-builder/ostree/ubuntu remote delete --if-exists ubuntu
21:48:32 eob.sh: chroot /var/cache/deb-ostree-builder/tmp/ostree-bin ostree --repo=/var/cache/deb-ostree-builder/ostree/ubuntu remote delete --if-exists ubuntu
error: Failed to open file ?/proc/sys/kernel/random/boot_id?: No such file or directory
21:48:32 eob_ostree: :
21:48:32 eob_ostree: cleanup_tmp_ostree
21:48:32 eob.sh: pkill -INT -f 'gpg-agent.*/etc/deb-ostree-builder/gnupg'
21:48:32 eob.sh: :
21:48:32 eob_ostree: eob_umount_all
21:48:32 eob.sh: local -i n
21:48:32 eob.sh: (( n = 1 - 1 ))
21:48:32 eob.sh: (( n >= 0 ))
21:48:32 eob.sh: umount /var/cache/deb-ostree-builder/tmp/ostree-bin/var/cache/deb-ostree-builder/ostree/ubuntu
21:48:32 eob.sh: (( n-- ))
21:48:32 eob.sh: (( n >= 0 ))
21:48:32 eob.sh: unset EOB_MOUNTS
21:48:32 eob.sh: declare -a EOB_MOUNTS
++ 21:48:32 buildscript: stage_error
++ 21:48:32 buildscript: eob_error
. /home/ubuntu/deb-ostree-builder/lib/eob.sh
++ shopt -s nullglob
++ shopt -s extglob
++ export 'PS4=+ \t ${BASH_SOURCE[0]##/}: '
++ PS4='+ \t ${BASH_SOURCE[0]##/}: '
++ 21:48:32 eob.sh: declare -a EOB_MOUNTS
++ 21:48:32 eob.sh: true
21:48:32 eob_error: run_hooks error
21:48:32 eob.sh: local hook interpreter
21:48:32 eob.sh: local group=error
21:48:32 eob.sh: local install_root=
21:48:32 eob.sh: echo 'Running error hooks'
Running error hooks
21:48:32 eob.sh: eval local 'hooks=${EOB_ERROR_HOOKS}'
++ 21:48:32 eob.sh: local hooks=50-clear-cache
++ 21:48:32 eob.sh: echo 50-clear-cache
++ 21:48:32 eob.sh: sort
++ 21:48:32 eob.sh: tr ' ' '\n'
21:48:32 eob.sh: local files=50-clear-cache
21:48:32 eob.sh: echo '50-clear-cache TOMMYY'
50-clear-cache TOMMYY
21:48:32 eob.sh: for hook in ${files}
21:48:32 eob.sh: echo 'running hooker 50-clear-cache TOMMY'
running hooker 50-clear-cache TOMMY
21:48:32 eob.sh: local hookpath=/home/ubuntu/deb-ostree-builder/hooks/error/50-clear-cache
21:48:32 eob.sh: '[' '!' -f /home/ubuntu/deb-ostree-builder/hooks/error/50-clear-cache ']'
21:48:32 eob.sh: '[' r-cache == .chroot ']'
21:48:32 eob.sh: echo 'Run hook: 50-clear-cache'
Run hook: 50-clear-cache
21:48:32 eob.sh: '[' -x /home/ubuntu/deb-ostree-builder/hooks/error/50-clear-cache ']'
21:48:32 eob.sh: . /home/ubuntu/deb-ostree-builder/hooks/error/50-clear-cache
++ 21:48:32 50-clear-cache: echo /var/cache/deb-ostree-builder/cache/ubuntu-focal-amd64-amd64
/var/cache/deb-ostree-builder/cache/ubuntu-focal-amd64-amd64
++ 21:48:32 50-clear-cache: rm -rf /var/cache/deb-ostree-builder/cache/ubuntu-focal-amd64-amd64
++ 21:48:32 buildscript: exit 2
Traceback (most recent call last):
File "./run-build", line 307, in
main()
File "./run-build", line 304, in main
builder.run()
File "./run-build", line 290, in run
subprocess.check_call([os.path.join(self.srcdir, 'buildscript')])
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/ubuntu/deb-ostree-builder/buildscript']' returned non-zero exit status 2.
Traceback (most recent call last):
File "./deb-ostree-builder", line 107, in
subprocess.check_call(['./run-build'] + sys.argv[1:])
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./run-build', '-a', 'amd64', '-p', 'ubuntu', 'focal', '--no-checkout', '-f']' returned non-zero exit status 1.`

Jianlin-lv · 2022-09-30T00:29:38Z

hi, @KevyVo Please create /etc/deb-ostree-builder directory in host firstly,
mkdir -p /etc/deb-ostree-builder

KevyVo · 2022-09-30T02:11:48Z

Hmm, it seems like I am still getting errors after creating the folder (/etc/deb-ostree-builder) you mention. Any ideas?

Run hook: 50-clear-cache
+ 02:04:48 eob.sh: '[' -x /home/ubuntu/deb-ostree-builder/hooks/error/50-clear-cache ']'
+ 02:04:48 eob.sh: . /home/ubuntu/deb-ostree-builder/hooks/error/50-clear-cache
++ 02:04:48 50-clear-cache: echo /var/cache/deb-ostree-builder/cache/ubuntu-focal-amd64-amd64
/var/cache/deb-ostree-builder/cache/ubuntu-focal-amd64-amd64
++ 02:04:48 50-clear-cache: rm -rf /var/cache/deb-ostree-builder/cache/ubuntu-focal-amd64-amd64
++ 02:04:48 buildscript: exit 2
Traceback (most recent call last):
  File "./run-build", line 307, in <module>
    main()
  File "./run-build", line 304, in main
    builder.run()
  File "./run-build", line 290, in run
    subprocess.check_call([os.path.join(self.srcdir, 'buildscript')])
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/ubuntu/deb-ostree-builder/buildscript']' returned non-zero exit status 2.
Traceback (most recent call last):
  File "./deb-ostree-builder", line 107, in <module>
    subprocess.check_call(['./run-build'] + sys.argv[1:])
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./run-build', '-a', 'amd64', '-p', 'ubuntu', 'focal', '--no-checkout', '-f']' returned non-zero exit status 1.

Jianlin-lv · 2022-09-30T02:26:17Z

Could you paste more log info?

KevyVo · 2022-09-30T12:09:55Z

This is the entire log message, I have put it in the txt file. It to long to paste here. If you see echo "TOMMY", it was my own echo to see more information so ignore that. Please let me know if you can help.
LogOutput.txt

Jianlin-lv · 2022-10-01T01:22:09Z

I have not encountered such issue, May be the lack of Python on host.

+ 02:04:48 eob_ostree: /home/ubuntu/deb-ostree-builder/helpers/split_passwd_files /var/cache/deb-ostree-builder/chroot
/usr/bin/env: 'python': No such file or directory

You can install the following tools before building ostree (some tools are redundant)

 apt-get -y install autoconf autotools-dev automake pkg-config autoconf-archive build-essential \
     libtool bison liblzma5 liblzma-dev e2fsprogs e2fslibs e2fslibs-dev libgpgme-dev libfuse-dev \
     gtk+-3.0 qemu-utils debootstrap  python3-glanceclient alien unzip golang python3-swiftclient buildah uuid

KevyVo · 2022-10-02T10:02:58Z

Hey @Jianlin-lv,

It looks like the build finish. Do you know where I can see the output of the ubuntu ostree?

+ . /home/ubuntu/deb-ostree-builder/lib/eob.sh
++ shopt -s nullglob
++ shopt -s extglob
++ export 'PS4=+ \t ${BASH_SOURCE[0]##*/}: '
++ PS4='+ \t ${BASH_SOURCE[0]##*/}: '
++ 09:54:07 eob.sh: declare -a EOB_MOUNTS
++ 09:54:07 eob.sh: true
+ 09:54:07 eob_publish: '[' false = true ']'
+ 09:54:07 eob_publish: run_hooks publish
+ 09:54:07 eob.sh: local hook interpreter
+ 09:54:07 eob.sh: local group=publish
+ 09:54:07 eob.sh: local install_root=
+ 09:54:07 eob.sh: echo 'Running publish hooks'
Running publish hooks
+ 09:54:07 eob.sh: eval local 'hooks=${EOB_PUBLISH_HOOKS}'
++ 09:54:07 eob.sh: local 'hooks=50-publish
55-update-summary
90-email-success'
++ 09:54:07 eob.sh: echo '50-publish
55-update-summary
90-email-success'
++ 09:54:07 eob.sh: sort
++ 09:54:07 eob.sh: tr ' ' '\n'
+ 09:54:07 eob.sh: local 'files=50-publish
55-update-summary
90-email-success'
+ 09:54:07 eob.sh: echo '50-publish
55-update-summary
90-email-success TOMMYY'
50-publish
55-update-summary
90-email-success TOMMYY
+ 09:54:07 eob.sh: for hook in ${files}
+ 09:54:07 eob.sh: echo 'running hooker 50-publish TOMMY'
running hooker 50-publish TOMMY
+ 09:54:07 eob.sh: local hookpath=/home/ubuntu/deb-ostree-builder/hooks/publish/50-publish
+ 09:54:07 eob.sh: '[' '!' -f /home/ubuntu/deb-ostree-builder/hooks/publish/50-publish ']'
+ 09:54:07 eob.sh: '[' publish == .chroot ']'
+ 09:54:07 eob.sh: echo 'Run hook: 50-publish'
Run hook: 50-publish
+ 09:54:07 eob.sh: '[' -x /home/ubuntu/deb-ostree-builder/hooks/publish/50-publish ']'
+ 09:54:07 eob.sh: . /home/ubuntu/deb-ostree-builder/hooks/publish/50-publish
++ 09:54:07 50-publish: '[' -n '' ']'
++ 09:54:07 50-publish: exit 0
+ 09:54:07 eob.sh: for hook in ${files}
+ 09:54:07 eob.sh: echo 'running hooker 55-update-summary TOMMY'
running hooker 55-update-summary TOMMY
+ 09:54:07 eob.sh: local hookpath=/home/ubuntu/deb-ostree-builder/hooks/publish/55-update-summary
+ 09:54:07 eob.sh: '[' '!' -f /home/ubuntu/deb-ostree-builder/hooks/publish/55-update-summary ']'
+ 09:54:07 eob.sh: '[' summary == .chroot ']'
+ 09:54:07 eob.sh: echo 'Run hook: 55-update-summary'
Run hook: 55-update-summary
+ 09:54:07 eob.sh: '[' -x /home/ubuntu/deb-ostree-builder/hooks/publish/55-update-summary ']'
+ 09:54:07 eob.sh: . /home/ubuntu/deb-ostree-builder/hooks/publish/55-update-summary
++ 09:54:07 55-update-summary: '[' -n '' ']'
++ 09:54:07 55-update-summary: exit 0
+ 09:54:07 eob.sh: for hook in ${files}
+ 09:54:07 eob.sh: echo 'running hooker 90-email-success TOMMY'
running hooker 90-email-success TOMMY
+ 09:54:07 eob.sh: local hookpath=/home/ubuntu/deb-ostree-builder/hooks/publish/90-email-success
+ 09:54:07 eob.sh: '[' '!' -f /home/ubuntu/deb-ostree-builder/hooks/publish/90-email-success ']'
+ 09:54:07 eob.sh: '[' success == .chroot ']'
+ 09:54:07 eob.sh: echo 'Run hook: 90-email-success'
Run hook: 90-email-success
+ 09:54:07 eob.sh: '[' -x /home/ubuntu/deb-ostree-builder/hooks/publish/90-email-success ']'
+ 09:54:07 eob.sh: . /home/ubuntu/deb-ostree-builder/hooks/publish/90-email-success
++ 09:54:07 90-email-success: '[' -n '' ']'
++ 09:54:07 90-email-success: exit 0```

dbnicholson · 2022-10-02T12:23:30Z

Since you're not publishing it anywhere, I think it's in the temporary repo in /var/cache/deb-ostree-builder/ostree/repo.

Jianlin-lv · 2022-10-02T13:51:46Z

@KevyVo Nicholson is right, the OSTree repo location is /var/cache/deb-ostree-builder/ostree/ubuntu/

KevyVo · 2022-10-03T23:37:52Z

Hey @dbnicholson @Jianlin-lv ,

Thank you for all your help so far, I've made a lot of progress. I've now been able to use the deb-ostree-builder to create a commit of ubuntu OS managed by ostree.

For my next steps, I am trying to use the create-deployment script to write this commit to some disk image, however, I'm running into a block:

ubuntu@ip-172-31-1-189:~/deb-ostree-builder$ ./create-deployment 
+ EIB_OSTREE_REPODIR=/var/cache/deb-ostree-builder/ostree/ubuntu
++ mktemp -d -p /var/tmp ostree-deploy.XXXXXXXXXX
+ EIB_OSTREE_SYSROOT=/var/tmp/ostree-deploy.3dmTLoq6fC
+ EIB_OSTREE_OS=ubuntu
+ EIB_OSTREE_URL=http://www.example.com/ostree
+ EIB_OSTREE_BRANCH_DEPLOY=unstable
+ REPOPATH=/var/tmp/ostree-deploy.3dmTLoq6fC/ostree/repo
+ BOOT=/var/tmp/ostree-deploy.3dmTLoq6fC/boot
+ ostree admin init-fs /var/tmp/ostree-deploy.3dmTLoq6fC
+ ostree admin --sysroot=/var/tmp/ostree-deploy.3dmTLoq6fC os-init ubuntu
ostree/deploy/ubuntu initialized as OSTree root
+ ostree --repo=/var/tmp/ostree-deploy.3dmTLoq6fC/ostree/repo remote add ubuntu http://www.example.com/ostree unstable
+ ostree --repo=/var/tmp/ostree-deploy.3dmTLoq6fC/ostree/repo pull-local --disable-fsync --remote=ubuntu /var/cache/deb-ostree-builder/ostree/ubuntu unstable

error: Refspec 'unstable' not found

Questions:

Could someone clarify what the script is trying to do with the remote ref?

Is it possible to build this image without sending it to a remote ostree repo?

How do you host a remote ostree repo? There is not much documentation on this, therefore I am quite lost on how to do this.
I have come across this repo called ostree-upload-server, I am not sure if this would be helpful to my goal.

Any clarity on how we can leverage this script to write the ostree managed ubuntu OS to a disk image, would be appreciated.

KevyVo · 2022-10-03T23:41:27Z

I have not encountered such issue, May be the lack of Python on host.

+ 02:04:48 eob_ostree: /home/ubuntu/deb-ostree-builder/helpers/split_passwd_files /var/cache/deb-ostree-builder/chroot
/usr/bin/env: 'python': No such file or directory

You can install the following tools before building ostree (some tools are redundant)

 apt-get -y install autoconf autotools-dev automake pkg-config autoconf-archive build-essential \
     libtool bison liblzma5 liblzma-dev e2fsprogs e2fslibs e2fslibs-dev libgpgme-dev libfuse-dev \
     gtk+-3.0 qemu-utils debootstrap  python3-glanceclient alien unzip golang python3-swiftclient buildah uuid

I was able to fix this problem by linking python3 to python since it was trying to reference it.
Python3 was installed but the alias was not linked.
https://askubuntu.com/questions/1144446/python-installed-in-ubuntu-but-python-command-not-found

sudo ln -s /usr/bin/python3 /usr/bin/python

Support to create ubuntu OS

18d3f7b

Running following command to create ubuntu ostree: './deb-ostree-builder -a amd64 -p ubuntu focal --no-checkout -f' Signed-off-by: Jianlin Lv <lvlin@mail.ustc.edu.cn>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support to create ubuntu OS #6

Support to create ubuntu OS #6

Jianlin-lv commented Mar 26, 2022

KevyVo commented Sep 29, 2022

Jianlin-lv commented Sep 30, 2022

KevyVo commented Sep 30, 2022

Jianlin-lv commented Sep 30, 2022 •

edited

Loading

KevyVo commented Sep 30, 2022 •

edited

Loading

Jianlin-lv commented Oct 1, 2022 •

edited

Loading

KevyVo commented Oct 2, 2022

dbnicholson commented Oct 2, 2022

Jianlin-lv commented Oct 2, 2022

KevyVo commented Oct 3, 2022 •

edited

Loading

KevyVo commented Oct 3, 2022 •

edited

Loading

Support to create ubuntu OS #6

Are you sure you want to change the base?

Support to create ubuntu OS #6

Conversation

Jianlin-lv commented Mar 26, 2022

KevyVo commented Sep 29, 2022

Jianlin-lv commented Sep 30, 2022

KevyVo commented Sep 30, 2022

Jianlin-lv commented Sep 30, 2022 • edited Loading

KevyVo commented Sep 30, 2022 • edited Loading

Jianlin-lv commented Oct 1, 2022 • edited Loading

KevyVo commented Oct 2, 2022

dbnicholson commented Oct 2, 2022

Jianlin-lv commented Oct 2, 2022

KevyVo commented Oct 3, 2022 • edited Loading

Questions:

KevyVo commented Oct 3, 2022 • edited Loading

Jianlin-lv commented Sep 30, 2022 •

edited

Loading

KevyVo commented Sep 30, 2022 •

edited

Loading

Jianlin-lv commented Oct 1, 2022 •

edited

Loading

KevyVo commented Oct 3, 2022 •

edited

Loading

KevyVo commented Oct 3, 2022 •

edited

Loading