refkit layers #147
refkit layers #147
Conversation
|
CI run (job #954) keeps doing in a loop: seems like a case of endless sleep-loop we have had before in early selftest-added times |
|
@mythi, @rpurdie: one question I have is around best practices for deciding between using vs. not using .bbappends. In some cases, a .bbappend has to be used (defining functions or adding files). But when merely changing variables like PACKAGECONFIG, the change can also be in a .inc file with a pn override. We've not been consistent in refkit regarding that: Any preference from you on this? I'm undecided myself. |
|
I think for me it depends what the reason for the change is. We group some things together as .inc files at the distro level if they make sense as a logically grouped set of changes using pn- overrides. I can see cases where something might not fit those includes/groups in which case a bbapend may be more appropriate. |
|
@pohly I also think that includes are preferred in cases where logical grouping (PACKAGECONFIG, SECURITY_FLAGS, etc.) is possible. Seems like we all agree the principles. I "learnt" to use the PACKAGECONFIG pn-override late so there might be some inconsistencies. FWIW the pulseaudio .bbappend should be moved to a patch to add a PACKAGECONFIG entry in the .bb and then we can do the right config in refkit-config.inc. |
|
@mythi I think we agree on the basics, but let's be specific. So we do PACKAGECONFIG changes in the .inc file, security flags in refkit_security_flags.inc, and everything else in .bbappends? |
|
I pulled pohly/refkit-layers branch and added meta-refkit-core to my bblayers. Builds fine. +1 |
|
The relationship of image-dsk and refkit-image bbclasses is incestuous. image-dsk requires REFKIT_VFAT_MB to be set otherwise you get a json parse error if you don't use refkit-image.bbclass. |
|
On Wed, 2017-05-17 at 10:20 -0700, Kevron Rees wrote:
The dependency of image-dsk and refkit-image bbclasses is incestuous.
image-dsk requires REFKIT_VFAT_MB to be set otherwise you get a json
parse error if you don't use refkit-image.bbclass.
|
|
On Thu, May 18, 2017 at 9:08 AM, Patrick Ohly <notifications@github.com>
wrote:
On Wed, 2017-05-17 at 10:20 -0700, Kevron Rees wrote:
> The dependency of image-dsk and refkit-image bbclasses is incestuous.
> image-dsk requires REFKIT_VFAT_MB to be set otherwise you get a json
> parse error if you don't use refkit-image.bbclass.
image-dsk was meant to be removed. @mythi, should we keep it as a
stand-alone class and ensure that it activates the wic image format with
UEFI combo app?
I'm afraid we might need a deprecation period for it. Unfortunately it does
not seem to work as a stand-alone class right now because of the
emmc-partitions-data/DSK_IMAGE_LAYOUT dependency. An easy fix
would be to skip emmc-partitions-data generation if DSK_IMAGE_LAYOUT is
empty. The stand-alone users would then set DSK_IMAGE_LAYOUT=""
|
|
On Thu, 2017-05-18 at 05:32 -0700, Mikko Ylinen wrote:
I'm afraid we might need a deprecation period for it.
What do you mean with that? Do you mean we have users outside of refkit
(who?), and those users need advance warning that the class will be
removed?
Right now, it is more like an internal class. We would have already
removed it if we had gotten around to it. But now it looks like it has
value on its own, if we fix it so that it actually works stand-alone.
|
|
On Thu, May 18, 2017 at 3:50 PM, Patrick Ohly <notifications@github.com>
wrote:
On Thu, 2017-05-18 at 05:32 -0700, Mikko Ylinen wrote:
> I'm afraid we might need a deprecation period for it.
What do you mean with that? Do you mean we have users outside of refkit
(who?), and those users need advance warning that the class will be
removed?
ref-os-iot uses the ostro-os-xt installer script that depends on
emmc-partitions-data info. I don't want to drag that in the new
implementation because the installer does not need it anymore. If we get
ref-os-iot to use image-installer then no deprecation is needed.
|
pohly
force-pushed
the
refkit-layers
branch
2 times, most recently
from
4aa288c
to
629ed97
Compare
|
Rebased, fixed, squashed, all my TODOs addressed -> this could be merged now. |
|
Note that we also agreed to standardize on putting PACKAGECONFIG into the .inc file. Cleaning that up isn't done yet and also is kind of orthogonal to the purpose of the PR, but I can add it. |
pohly
force-pushed
the
refkit-layers
branch
2 times, most recently
from
9133938
to
4dfae98
Compare
|
test this please |
|
Rebased once more, fixed refkit_image_muted, added build test of refkit-image-computervision. The latter one is potentially slow - let's see how slow. I also don't know yet whether it really passes (local build still running). |
| @@ -0,0 +1 @@ | |||
| require python-readline.inc | |||
There was a problem hiding this comment.
Well, it's not only readline which is removed... also gdbm and db are not compiled in.
There was a problem hiding this comment.
Should I just rename it to refkit-python.inc?
| @@ -33,33 +122,22 @@ PACKAGECONFIG_pn-python3-pygobject_refkit-config = "" | |||
| # pango, a graphical text rendering library, is not needed by us. | |||
| PACKAGECONFIG_remove_pn-gstreamer1.0-plugins-base_refkit-config = "pango" | |||
|
|
|||
| # remove readline support from Bluez to reduce GPLv3 dependencies | |||
| BAD_RECOMMENDATIONS_append_refkit-config = " bluez5-client" | |||
There was a problem hiding this comment.
This appears not to be recreated anywhere.
|
test this please |
|
test this please KVM was not active in the CI, unrelated to this PR. |
|
test this please |
|
Can one of the admins verify this patch? |
…E_MODE Forcing developers to choose between production and development builds was non-standard (thus tripping up new developers) and inflexible (couldn't build product and development images at the same time). The new image-mode.bbclass replaces that with a per-image IMAGE_MODE variable that can be set as desired in local.conf for existing image recipes or fixed on a specific value in specific images. It also customizes /etc/motd in the images, just like the refkit-specific code did before. The default (in our local.conf.sample) is to build images in development mode. This is a bit more relaxed than the approach we inherited from Ostro OS where a developer was explicitly forced to choose. Because setting the Secure Boot signing keys no longer depends on the .inc files, the default development keys can be moved to meta-refkit-core, where they are used as default for IMAGE_MODE != production. It is possible that the same build may now need an refkit-initramfs in development mode (= default dm-verity keys) and one in production mode. This is solved by providing refkit-initramfs in different variants, generated automatically using image-mode-variants.bbclass. For this to work, we have to set valid refkit modes globally. Extra care is needed to avoid setting IMAGE_MODE and IMAGE_MODE_VALID themselves globally: the values that are valid for refkit images are not valid for other images, like core-image-minimal. While that particular image doesn't do anything with IMAGE_MODE, others might. Therefore we set REFKIT_IMAGE_MODE[_VALID] globally and copy to the values checked by the helper classes in the refkit images. The refkit images use the REFKIT_IMAGE_MODE set in our local.conf.sample, which may vary in actual builds. Testing continues to use that because if already built in the right mode, there's no need to rebuilt an image with a fixed mode. If someone wants, they now can also built images with a specific mode that also gets included in the resulting image files (for example, refkit-image-computervision-development). Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Getting an emergency shell during boot failures is useful during development but should not be enabled in production. We can use the image mode to enable it, then developers will have it automatically. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
.rst is more useful as format, because it gets rendered nicely on Github. Also include some actual information. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The refkit-conf.inc is now split into distinct sections, with more general ones at the top and specific recipe changes at the bottom. PACKAGECONFIG changes are meant to be done in the .inc file, so move the systemd change there. More complex changes are meant to be in .bbappends, so recreate the python .bbappends (but with an .inc file for the common code this time, and in a simpler way that can be made conditional). Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
refkit-image.bbclass might be used without systemd, so ignore the situation where the files that we want to edit resp. remove aren't installed. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Actually trying to build finds issues in our code that does not show up during the "bitbake -S none" parse and dependency analysis, like broken code that is based on assumptions about the refkit distro. For example, refkit_image_muted only worked for images containing systemd. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
REFKIT_VFAT_MB was already needed by image-dsk.bbclass and not setting it broke the stand-alone usage of that class. Setting WKS_FILE in image-dsk.bbclass also has the advantage that we don't interfere with the defaults when the DSK format is not active. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
meta-intel sets a WKS_FILE (systemd-bootdisk-uuid.wks) which seems to assume that the class in EFI_PROVIDER (rmc-boot) gets inherited by the images. This used to be done indirectly through the "live" IMAGE_FSTYPES entry, but doesn't not happen in refkit images because we disable that type. Instead we need to load that class when our own image-dsk.bbclass is not active. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The code used to fail when using refkit-image.bbclass with DISTRO=poky
because the os-release file wasn't installed. Now we check that
instead of failing.
While at it, other copies of that file are also patched. /etc and
${sysconfdir} are both tried, just in case.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
When using the image without systemd we cannot update its configuration. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Disabling the dependency on syslinux and grub-efi is only valid for refkit images and (besides saving some resources) necessary when using refkit.conf because we do not list those components as supported. When using other distros, we need to keep these dependencies, because otherwise non-refkit images built without errors but then do not boot as no bootx64.efi gets installed. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The image feature should not be enabled when not using systemd. Provide a better error message where it previously just failed due to missing files. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
With several minor problems fixed, we can now build and boot the common image both with the default Poky configuration (test_common_poky_config) and with DISTRO=poky + refkit-config (test_common_refkit_config). This replaces build-testing of the computer vision image. The problems found with that test were all in common code, so it makes sense to keep the test a bit simpler and focus just on that. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
nodejs (meta-oe), IoTivity (meta-oic), and packagegroup-nodejs-runtime (meta-iot-web) are needed by refkit-image-gateway. This was not detected earlier because images were excluded from the world build breakage test. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Not listing them had the effect of excluding them from a world build. Explicitly building them was possible without triggering warnings. We want our package groups to be part of a world build to detect dependency errors. The same goes for images: they are excluded because of EXCLUDE_FROM_WORLD in image.bbclass, but a test might want to change that. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
yocto-compat-layer.py did not detect missing dependencies in meta-refkit-gateway because the refkit-image-gateway was excluded from world builds. We need to explicitly override that default. However, this only works for images added by the layer that is getting tested, because existing images change their signature (unnecessarily?) when BBLAYERS gets changed. For this config change to work with self.append_config(), test_compat_* tests must use the same scaffolding as the rest of the test, so we might as well move that to setUp/tearDownLocal(). Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The artificial Poky build directory did not have QEMU_USE_KVM set up, so tests ran without KVM. We need to copy that from the build configuration in which oe-selftest gets started. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Both package groups have dependencies which are no longer considered supported in refkit: packagegroup-core-can -> *can-init-scripts packagegroup-core-can -> *can-utils packagegroup-tools-interactive -> *libusb-compat packagegroup-tools-interactive -> *usbutils -> *libusb-compat Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
|
test this please Squashed some fixes into "refkit: replace development/production .inc files with per-image IMAGE_MODE"
Added one new commit: c0ad16a "refkit-image.bbclass: enable emergency shell in development mode" |
The project decided at some point to use NodeJS from meta-iot-web. Therefore that's the layer that we should check for. meta-oe isn't actually needed for meta-refkit-gateway. Less dependencies is better for reuse. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
|
test this please Previous build failed with a temporary download error: It should have used the DLDIR from the main build, not sure why it wasn't cached. Something to investigate... |
|
retest this please |
|
test this please |
|
retest this please |
The intention is to make it simpler for other projects to use content from
intel-iot-refkit without having to use the entire distro. This first commit
is good enough to keep building the refkit distro and just moves files
around without modifying them. Further work is needed to make the
individual pieces also functional stand-alone.