Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly serialize exports in service rumors #5987

Merged
merged 3 commits into from
Jan 11, 2019

Conversation

jamesc
Copy link

@jamesc jamesc commented Dec 15, 2018

Solves #5854 by making sure that we always serialize the export configuration as valid TOML.

This PR moves the logic into Cfg::to_exported from the Service::new constructor so we can handle the error more sanely and reuse some of the serialization logic that is already there.

It feels like there might be a more natural way to do this via Serde by constructing some more objects to represent this service config but not sure if it's worth the extra code and abstraction. Would love some input on this.

@thesentinels
Copy link
Contributor

Thanks for the pull request! Here is what will happen next:

  1. Your PR will be reviewed by the maintainers
  2. If everything looks good, one of them will approve it, and your PR will be merged.

Thank you for contributing!

@jamesc jamesc force-pushed the jc/5854/pkg_exports-panic branch from 8368821 to 6b7b07c Compare December 17, 2018 20:16
@baumanj
Copy link
Contributor

baumanj commented Dec 17, 2018

This is code in an area I'm not at all familiar with, so it's going to take me some time to give you good feedback. What's the urgency level of this change?

@jamesc
Copy link
Author

jamesc commented Dec 17, 2018

The bug has been there for a while already and there are workarounds. Doesn't need to be squeezed into this release.

@jamesc jamesc force-pushed the jc/5854/pkg_exports-panic branch 2 times, most recently from 1a77748 to 4308a20 Compare December 19, 2018 19:05
@jamesc jamesc force-pushed the jc/5854/pkg_exports-panic branch 3 times, most recently from 7ce497d to 373d118 Compare January 5, 2019 03:41
@baumanj
Copy link
Contributor

baumanj commented Jan 7, 2019

Apologies for the delay; I'll start digging into this tomorrow

@baumanj
Copy link
Contributor

baumanj commented Jan 8, 2019

I've mostly got my head wrapped around this code now (it's kind of a doozy). Hopefully I'll be able to finish this review tomorrow.

Copy link
Contributor

@baumanj baumanj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I think we can make this change smaller (see inline comments) and reduce the amount of code we have for serialization by relying more on the external crates.

Additionally, I think we should add a test to components/butterfly/src/rumor/service.rs which explicitly tests the regression. I've confirmed this panics the existing code (with the minor change that Some(map) becomes Some(&map) if we change the cfg parameter to Service::new.

    #[test]
    fn service_cfg_serialization() {
        let package: PackageIdent = "core/foo/1.0.0/20180701125610".parse().unwrap();
        let sg = ServiceGroup::new(None, "foo", "default", None).unwrap();

        // This map contains a scalar value and a table such that the serialization order
        // would trigger a ValueAfterTable error. This test ensures we avoid it.
        // See https://github.com/habitat-sh/habitat/issues/5854
        // See https://github.com/alexcrichton/toml-rs/issues/142
        let mut map = toml::value::Table::default();
        let sub_map = toml::value::Table::default();
        map.insert("foo".into(), 5.into());
        map.insert("a".into(), sub_map.into());

        Service::new("member_id_val", &package, sg, SysInfo::default(), Some(map));
    }

components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
components/butterfly/src/rumor/service.rs Outdated Show resolved Hide resolved
components/butterfly/src/rumor/service.rs Outdated Show resolved Hide resolved
components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
components/sup/src/manager/service/config.rs Outdated Show resolved Hide resolved
James Casey added 3 commits January 11, 2019 09:35
The root of the panic is due to us trying to serialize a
`Toml::Value::Table` to a Vec<u8> inside a constructor where we can't
handle the error cleanly

This commit pulls the TOML parsing out of Service::new() into
Cfg::to_exported() so we can properly handle the Result<> on toml parsing

Note: this does not yet handle the underlying issue which is that the
TOML serializer doesn't handle tables well.  In particular it requires
all keys with non-table values must be emitted first.  This is not the
default

See https://github.com/alexcrichton/toml-rs/blob/master/src/ser.rs#L7-L12

Signed-off-by: James Casey <james@chef.io>
exported config

Abstract out existing special casing for serialization of config and
re-use it when creating the exported config
Signed-off-by: James Casey <james@chef.io>
Signed-off-by: James Casey <james@chef.io>
@jamesc jamesc force-pushed the jc/5854/pkg_exports-panic branch from 373d118 to 26457fa Compare January 11, 2019 20:03
@jamesc jamesc merged commit e2ba688 into habitat-sh:master Jan 11, 2019
@jamesc jamesc deleted the jc/5854/pkg_exports-panic branch January 11, 2019 21:24
chef-ci added a commit that referenced this pull request Jan 11, 2019
Obvious fix; these changes are the result of automation not creative thinking.
@christophermaier christophermaier added the Type:BugFixes PRs that fix an existing bug label Jul 24, 2020
@christophermaier christophermaier added Focus:Supervisor Related to the Habitat Supervisor (core/hab-sup) component and removed X-fix labels Jul 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Focus:Supervisor Related to the Habitat Supervisor (core/hab-sup) component Type:BugFixes PRs that fix an existing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants