Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow for .proto editing #18

Merged
merged 7 commits into from
Sep 6, 2017
Merged

allow for .proto editing #18

merged 7 commits into from
Sep 6, 2017

Conversation

mapsam
Copy link
Contributor

@mapsam mapsam commented Sep 5, 2017

Implementing idea from @springmeyer in #16 where instead of a manipulate function, we generate a fixture with a manipulated .proto file which allows us to be more confident in value types for each fixture. This does a few things:

  • removes the manipulate field from the source fixtures
  • adds a proto fields to the source fixtures. This can be a string representing the version, such as 2.1 or a string representing a full .proto file.
  • in order to make the proto field less verbose, you can use the util.replace() method to replace a portion of a currently existing spec .proto file. For example, here's how update a field type.
  • creates a /lib directory with some utility scripts and a generateBuffer method which handles the logic of creating a buffer from a pre-existing .proto file or a custom one
  • updates each source fixture to not include the schema object in the function. Instead we can use literal values for enums

One thing to be aware of is our usage of the mvt.Tile.write() method here. This assumes your custom protocol buffer has a Tile message at the root of the proto spec. I'm okay with this for now, but you can see in the create.test.js file that the custom .proto file still needs this one reference. Perhaps we want to update generateBuffer to take this into account?

Up for review @springmeyer!

Copy link
Contributor

@springmeyer springmeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks excellent @mapsam. I made some minor comments inline. The only general question I have is whether you considered making the json property a function? If it were a function, and took the schema as an arg, then you could keep the ability to use its enums.

index.js Outdated
mvt.Tile.write(json, pbf);
return new Buffer(pbf.finish());
};
const eight = require('./src/008.js');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this maybe left behind from testing, or does it do something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leftover, removing!

}

final.id = id;
final.description = fixture.description;
final.specification_reference = fixture.specification_reference;
final.json = fixture.json;
final.buffer = generateBuffer(fixture.json);
final.proto = fixture.proto;
final.buffer = generateBuffer(fixture.json, fixture.proto);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed you removed manipulate from the fixtures, but left it here. I like that: we may still want to hook into it for certain fixtures where adding bad data is not easy via just an alternative scheme.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heh, this was a mistake, but yeah might as well keep it! I'll make sure the starter template includes it with a note.

*/
module.exports = function(json, proto) {
let proto_schema;
if (!proto) proto = '2.1';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about throwing if no proto is provided? That feels more robust to me since it would ensure each fixture self-documents which spec version it applies do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

scripts/new.js Outdated
{
id: 1,
tags: [],
type: schema.Tile.GeomType.POINT.value,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need updated to be 1 given the current refactor, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@springmeyer springmeyer mentioned this pull request Sep 6, 2017
const Pbf = require('pbf');
const Compile = require('pbf/compile');

const MVT_SPEC_VERSIONS = ['1.0.0', '1.0.1', '2.0', '2.1'];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should ideally be dynamic. Can you think of a way to read these dynamically to avoid hardcoding?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

@mapsam
Copy link
Contributor Author

mapsam commented Sep 6, 2017

The only general question I have is whether you considered making the json property a function?

I removed the function for simplicity, since using the ENUMs provided by pbf felt more complicated than referring to the actual specification (which seems like a good habit to build). Happy to add it back if you think it's important!

@mapsam
Copy link
Contributor Author

mapsam commented Sep 6, 2017

@springmeyer I've made changes based on your edits - thank you for the review! Lemme know how things look.

@mapsam mapsam closed this Sep 6, 2017
@mapsam mapsam reopened this Sep 6, 2017
@mapsam mapsam merged commit eab8801 into master Sep 6, 2017
@springmeyer
Copy link
Contributor

changes look great @mapsam.

I removed the function for simplicity, since using the ENUMs provided by pbf felt more complicated than referring to the actual specification (which seems like a good habit to build). Happy to add it back if you think it's important!

Sounds good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants