Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let irrd_load_database do more work #233

Closed
job opened this issue Jun 19, 2019 · 4 comments
Closed

Let irrd_load_database do more work #233

job opened this issue Jun 19, 2019 · 4 comments
Assignees
Labels
enhancement New feature or request

Comments

@job
Copy link
Member

job commented Jun 19, 2019

Is your feature request related to a problem? Please describe.
Right now I have to provide a gunzipped dump and serial to the irrd_load_database tool, but the tool could also download the dump & serial based on the URL known in the yml config.

Describe the solution you'd like

$ grep -A6 RIPE: /etc/irrd.yaml | grep import
            import_serial_source: ftp://ftp.ripe.net/ripe/dbase/RIPE.CURRENTSERIAL
            import_source: ftp://ftp.ripe.net/ripe/dbase/ripe.db.gz
$ irrd_load_database -s RIPE   # the tool can fetch & gunzip & fetch serial itself based on the config
   ... 

Describe alternatives you've considered
Flipping a flag in the database?

@job job added the enhancement New feature or request label Jun 20, 2019
@mxsasha
Copy link
Collaborator

mxsasha commented Jun 20, 2019

I'm curious about your usecase. This is basically something the force_reload flag already does. We can add a command to set that flag for a source?

The load_database command was intended for script-generated databases, i.e. those that update relatively rarely, and need stricter validation than regular mirrors. I think it would also refuse to import RIPE, because of unknown object classes. The load_database command is quite strict, as for errors the assumption is "the script generating that file made a mistake" vs "do a best effort import of valid data" for regular mirrors.

@job
Copy link
Member Author

job commented Jun 20, 2019

Yes, i guess my request is to add a CLI command to set the force_reload flag so we don't need operations to connect directly to the database. Perhaps some luxury such as progress bar would be nice too.

@mxsasha
Copy link
Collaborator

mxsasha commented Jun 24, 2019

The behaviour of the new command is now documented in: https://irrd4.readthedocs.io/en/latest/users/mirroring.html#nrtm-mode

Progress bars are difficult - the actual reload happens asynchronously. The command just flips the flag in the DB, the next run of the importer for that source reads it and acts on it.

@job
Copy link
Member Author

job commented Jun 26, 2019

this works as advertised!

mxsasha added a commit that referenced this issue Jun 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants