Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define a fst_proxy class #12

Closed
MarcusKlik opened this issue Mar 27, 2018 · 1 comment
Closed

Define a fst_proxy class #12

MarcusKlik opened this issue Mar 27, 2018 · 1 comment

Comments

@MarcusKlik
Copy link
Collaborator

The fsttable package has two main tasks that are best separated into two distinct components:

  • interpret the data.table interface. This requires non-standard evaluation where we compute on the language R itself. The code for this will be specific to the data.table interface.
  • a proxy object to the underlying fst file. This object stores row- and column- selections, virtual columns and ordering information. Given a specific configuration, data can be retrieved from the fst file. This component should not use any language features or methods specific to the data.table package. And it should be made abstract enough so that it can be re-used for the fstplyr package (or any other future interface, e.g. SQL).

Perhaps it's best to separate these components from the start. The second component will also make clear what the best API will be for the fst package to expose. In the future this component could go into a separate package that can act as a dependency for other interfaces (package fstref or fstproxy).

@MarcusKlik
Copy link
Collaborator Author

all calls to the fst package now live in the fst_proxy object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant