Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(blog): portable dataflows with Ibis and Hamilton #8798

Merged
merged 6 commits into from
Apr 2, 2024

Conversation

zilto
Copy link
Contributor

@zilto zilto commented Mar 27, 2024

Description of changes

Ported a blog post from the Hamilton Substack to the Quarto format for cross-posting on the Ibis blog. This was discussed with @lostmygithubaccount on Zulip.

First time using Quarto. I managed to successfully build .qmd files and preview the post. Everything seemed fine. However, I see other blogs have files under _freeze/posts/, but I couldn't find how to generate them.

Copy link
Contributor

ACTION NEEDED

Ibis follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message.

Please update your PR title and description to match the specification.

@lostmygithubaccount lostmygithubaccount changed the title docs[blog]: Portable dataflows with Ibis and Hamilton docs(blog): Portable dataflows with Ibis and Hamilton Mar 27, 2024
@lostmygithubaccount lostmygithubaccount changed the title docs(blog): Portable dataflows with Ibis and Hamilton docs(blog): portable dataflows with Ibis and Hamilton Mar 27, 2024
@lostmygithubaccount
Copy link
Member

thanks! I think this will fail the docs PR check, as hamilton will not be installed in the environment. you'd need to render the document yourself with quarto (via quarto preview or quarto render) and check in the changed _freeze files. this will have quarto use the saved execution output for the blog instead of trying to run it

@cpcloud
Copy link
Member

cpcloud commented Mar 27, 2024

@lostmygithubaccount It's passing because the code blocks aren't executed.

@lostmygithubaccount
Copy link
Member

ah missed that -- will review soon! @zilto you may want to have the code blocks actually execute (and then my comment above would apply)

other offhand comment before a proper review would be wrapping the lines to ~90 characters -- see https://ibis-project.org/contribute/03_style#text-formatting

@zilto
Copy link
Contributor Author

zilto commented Mar 27, 2024

@lostmygithubaccount It's passing because the code blocks aren't executed.

aaah, that makes a lot sense. I was scratching my head at why no _freeze/posts/hamilton-ibis directory was generated.

Will make the changes

@zilto
Copy link
Contributor Author

zilto commented Mar 28, 2024

I updated the .qmd file to use executable code blocks and generate the _freeze/ file.

Now, my code examples depend on a parquet dataset that is 23KB. I didn't commit it since .parquet files are git ignored by default. Should I include it to make the code runnable?

@lostmygithubaccount
Copy link
Member

@zilto apologies for the slow response here!

Now, my code examples depend on a parquet dataset that is 23KB. I didn't commit it since .parquet files are git ignored by default. Should I include it to make the code runnable?

I think this is small enough that just adding it is fine. alternatives would be:

  1. host this somewhere publicly that can be read from
  2. switch to using a built-in ibis.examples dataset

neither really seem worth it in this case, and 2) would mean changing a lot of the code

@gforsyth
Copy link
Member

@zilto -- if you add the parquet file here (or you can send it over Zulip to me -- at least I think that's possible) I can stick it in one of our buckets and give you a public http link to pull from.
I agree checking it in is not a huge deal, but it's also easy to stick in the bucket and have a reasonably permanent link (in the medium-term, anyway)

@lostmygithubaccount
Copy link
Member

it is trying to run the Python code (and not finding the hamilton import), suggesting the execution freeze doesn't exactly match after you made some changes. can you try re-executing and ensuring it's up to date?

@zilto
Copy link
Contributor Author

zilto commented Apr 1, 2024

Ok, odd bug... I just updated the files

@lostmygithubaccount lostmygithubaccount added docs Documentation related issues or PRs docs-preview Add this label to trigger a docs preview labels Apr 1, 2024
@ibis-docs-bot ibis-docs-bot bot removed the docs-preview Add this label to trigger a docs preview label Apr 1, 2024
@lostmygithubaccount lostmygithubaccount added the docs-preview Add this label to trigger a docs preview label Apr 1, 2024
@ibis-docs-bot ibis-docs-bot bot removed the docs-preview Add this label to trigger a docs preview label Apr 1, 2024
@ibis-docs-bot
Copy link

ibis-docs-bot bot commented Apr 1, 2024

@cpcloud cpcloud added this to the 9.0 milestone Apr 2, 2024
@cpcloud cpcloud force-pushed the posts/hamilton-ibis branch from 278d431 to 1dcada6 Compare April 2, 2024 10:20
@cpcloud cpcloud enabled auto-merge (squash) April 2, 2024 10:21
@cpcloud cpcloud merged commit e97aa46 into ibis-project:main Apr 2, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation related issues or PRs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants