Support writing to a branch

[Fokko]

Feature Request / Improvement

Right now we hardcoded that we write to the main branch all the time, would be great to make this configurable.

[kevinjqliu]

The API to write to a branch should look something like

def append(self, df: pa.Table, branch: str = MAIN_BRANCH)
...
def overwrite(self, df: pa.Table, overwrite_filter: BooleanExpression = ALWAYS_TRUE, branch: str = MAIN_BRANCH)
...

But in order to write to a branch, the branch needs to be created first.

From https://iceberg.apache.org/docs/latest/spark-writes/#writing-to-branches:

the branch must exist before performing the write. The operation does not create the branch if it does not exist.

[kevinjqliu]

First pass, just refactoring
#312

[kevinjqliu]

We first need a create branch API.

Then update places currently gated by MAIN_BRANCH.

1. iceberg-python/pyiceberg/table/metadata.py

   Line 124 in 9e03949

   if MAIN_BRANCH not in table_metadata.refs:

2. iceberg-python/pyiceberg/table/__init__.py

   Line 592 in 9e03949

   if update.ref_name == MAIN_BRANCH:

And finally add a test for writing to a branch

[kevinjqliu]

also note dev/provision.py which is used for integration tests already have statements to create tags and branchs

iceberg-python/dev/provision.py

Lines 146 to 148 in 9e03949

	spark.sql(f"ALTER TABLE {catalog_name}.default.test_positional_mor_deletes CREATE TAG tag_12")
	spark.sql(f"ALTER TABLE {catalog_name}.default.test_positional_mor_deletes CREATE BRANCH without_5")

[Gowthami03B]

I had an offline chat with @kevinjqliu , I shall work on this to build off of the PR created by kevin.

[Gowthami03B]

Hello, can I hop back on this train if no one else is actively working on this (again building off of kevin's work)? @kevinjqliu @Fokko

[kevinjqliu]

@Gowthami03B definitely, I've assigned the issue to you!

[Gowthami03B]

@Gowthami03B definitely, I've assigned the issue to you!

Opening up this to the community as I am gonna be out for the next month! @kevinjqliu

[vinjai]

@kevinjqliu @Fokko
I can take this forward if no one is actively working on this.

[kevinjqliu]

@vinjai yes! please go ahead.

[sungwy]

Hi @vinjai thank you very much for working on this issue. I'm just working through the list of open items to check if they are still actively being worked on. Are you still interested in contributing this feature to PyIceberg? 🧊

[vinjai]

Hey @sungwy
I am working on this at the moment
Will open the PR for review by next week

[vinjai]

PR is ready for review