Marten V5 Planning?!?

[jeremydmiller]

Hey all, there's been plenty of cross talk between Babu, Oskar, and I about a quickly forthcoming V5, and I wouldn't mind opening up a formal discussion here about what will be or not be in scope. Let me immediately throw down the marker that this will not be anywhere as big a release as the two year V4 release. The initial driver for going to a full point version bump is .Net 6 support, with Npgsql actually being our "breaking" change to APIs. Oskar has already done most of the work necessary to roll out a new Marten v5 with just .Net 6 updates plus some minor API cleanups replacing DateTime with DateTimeOffset in some event sourcing APIs.

So, we could go quickly and release V5, or maybe even just a V5 alpha to get folks up and going with .Net 6 then do a Marten V6 with some of the potentially breaking changes I wanna talk about here. Or we go a little slower and release Marten 5 in maybe January (I think it's really important to have it available for new projects in the new year).

That being said, here's the big changes I think I want to propose and discuss here for V5:

1. Database per Client -- it's coming up a lot all the sudden
2. First class support for a single application to talk to multiple, completely different Marten databases. There's no issue for this yet, but it comes up sometimes, and I think there's some synergy with potential solutions for this and the "database per tenant" work. I mean having DI integration, and multiple database migration support from the same app
3. New "auto-generate" mode for the code-ahead model to make it much easier for folks to use and not require the CLI setup. I wanna try this before we give up on the current codegen model and retreat to IL generation (Expression to Lambda compilation is not sufficient for our needs IMO and source generators aren't as well)
4. Pull the async daemon bootstrapping to a separate addon to AddMarten(). So AddMarten(blah, blah, blah).IncludeAsyncProjectionDaemon(HotCold/Solo). This is for Azure Function friendliness
5. Pull the initial data stuff out so that it runs inside an IHostedService. Helps us not need mixed async/sync code
6. Make all synchronous data access apis be deprecated?????? Holy hell that's gonna hurt, but still
7. Formal event streaming through the async daemon just for the purpose of having a simple out of the box solution that gives you ordering guarantees.

There's more things that have come up, but that's what I've been thinking of. Now, everyone else's turn.

[wastaz]

@jeremydmiller I remember seeing an issue somewhere about some potential case where the async projection daemon could drop some events. Even if it is rare (which I hope it is) any fixes on that would be very appreciated. (Also since I cant find where I read about it, if you have any tips on when this may happen Id love to hear them :) )

[jeremydmiller]

The async daemon was 100% rewritten in V4

[wastaz]

So with the v4 rewrite there are no known edge cases where at least once processing cant be guaranteed?

[jeremydmiller]

"known" is correct

[jeremydmiller]

Couple more thoughts about V5 and now V6:

• Dedicated DevelopmentMode() and ProductionMode() that in one fell swoop, configures AutoCreate and the pre-built codegen. Make it real easy for folks to use the best, idiomatic approach for using Marten in prod v. dev.
• Make all synchronous API calls that are IO bound [Obsolete]. Much to gain here in terms of code simplification, lots of potential aggravation on users. Now I'm proposing that we kill off synchronous APIs like SaveChanges() in V6 that would follow shortly, then start adding big new features in V6 with the smaller test set and smaller codebase.
• Add environment checks to the command line support. Easy task
• Conditional start up tasks within Marten to register IHostedService to apply latest changes to the schema at start up time? Also low hanging fruit.

[jannikbuschke]

+1 for database per tenant

[lngr]

So, we could go quickly and release V5, or maybe even just a V5 alpha to get folks up and going with .Net 6

I'm in favor of a quick V5 with the Npgsql 6 upgrade (#1987) to sync the stack to .NET 6, which is an LTS release
(by "sync", I mean e.g. EF Core 6 which requires Npgsql 6).

[jcdickinson]

I'm definitely in favor of the quick V5. Long story short, net6 is required for any reasonable level of .Net development on Apple M1 (and M1 is your only option if Apple is the corp policy).

If there are any blockers to that end, I'm more than willing to get my hands dirty and contribute.

[jeremydmiller]

@oskardudycz @mysticmind I'm just about to have significant time to start working on Marten again for the other improvements up there, but I think my vote now is to do the quick V5 that's all about .Net 6, then roll out V6 with the other improvements (and the expectation that 4 to 5 to 6 is a much easier transition than 3 to 4 has been). And let that V6 stabilize at that point for awhile before we do a V7 that cuts off the sync path?

[mysticmind]

@oskardudycz @mysticmind I'm just about to have significant time to start working on Marten again for the other improvements up there, but I think my vote now is to do the quick V5 that's all about .Net 6, then roll out V6 with the other improvements (and the expectation that 4 to 5 to 6 is a much easier transition than 3 to 4 has been). And let that V6 stabilize at that point for awhile before we do a V7 that cuts off the sync path?

Yes, the above makes sense. We will plan to push out v5 this week then.

[jeremydmiller]

Hey folks, since V5 is sitting as alpha, I want to ask y'all to give me at least the first week of January to see if I can get this in now. I know, changing my mind over and over again, but I've got momentum on this stuff now