Add replication support to dbatools!

[jpomfret]

Please read -- recent changes to our repo

On November 10, 2022, we removed some bloat from our repository (for the second and final time). This change requires that all contributors reclone or refork their repo.

PRs from repos that have not been recently reforked or recloned will be closed and @potatoqualitee will cherry-pick your commits and open a new PR with your changes.

• Please confirm you have the smaller repo (85MB .git directory vs 275MB or 110MB or 185MB .git directory)

Type of Change

• Bug fix (non-breaking change, fixes # )
• New feature (non-breaking change, adds functionality, fixes # )
• Breaking change (affects multiple commands or functionality, fixes # )
• Ran manual Pester test and has passed (.\tests\manual.pester.ps1)
• Adding code coverage to existing functionality
• Pester test is included
• If new file reference added for test, has is been added to github.com/dataplat/appveyor-lab ?
• Unit test is included
• Documentation
• Build system

Purpose

Adding replication support to dbatools!

Approach

There are many commands added with this PR to cover most of the replication functionality we can add - there is still work to do on these commands, more options and settings that can be added - and more code paths. However, they are all working and at this point I want to create this PR and start moving them towards the dbatools dev branch so we can start getting feedback.

I realise I've included the psd1 in this PR - we can remove it before we merge in - just makes testing this branch easier

Commands to test - New commands

Distribution

• Enable-DbaReplDistributor
• Disable-DbaReplDistributor

Publishing

• Enable-DbaReplPublishing
• Disable-DbaReplPublishing

Publications

• New-DbaReplPublication
• Remove-DbaReplPublication

Articles

• Get-DbaReplArticle
• Get-DbaReplArticleColumn
• Add-DbaReplArticle
• Remove-DbaReplArticle
• New-DbaReplCreationScriptOptions

Subscriptions

• New-DbaReplSubscription - works but doesn't output anything - waiting on Get-DbaReplSubscription
• Remove-DbaReplSubscription - once Get-DbaReplSubscription works I want to rework this one to enable piping
• Get-DbaReplSubscription -- this command still needs work - can't currently get the subscription information from the subscriber instance

Commands to test - Existing commands that have been spruced up

• Get-DbaReplServer
• Get-DbaReplDistributor
• Get-DbaReplPublication

Commands that still need some love

• Test-DbaRepLatency - haven't got to this one yet, but it doesn't work
• Export-DbaRepServerSetting - this exports server settings - could be expanded to include publications etc

Testing

The following script takes you through building and destroying replication:

## based off these two containers
# create a shared network
docker network create localnet

# Expose engines and setup shared path for migrations
docker run -p 2500:1433  --volume shared:/shared:z --name mssql1 --hostname mssql1 --network localnet -d dbatools/sqlinstance
docker run -p 2600:1433 --volume shared:/shared:z --name mssql2 --hostname mssql2 --network localnet -d dbatools/sqlinstance2

# create the repl folder
docker exec mssql1 mkdir /var/opt/mssql/ReplData

# also need these folders for setting up replication
docker exec mssql1 mkdir /shared/data /shared/repldata


# using psdefaultparametervalues to make connections easy
$securePassword = ('dbatools.IO' | ConvertTo-SecureString -AsPlainText -Force)
$credential = New-Object System.Management.Automation.PSCredential('sqladmin', $securePassword)

$PSDefaultParameterValues = @{
    "*:SqlCredential"            = $credential
    "*:DestinationCredential"    = $credential
    "*:DestinationSqlCredential" = $credential
    "*:SourceSqlCredential"      = $credential
    "*:PublisherSqlCredential"   = $credential
}


# enable distribution
Enable-DbaReplDistributor -SqlInstance mssql1

# enable publishing
Enable-DbaReplPublishing -SqlInstance mssql1

# Get the distributor
Get-DbaReplDistributor -SqlInstance mssql1

# add a transactional publication
$pub = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Name        = 'testPub'
    Type        = 'Transactional'
}
New-DbaReplPublication @pub -verbose

# add a merge publication
$pub = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Name        = 'mergey'
    Type        = 'Merge'
}
New-DbaReplPublication @pub

# add a snapshot publication
$pub = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Name        = 'snappy'
    Type        = 'Snapshot'
}
New-DbaReplPublication @pub

# view publications
Get-DbaReplPublication -SqlInstance mssql1

# add an article to each publication
$article = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Publication = 'testpub'
    Name        = 'publishers'
    Filter      = "city = 'seattle'"
}
Add-DbaReplArticle @article

$article = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Publication = 'Mergey'
    Name        = 'publishers'
}
Add-DbaReplArticle @article

$article = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Publication = 'snappy'
    Name        = 'publishers'
}
Add-DbaReplArticle @article

# view articles
Get-DbaReplArticle -SqlInstance mssql1

# add subscriptions
$sub = @{
    SqlInstance               = 'mssql2'
    Database                  = 'pubs'
    PublicationDatabase       = 'pubs'
    PublisherSqlInstance      = 'mssql1'
    PublicationName           = 'testpub'
    Type                      = 'Push'
    SubscriptionSqlCredential = $credential

}
New-DbaReplSubscription @sub -enableexception

$sub = @{
    SqlInstance               = 'mssql2'
    Database                  = 'Mergeypubs'
    PublicationDatabase       = 'pubs'
    PublisherSqlInstance      = 'mssql1'
    PublicationName           = 'Mergey'
    Type                      = 'Push'
    SubscriptionSqlCredential = $credential

}
New-DbaReplSubscription @sub

$sub = @{
    SqlInstance               = 'mssql2'
    Database                  = 'Snappypubs'
    PublicationDatabase       = 'pubs'
    PublisherSqlInstance      = 'mssql1'
    PublicationName           = 'Snappy'
    Type                      = 'Push'
    SubscriptionSqlCredential = $credential
}
New-DbaReplSubscription @sub

# remove subscriptions
$sub = @{
    SqlInstance          = 'mssql2'
    SubscriptionDatabase = 'pubs'
    PublisherSqlInstance = 'mssql1'
    PublicationDatabase  = 'pubs'
    PublicationName      = 'testPub'
}
Remove-DbaReplSubscription @sub

$sub = @{
    SqlInstance          = 'mssql2'
    SubscriptionDatabase = 'Mergeypubs'
    PublisherSqlInstance = 'mssql1'
    PublicationDatabase  = 'pubs'
    PublicationName      = 'Mergey'
}
Remove-DbaReplSubscription @sub

$sub = @{
    SqlInstance          = 'mssql2'
    PublisherSqlInstance = 'mssql1'
    PublicationName      = 'snappy'
    PublicationDatabase  = 'pubs'
    SubscriptionDatabase = 'Snappypubs'
}
Remove-DbaReplSubscription @sub

# remove an article
$article = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Publication = 'testpub'
    Name        = 'publishers'
}
Remove-DbaReplArticle @article

# remove an article
$article = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Publication = 'Mergey'
    Name        = 'publishers'
}
Remove-DbaReplArticle @article

# remove an article
$article = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Publication = 'snappy'
    Name        = 'publishers'
}
Remove-DbaReplArticle @article

## remove publications
$pub = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Name        = 'Snappy'
}
Remove-DbaReplPublication @pub

$pub = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Name        = 'TestPub'
}
Remove-DbaReplPublication @pub

$pub = @{
    SqlInstance = 'mssql1'
    Database    = 'pubs'
    Name        = 'Mergey'
}
Remove-DbaReplPublication @pub


# disable publishing - bug in Remove-DbaReplPublication - need to check on whether the database gets disabled for replication when last pub gets dropped
Disable-DbaReplPublishing -SqlInstance mssql1 -force

# disable distribution
Disable-DbaReplDistributor -SqlInstance  mssql1

[jpomfret]

Hey @wsmelton thanks for going through this - I've addressed all your concerns (I think) except the naming of these four commands:

• Enable-DbaReplDistributor
• Disable-DbaReplDistributor
• Enable-DbaReplPublishing
• Disable-DbaReplPublishing

Also I can only apologise for the massive PR - it was hard to break into chunks since so many of the commands work together.

[jpomfret]

Yeah makes sense - will wait for @potatoqualitee steer on this one.

[jpomfret]

Have gone through and made sure -EnableException:$EnableException is included on all public functions - thanks

[jpomfret]

failed tests seem to be unrelated

[potatoqualitee]

failed tests seem to be unrelated

Login to appveyor using your GH creds and see if you can do the rerun incomplete button

[potatoqualitee]

@wsmelton @jpomfret sorry for the delay. I can see it both ways. You Enable Replication but create a New Publisher eh?

What verbiage do you use when talking to a customer?

If it's not Enable/Disable, that would be the only thing I could see that would override SSMS.

[jpomfret]

Hey @potatoqualitee I'd say enable\disable - that's the wording around replication in the SQL Server world.

For appveyor I got logged in but don't see an option to rerun jobs 🤔

[jpomfret]

Hey @wsmelton \ @potatoqualitee - we happy to stick with enable\disable since it matches the experience and the terminology of replication, while still using approved verbs?

I have a presentation next week so would love to be able to say it's in the dev branch ready for the next release if possible. :)

[potatoqualitee]

YOOOO lez do this! Congrats!! <3

[potatoqualitee]

Thank you both so very much, what a huge amount of work. Looking forward to promoting this on socials 💯