Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging Develop into Main #868

Merged
merged 196 commits into from
Dec 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
196 commits
Select commit Hold shift + click to select a range
2c6e49c
Making more util trcsub for common.
joel-rieke Nov 3, 2023
8193f26
Apply state changes.
joel-rieke Nov 3, 2023
c6593f9
Checking in progress.
joel-rieke Nov 3, 2023
ea18deb
Reverting some hardcodings.
joel-rieke Nov 3, 2023
9c09487
Adding some robustness handling and changes to initial states.
joel-rieke Nov 6, 2023
04ac7c3
More state updates for windows side.
joel-rieke Nov 6, 2023
5e5e65d
Allow for definition of service.
joel-rieke Nov 7, 2023
5c00886
Adding Refresh syncMode (#775)
KarnveerSGill Nov 7, 2023
074d3ab
Merge branch 'main' into develop
joel-rieke Nov 8, 2023
fe979e2
Fix missing import.
joel-rieke Nov 8, 2023
37927e9
Merge branch 'main' into develop
joel-rieke Nov 9, 2023
3584f67
Better support for server cert.
joel-rieke Nov 9, 2023
ce4e030
Update to go 1.21.4
joel-rieke Nov 9, 2023
b30057a
Merge branch 'main' into develop
joel-rieke Nov 9, 2023
4a852c5
More robust state handling.
joel-rieke Nov 9, 2023
a153f4c
Merge branch 'main' into develop
joel-rieke Nov 9, 2023
c9d8d3f
Prevent premature exits.
joel-rieke Nov 9, 2023
b4e6f83
Switch to ranging over channel.
joel-rieke Nov 9, 2023
3a20753
Merge branch 'main' into develop
joel-rieke Nov 9, 2023
87d54b6
Cleanup output for deployments.
joel-rieke Nov 9, 2023
71518bf
Add deployment for TrcDeploy
joel-rieke Nov 9, 2023
9706d41
Adding channel unblocker
KarnveerSGill Nov 10, 2023
0a16d69
Adding back timer
KarnveerSGill Nov 10, 2023
b0b02a5
Add template for agent deploys.
joel-rieke Nov 10, 2023
afbdfa8
Update to latest hat.
joel-rieke Nov 10, 2023
0dd7c90
Fixing go.mod
joel-rieke Nov 10, 2023
b6ef0e0
Merge branch 'main' into develop
joel-rieke Nov 10, 2023
308f77d
Merge branch 'releases/plugin/1.0.0' into develop
joel-rieke Nov 10, 2023
0ca4d0d
Merge branch 'main' into develop
joel-rieke Nov 10, 2023
d6e41d2
Cleanup config and certify to handle errors more gracefully for bette…
joel-rieke Nov 10, 2023
8a99a6d
Certify exit with error code.
joel-rieke Nov 10, 2023
0de9ee1
Fixing some crashing.
joel-rieke Nov 10, 2023
9969882
Let this happen async.
joel-rieke Nov 10, 2023
740be3f
Adding some more commands.
joel-rieke Nov 11, 2023
1ae0e1e
Select more suitable param name.
joel-rieke Nov 11, 2023
ac351e7
Merge branch 'main' into develop
joel-rieke Nov 12, 2023
7596b09
Add some error handling with early exit.
joel-rieke Nov 12, 2023
00607cf
Merge branch 'main' into develop
joel-rieke Nov 12, 2023
b6b2c3a
Make deployments re-runnable.
joel-rieke Nov 12, 2023
b247ba2
Set timer for each run. Make CtlMessage a buffered channel.
joel-rieke Nov 12, 2023
0325d08
Merge branch 'main' into develop
joel-rieke Nov 12, 2023
894c28e
Couple more fixes for multi-deploy.
joel-rieke Nov 12, 2023
869f059
Small spelling correction.
joel-rieke Nov 12, 2023
ee7ebc0
Get everything at least running... Output to correct directory not ye…
joel-rieke Nov 12, 2023
3b96306
A little more cleanup.
joel-rieke Nov 12, 2023
bdc7ab7
Add support for projectservice.
joel-rieke Nov 13, 2023
fd5ff26
Add read support for projectservice.
joel-rieke Nov 13, 2023
433ed21
Fix missing import.
joel-rieke Nov 13, 2023
89ca170
Fixing couple bugs and log cleanup.
joel-rieke Nov 13, 2023
7b31c75
Revert breaking change.
joel-rieke Nov 13, 2023
a4c966f
Merge branch 'main' into develop
joel-rieke Nov 13, 2023
a6e8b5f
Revert breaking change.
joel-rieke Nov 13, 2023
5dda4a1
Provide appropriate inputs. Handle bad inputs.
joel-rieke Nov 13, 2023
4873824
Merge branch 'main' into develop
joel-rieke Nov 13, 2023
8f18026
Fix hacky logic.
joel-rieke Nov 13, 2023
d58d70e
Merge branch 'main' into develop
joel-rieke Nov 13, 2023
2b50536
Merge branch 'main' into develop
joel-rieke Nov 13, 2023
8286123
Merge branch 'main' into develop
joel-rieke Nov 13, 2023
c404f06
Merge branch 'main' into develop
joel-rieke Nov 13, 2023
92114f0
Fix so trcsh works with empty directory.
joel-rieke Nov 13, 2023
6332ee7
Merge branch 'main' into develop
joel-rieke Nov 14, 2023
77f2eba
Instrument interruptibility.
joel-rieke Nov 14, 2023
90d8e40
Adjust wait times.
joel-rieke Nov 14, 2023
bb639e9
Adding window utility function to make life easier.
joel-rieke Nov 14, 2023
0f9f91c
Fix downloading.
joel-rieke Nov 14, 2023
99c2364
Better error handling of keystore creation.
joel-rieke Nov 14, 2023
3db2d39
Honor enddir and startdir when provided.
joel-rieke Nov 14, 2023
8930981
Merge branch 'main' into develop
joel-rieke Nov 14, 2023
ef73200
Provide more concise override options.
joel-rieke Nov 14, 2023
b21676f
Merge branch 'main' into develop
joel-rieke Nov 14, 2023
3d5fe56
Prevent double completion.
joel-rieke Nov 14, 2023
7a684bf
Update version for confirmation.
joel-rieke Nov 14, 2023
9027eb4
Removing a deadlock.
joel-rieke Nov 14, 2023
0d85fe2
Increase timeout.
joel-rieke Nov 15, 2023
aad1376
Adding some retries.
joel-rieke Nov 15, 2023
3e81de5
Make re-entrant safe.
joel-rieke Nov 15, 2023
2c634c5
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
fc6b3a3
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
c2ecb30
Make this more interruptible.
joel-rieke Nov 15, 2023
0741d4e
Fixing flags and usage.
joel-rieke Nov 15, 2023
113ea28
Changes achieve the following:
joel-rieke Nov 15, 2023
4a897a1
Update trcx to use flagset as well.
joel-rieke Nov 15, 2023
83abadb
Update xp to provide proper args as well.
joel-rieke Nov 15, 2023
8c9eeb9
Fixing improper references...
joel-rieke Nov 15, 2023
b56b78b
Expose diff flag for trcctl.
joel-rieke Nov 15, 2023
5b664f3
Fixing some brokenness.
joel-rieke Nov 15, 2023
9cacd5e
Timeout wasn't issue. Reverting change.
joel-rieke Nov 15, 2023
3f9a738
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
477bd8e
Concurrency fix:
joel-rieke Nov 15, 2023
9ca2753
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
50c8b0c
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
d046ceb
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
63478c2
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
054c166
Merge branch 'main' into develop
joel-rieke Nov 15, 2023
a597193
Merge branch 'main' into develop
joel-rieke Nov 16, 2023
72540c7
Merge branch 'main' into develop
joel-rieke Nov 16, 2023
28d451e
Merge branch 'main' into develop
joel-rieke Nov 16, 2023
f024bf3
Merge branch 'main' into develop
joel-rieke Nov 16, 2023
46cdc8d
Update wording for any.
joel-rieke Nov 16, 2023
07594a6
Feathering only in lower environments.
joel-rieke Nov 16, 2023
136e368
A little cleanup and a little error handling.
joel-rieke Nov 16, 2023
9f8dd8b
Fix env selection problem in trcsh.
joel-rieke Nov 16, 2023
703fa01
Merge branch 'main' into develop
joel-rieke Nov 16, 2023
9a42850
Merge branch 'main' into develop
joel-rieke Nov 16, 2023
80f3ad3
Fix cert validation for rqa and auto.
joel-rieke Nov 20, 2023
f242203
Add support for subpath deployments on codedeploy.
joel-rieke Nov 20, 2023
2c9d74d
This should return an error, not exit.
joel-rieke Nov 21, 2023
725b2da
Only exit on failure for command driven.
joel-rieke Nov 21, 2023
394fb02
Merge branch 'main' into develop
joel-rieke Nov 21, 2023
d0a036e
Better handling of acceptable errors.
joel-rieke Nov 21, 2023
f1c7f6c
Merge branch 'main' into develop
joel-rieke Nov 21, 2023
f812e4e
Concurrency improvements. (#823)
joel-rieke Nov 22, 2023
d36ae54
Merge branch 'main' into develop
joel-rieke Nov 22, 2023
4f86bcf
Fixing a couple logic bugs.
joel-rieke Nov 22, 2023
2e185fe
Update to latest.
joel-rieke Nov 23, 2023
45f3481
Fix HTTP/2
joel-rieke Nov 23, 2023
6a3a018
Fix HTTP/2
joel-rieke Nov 23, 2023
9e3936d
Fix swift-nio-http2
joel-rieke Nov 23, 2023
8680ba0
Update for consistency
joel-rieke Nov 23, 2023
a0ed64c
More security updates.
joel-rieke Nov 23, 2023
9f4c9b6
Update to latest.
joel-rieke Nov 27, 2023
8c6b6e8
Using more concise session identifier.
joel-rieke Nov 27, 2023
09ee618
Merge branch 'main' into develop
joel-rieke Nov 28, 2023
3c99778
Log errors when it makes sense to.
joel-rieke Nov 28, 2023
c63f4e3
Small changes to provide better error handling.
joel-rieke Nov 28, 2023
04d5b98
Merge branch 'main' into develop
joel-rieke Nov 28, 2023
6f989eb
Fix for trcsh state processing and windows.
joel-rieke Nov 28, 2023
8ab8f07
A little cleanup.
joel-rieke Nov 29, 2023
45c175c
Merge branch 'main' into develop
joel-rieke Nov 29, 2023
53850bb
Merge branch 'main' into develop
joel-rieke Nov 29, 2023
d060d50
Merge branch 'main' into develop
joel-rieke Nov 30, 2023
38b0d1e
Clean up token selection.
joel-rieke Nov 30, 2023
cbca06c
Adding error messaging for incorrect project setup.
joel-rieke Nov 30, 2023
96dfaa7
Reduce logging.
joel-rieke Nov 30, 2023
d1f1080
Tweak wording.
joel-rieke Nov 30, 2023
7f335ae
Merge branch 'main' into develop
joel-rieke Nov 30, 2023
75a0dd1
Funnel errors upstream better.
joel-rieke Nov 30, 2023
01f5bd9
Make pluginName come from deployer.
joel-rieke Dec 1, 2023
9749cae
More assurance that tomcat goes down before copying.
joel-rieke Dec 1, 2023
a01146d
Fix problem with certifying.
joel-rieke Dec 1, 2023
4d8084b
Merge branch 'main' into develop
joel-rieke Dec 1, 2023
f783c09
Update to latest.
joel-rieke Dec 1, 2023
d19ccd3
Adding some assurances.
joel-rieke Dec 1, 2023
3e6128d
Add clarifying comment.
joel-rieke Dec 1, 2023
042ae8b
Merge branch 'main' into develop
joel-rieke Dec 1, 2023
bc3c425
Merge branch 'main' into develop
joel-rieke Dec 1, 2023
6812eae
Merge branch 'main' into develop
joel-rieke Dec 2, 2023
ad29d71
Some logging cleanup
joel-rieke Dec 2, 2023
af0b234
Fixing break on windows.
joel-rieke Dec 2, 2023
c0f90ca
Add more robust support for servicesWanted and filter.
joel-rieke Dec 3, 2023
599d204
Deployments should match a deployment run. Always load deployment du…
joel-rieke Dec 4, 2023
5a0892f
Merge branch 'main' into develop
joel-rieke Dec 4, 2023
ccbad5e
Merge branch 'main' into develop
joel-rieke Dec 4, 2023
073fc96
Provide service name always if defined.
joel-rieke Dec 5, 2023
b789f96
Merge branch 'main' into develop
joel-rieke Dec 5, 2023
b4c504c
Switch to something compatible with nssm
joel-rieke Dec 5, 2023
dd8d859
Merge branch 'main' into develop
joel-rieke Dec 5, 2023
d580c03
Go to combination of net stop, taskkill, and sc start.
joel-rieke Dec 5, 2023
3be30f9
Merge branch 'main' into develop
joel-rieke Dec 5, 2023
669d45d
Some mod cleanup.
joel-rieke Dec 6, 2023
d943d4f
Updating to go 1.21.5
joel-rieke Dec 6, 2023
5143b39
Merge branch 'main' into develop
joel-rieke Dec 6, 2023
f05aee6
Removing log lines
KarnveerSGill Dec 6, 2023
17fd446
Fixing extra bracket
KarnveerSGill Dec 6, 2023
8012652
Merge fix
KarnveerSGill Dec 6, 2023
84dbdec
Fixing token request
KarnveerSGill Dec 6, 2023
fb1e474
Merge branch 'main' into develop
joel-rieke Dec 6, 2023
659bc7d
Adding support in trcshell for environments like dev-1.
joel-rieke Dec 7, 2023
a332bad
Provide env overrides.
joel-rieke Dec 7, 2023
b5c2932
Merge branch 'main' into develop
joel-rieke Dec 7, 2023
d09d8f2
Support plugin config loading even when utilizing custom environments.
joel-rieke Dec 8, 2023
c7b9ccc
Merge branch 'main' into develop
joel-rieke Dec 8, 2023
2a50026
Merge branch 'main' into develop
joel-rieke Dec 8, 2023
4c2ed3b
Create SECURITY.md
joel-rieke Dec 8, 2023
5379eec
Create LICENSE
joel-rieke Dec 8, 2023
ad33b34
Delete LICENSE.txt
joel-rieke Dec 8, 2023
4075c1e
Attempting to fix serialization
KarnveerSGill Dec 8, 2023
5babaa7
Updating channel length check
KarnveerSGill Dec 8, 2023
f010cf9
Merge branch 'main' into develop
joel-rieke Dec 8, 2023
d9d5516
Merge branch 'main' into develop
joel-rieke Dec 9, 2023
45ec953
Fix locking issue.
joel-rieke Dec 11, 2023
0493f31
Roll back since not needed anymore.
joel-rieke Dec 12, 2023
1012487
Update to more concurrent safe go-mysql
joel-rieke Dec 12, 2023
25699a5
Update to latest go-mysql-server build.
joel-rieke Dec 12, 2023
93d7896
Merge branch 'main' into develop
joel-rieke Dec 12, 2023
1e79348
Adding bounds check.
joel-rieke Dec 13, 2023
97a5c56
Add actual bounds checking.
joel-rieke Dec 13, 2023
c96c8dc
Bounds check totalKeyShard
joel-rieke Dec 13, 2023
becd0f2
Disallow complex path navigation.
joel-rieke Dec 13, 2023
b50b392
Add log sanitizing.
joel-rieke Dec 13, 2023
bb8a5f2
More sanitization.
joel-rieke Dec 13, 2023
02a34e3
Directly applying recommendation from dependabot
joel-rieke Dec 13, 2023
7f4eef7
Merge branch 'main' into develop
joel-rieke Dec 13, 2023
64dbea8
Adding support for split model.
joel-rieke Dec 13, 2023
2cd5918
Re-enable dfs.
joel-rieke Dec 13, 2023
a07d776
Adding addr to modifer cache key
KarnveerSGill Dec 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions trcflow/core/context.go
Original file line number Diff line number Diff line change
Expand Up @@ -650,8 +650,8 @@ func (tfmContext *TrcFlowMachineContext) SyncTableCycle(tfContext *TrcFlowContex

// Second row here
// Not sure if necessary to copy entire ReportStatistics method
//tenantIndexPath, tenantDFSIdPath := utilcore.GetDFSPathName()
//df.FinishStatistic(tfmContext, tfContext, tfContext.GoMod, "flume", tenantIndexPath, tenantDFSIdPath, tfmContext.Config.Log, false)
tenantIndexPath, tenantDFSIdPath := utilcore.GetDFSPathName()
df.FinishStatistic(tfmContext, tfContext, tfContext.GoMod, "flume", tenantIndexPath, tenantDFSIdPath, tfmContext.Config.Log, false)

//df.FinishStatistic(tfmContext, tfContext, tfContext.GoMod, ...)
tfmContext.FlowControllerLock.Lock()
Expand Down
2 changes: 2 additions & 0 deletions trctest/maintest.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ func main() {
pluginConfig := testopts.GetTestConfig(*tokenPtr, false)
pluginConfig["address"] = ""
pluginConfig["vaddress"] = ""
pluginConfig["caddress"] = ""
pluginConfig["token"] = ""
pluginConfig["ctoken"] = ""
pluginConfig["env"] = "dev"
pluginConfig["insecure"] = true

Expand Down
46 changes: 23 additions & 23 deletions vaulthelper/kv/Modifier.go
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,8 @@ func PreCheckEnvironment(environment string) (string, string, bool, error) {
// Any errors generated in creating the client
func NewModifier(insecure bool, token string, address string, env string, regions []string, useCache bool, logger *log.Logger) (*Modifier, error) {
if useCache {
PruneCache(env, 10)
checkoutModifier, err := cachedModifierHelper(env)
PruneCache(env, address, 10)
checkoutModifier, err := cachedModifierHelper(env, address)
if err == nil && checkoutModifier != nil {
checkoutModifier.Insecure = insecure
checkoutModifier.RawEnv = env
Expand Down Expand Up @@ -142,20 +142,20 @@ func NewModifier(insecure bool, token string, address string, env string, region
return newModifier, nil
}

func checkInitModCache(env string) {
if _, ok := modifierCache[env]; !ok {
func checkInitModCache(env string, addr string) {
if _, ok := modifierCache[fmt.Sprintf("%s+%s", env, addr)]; !ok {
modifierCachLock.Lock()
modifierCache[env] = &modCache{modCount: 0, modifierChan: make(chan *Modifier, 20)}
modifierCache[fmt.Sprintf("%s+%s", env, addr)] = &modCache{modCount: 0, modifierChan: make(chan *Modifier, 20)}
modifierCachLock.Unlock()
}
}

func cachedModifierHelper(env string) (*Modifier, error) {
checkInitModCache(env)
func cachedModifierHelper(env string, addr string) (*Modifier, error) {
checkInitModCache(env, addr)

for {
select {
case checkoutModifier := <-modifierCache[env].modifierChan:
case checkoutModifier := <-modifierCache[fmt.Sprintf("%s+%s", env, addr)].modifierChan:
atomic.AddUint64(&modifierCache[env].modCount, ^uint64(0))
return checkoutModifier, nil
case <-time.After(time.Millisecond * 200):
Expand All @@ -174,32 +174,32 @@ func (m *Modifier) Release() {
}

func (m *Modifier) releaseHelper(env string) {
checkInitModCache(env)
checkInitModCache(env, m.client.Address())

// Since modifiers are re-used now, this may not be necessary or even desired for that
// matter.
// m.httpClient.CloseIdleConnections()
if modifierCache[env].modCount > 10 {
if modifierCache[fmt.Sprintf("%s+%s", env, m.client.Address())].modCount > 10 {
m.CleanCache(10)
}

atomic.AddUint64(&modifierCache[env].modCount, 1)
modifierCache[env].modifierChan <- m
atomic.AddUint64(&modifierCache[fmt.Sprintf("%s+%s", env, m.client.Address())].modCount, 1)
modifierCache[fmt.Sprintf("%s+%s", env, m.client.Address())].modifierChan <- m
}

func (m *Modifier) RemoveFromCache() {
m.CleanCache(20)
}

func cleanCacheHelper(env string, limit uint64) {
func cleanCacheHelper(env string, addr string, limit uint64) {
modifierCachLock.Lock()
if modifierCache[env].modCount > 1 {
if modifierCache[fmt.Sprintf("%s+%s", env, addr)].modCount > 1 {
emptied:
for i := uint64(0); i < limit; i++ {
select {
case mod := <-modifierCache[env].modifierChan:
case mod := <-modifierCache[fmt.Sprintf("%s+%s", env, addr)].modifierChan:
mod.Close()
atomic.AddUint64(&modifierCache[env].modCount, ^uint64(0))
atomic.AddUint64(&modifierCache[fmt.Sprintf("%s+%s", env, addr)].modCount, ^uint64(0))
default:
break emptied
}
Expand All @@ -208,11 +208,11 @@ func cleanCacheHelper(env string, limit uint64) {
modifierCachLock.Unlock()
}

func PruneCache(env string, limit uint64) {
if modifierCache != nil && modifierCache[env] != nil {
if modifierCache[env].modCount > limit {
if _, ok := modifierCache[env]; ok {
cleanCacheHelper(env, limit)
func PruneCache(env string, addr string, limit uint64) {
if modifierCache != nil && modifierCache[fmt.Sprintf("%s+%s", env, addr)] != nil {
if modifierCache[fmt.Sprintf("%s+%s", env, addr)].modCount > limit {
if _, ok := modifierCache[fmt.Sprintf("%s+%s", env, addr)]; ok {
cleanCacheHelper(env, addr, limit)
}
}
}
Expand All @@ -221,9 +221,9 @@ func PruneCache(env string, limit uint64) {
func (m *Modifier) CleanCache(limit uint64) {
m.Close()
if _, ok := modifierCache[m.Env]; ok {
cleanCacheHelper(m.Env, limit)
cleanCacheHelper(m.Env, m.client.Address(), limit)
} else {
cleanCacheHelper(m.RawEnv, limit)
cleanCacheHelper(m.RawEnv, m.client.Address(), limit)
}
}

Expand Down
Loading