Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres and Hibernate fixes relating to Kata #228

Merged
merged 5 commits into from
Mar 24, 2019
Merged

Postgres and Hibernate fixes relating to Kata #228

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e0c02b9
Additional fixes around Hibernate functionality for kata
shantstepanian Mar 20, 2019
e3124e4
checkstyle upgrade to avoid security hole
shantstepanian Mar 20, 2019
7e6296b
Postgres view drop use case fixes
shantstepanian Mar 20, 2019
bac766a
Additional postgres reverse engineering fixes
shantstepanian Mar 23, 2019
4e78af6
Supporting postgres in-memory db translation for HSQL
shantstepanian Mar 23, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,13 @@
### Functionality Improvements
#182: Adding Hibernate reverse-engineering API. See [ORM Integration docs](https://goldmansachs.github.io/obevo/orm-integration.html) for more details.
#221 #223 #225: Oracle reverse-engineering improvements - unicode characters, nested tables, types, comments
#227: PostgreSQL improvements for kata - reverse-engineering, in-memory databases

### Technical Improvements
#227: Upgrading checkstyle version to avoid security prompts at Github site

### Bug Fixes
#227: Fixed drop behavior of PostgreSQL for multiple views that depended on each other


## 7.0.2
Expand Down
20 changes: 20 additions & 0 deletions obevo-bom/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,26 @@
<artifactId>obevo-reladomo-util</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.goldmansachs.obevo</groupId>
<artifactId>obevo-hibernate-util</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.goldmansachs.obevo</groupId>
<artifactId>obevo-hibernate3-util</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.goldmansachs.obevo</groupId>
<artifactId>obevo-hibernate4-util</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.goldmansachs.obevo</groupId>
<artifactId>obevo-hibernate5-util</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</dependencyManagement>

Expand Down
8 changes: 4 additions & 4 deletions obevo-core/src/main/java/com/gs/obevo/api/factory/PlatformConfigReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ public String valueOf(PropertyInput propertyInput) {
return propertyInput.getFileName();
}
});
final MutableList<String> warnings = Lists.mutable.empty();
final MutableList<String> debugMessages = Lists.mutable.empty();
final MutableList<String> errors = Lists.mutable.empty();

propertiesByFileName.forEachKeyMultiValues(new Procedure2<String, Iterable<PropertyInput>>() {
Expand All @@ -157,7 +157,7 @@ public URL valueOf(PropertyInput propertyInput) {
}
}) + "]. Please ensure that priorities are distinct.");
} else if (priorities.size() > 1) {
warnings.add("File name [" + fileName + "] was found in multiple locations [" + propertyInputs.collect(new Function<PropertyInput, URL>() {
debugMessages.add("File name [" + fileName + "] was found in multiple locations [" + propertyInputs.collect(new Function<PropertyInput, URL>() {
@Override
public URL valueOf(PropertyInput propertyInput) {
return propertyInput.getPropertyFilePath();
Expand All @@ -167,8 +167,8 @@ public URL valueOf(PropertyInput propertyInput) {
}
});

if (warnings.notEmpty()) {
LOG.warn("Warnings on platform configuration file setup; please address in the future, but program will proceed:\n{}", warnings.makeString("\n"));
if (debugMessages.notEmpty()) {
LOG.debug("Debug notices on platform configuration file setup:\n{}", debugMessages.makeString("\n"));
}

if (errors.notEmpty()) {
Expand Down
49 changes: 39 additions & 10 deletions obevo-core/src/main/java/com/gs/obevo/impl/MainDeployer.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,25 @@
*/
package com.gs.obevo.impl

import com.gs.obevo.api.appdata.*
import com.gs.obevo.api.appdata.Change
import com.gs.obevo.api.appdata.ChangeIncremental
import com.gs.obevo.api.appdata.ChangeKey
import com.gs.obevo.api.appdata.ChangeRerunnable
import com.gs.obevo.api.appdata.DeployExecution
import com.gs.obevo.api.appdata.DeployExecutionImpl
import com.gs.obevo.api.appdata.DeployExecutionStatus
import com.gs.obevo.api.appdata.Environment
import com.gs.obevo.api.factory.PlatformConfiguration
import com.gs.obevo.api.platform.*
import com.gs.obevo.api.platform.ChangeAuditDao
import com.gs.obevo.api.platform.ChangeCommand
import com.gs.obevo.api.platform.CommandExecutionContext
import com.gs.obevo.api.platform.DeployExecutionDao
import com.gs.obevo.api.platform.DeployExecutionException
import com.gs.obevo.api.platform.DeployMetrics
import com.gs.obevo.api.platform.FailedChange
import com.gs.obevo.api.platform.GraphExportFormat
import com.gs.obevo.api.platform.MainDeployerArgs
import com.gs.obevo.api.platform.Platform
import com.gs.obevo.impl.graph.GraphEnricher
import com.gs.obevo.impl.graph.GraphUtil
import com.gs.obevo.impl.text.TextDependencyExtractableImpl
Expand All @@ -35,13 +51,19 @@ import org.eclipse.collections.impl.block.factory.StringFunctions
import org.eclipse.collections.impl.factory.Lists
import org.eclipse.collections.impl.factory.Sets
import org.jgrapht.DirectedGraph
import org.jgrapht.ext.*
import org.jgrapht.ext.DOTExporter
import org.jgrapht.ext.GmlExporter
import org.jgrapht.ext.GraphMLExporter
import org.jgrapht.ext.IntegerEdgeNameProvider
import org.jgrapht.ext.IntegerNameProvider
import org.jgrapht.ext.MatrixExporter
import org.jgrapht.ext.VertexNameProvider
import org.jgrapht.graph.DefaultEdge
import org.slf4j.LoggerFactory
import java.io.FileWriter
import java.io.Writer
import java.sql.Timestamp
import java.util.*
import java.util.Date
import java.util.concurrent.TimeUnit

/**
Expand Down Expand Up @@ -188,23 +210,30 @@ class MainDeployer<P : Platform, E : Environment<P>>(

// TODO refactor into separate method
if (env.platform.isDropOrderRequired) {
// In this block, we set the "dependentChanges" field on the drop objects to ensure they can be sorted for dependencies later on
val dropsToEnrich = changePairs
.filter { it.changeKey.changeType.isRerunnable && it.sourceChange == null && it.deployedChange != null }
.map { it.deployedChange!! }

val drops = dropsToEnrich.map {drop ->
val dropsByObjectName = dropsToEnrich.associateBy { env.platform.convertDbObjectName().valueOf(it.objectName) }

val dropsForTextProcessing = dropsToEnrich.map { drop ->
val sql = changeTypeBehaviorRegistry.getChangeTypeBehavior(drop.changeType).getDefinitionFromEnvironment(drop);
LOG.debug("Found the sql from the DB for dropping: {}", sql)
TextDependencyExtractableImpl(drop.objectName, sql ?: "", drop)
}

val dropDependencies = this.textDependencyExtractor.calculateDependencies(drops)

drops.forEach { it.codeDependencies = Sets.immutable.ofAll(dropDependencies.get(it)) }
val dropDependencies = this.textDependencyExtractor.calculateDependencies(dropsForTextProcessing)

val dependencyGraph = graphEnricher.createDependencyGraph(sourceChanges, deployerArgs.isRollback)
dropsForTextProcessing.forEach { it.codeDependencies = Sets.immutable.ofAll(dropDependencies.get(it)) }

dropsToEnrich.forEach { it.dependentChanges = GraphUtil.getDependencyNodes(dependencyGraph, it) }
for (drop in dropsForTextProcessing) {
drop.codeDependencies?.let { deps ->
if (deps.notEmpty()) {
drop.payload.dependentChanges = Sets.immutable.ofAll(deps.map { dropsByObjectName[it.target] })
}
}
}
}


Expand Down
2 changes: 1 addition & 1 deletion obevo-core/src/main/java/com/gs/obevo/impl/text/TextDependencyExtractable.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ interface TextDependencyExtractable {
*
* @since 6.4.0
*/
val codeDependencies: ImmutableSet<CodeDependency>
val codeDependencies: ImmutableSet<CodeDependency>?

/**
* The dependencies to exclude from the text. This is to let the user specify the false-positive dependencies that
Expand Down
2 changes: 1 addition & 1 deletion obevo-core/src/main/java/com/gs/obevo/impl/text/TextDependencyExtractableImpl.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ class TextDependencyExtractableImpl<T> (
override val contentForDependencyCalculation: String,
val payload: T
) : TextDependencyExtractable {
override var codeDependencies: ImmutableSet<CodeDependency> = Sets.immutable.empty()
override var codeDependencies: ImmutableSet<CodeDependency>? = null

override val excludeDependencies: ImmutableSet<String>
get() = Sets.immutable.empty()
Expand Down
4 changes: 4 additions & 0 deletions obevo-db-impls/obevo-db-postgresql/obevo-db-postgresql.iml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@
<orderEntry type="library" scope="RUNTIME" name="Maven: com.google.protobuf:protobuf-java:2.6.0" level="project" />
<orderEntry type="library" scope="RUNTIME" name="Maven: us.fatehi:schemacrawler-db2:14.16.04-java7" level="project" />
<orderEntry type="library" name="Maven: org.postgresql:postgresql:42.1.1.jre7" level="project" />
<orderEntry type="module" module-name="obevo-db-h2" />
<orderEntry type="library" name="Maven: com.h2database:h2:1.4.187" level="project" />
<orderEntry type="module" module-name="obevo-db-hsql" />
<orderEntry type="library" name="Maven: org.hsqldb:hsqldb:2.3.4" level="project" />
<orderEntry type="module" module-name="obevo-db-internal-test-helper" scope="TEST" />
<orderEntry type="module" module-name="obevo-db" scope="TEST" production-on-test="" />
<orderEntry type="module" module-name="obevo-db-unittest-util" scope="TEST" />
Expand Down
15 changes: 15 additions & 0 deletions obevo-db-impls/obevo-db-postgresql/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,21 @@
<artifactId>postgresql</artifactId>
</dependency>

<dependency>
<groupId>com.goldmansachs.obevo</groupId>
<artifactId>obevo-db-h2</artifactId>
<version>${project.version}</version>
<optional>true</optional>
<!-- optional for the translation code -->
</dependency>
<dependency>
<groupId>com.goldmansachs.obevo</groupId>
<artifactId>obevo-db-hsql</artifactId>
<version>${project.version}</version>
<optional>true</optional>
<!-- optional for the translation code -->
</dependency>

<dependency>
<groupId>com.goldmansachs.obevo</groupId>
<artifactId>obevo-db-internal-test-helper</artifactId>
Expand Down
Loading