Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create sub directories for ThirdPartyAudit dependency metadata #16844

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Create sub directories for ThirdPartyAudit dependency metadata #16844

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f5f0ac0
Extract jars to sub dirs during thirdPartyAudit task.
finnegancarroll Dec 23, 2024
ffa445e
Change regex to split on '-'/'.'. Ignore version.
finnegancarroll Dec 24, 2024
060b6a2
Split on .jar for sub folder prefix.
finnegancarroll Dec 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 24 additions & 9 deletions buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,8 +229,7 @@
@TaskAction
public void runThirdPartyAudit() throws IOException {
Set<File> jars = getJarsToScan();

extractJars(jars);
Set<File> extractedJars = extractJars(jars);

Check warning on line 232 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L232 

Added line #L232 was not covered by tests

final String forbiddenApisOutput = runForbiddenAPIsCli();

Expand All @@ -248,7 +247,7 @@

Set<String> jdkJarHellClasses = null;
if (this.jarHellEnabled) {
jdkJarHellClasses = runJdkJarHellCheck();
jdkJarHellClasses = runJdkJarHellCheck(extractedJars);

Check warning on line 250 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L250 

Added line #L250 was not covered by tests
}

if (missingClassExcludes != null) {
Expand Down Expand Up @@ -301,16 +300,26 @@
getLogger().error("Forbidden APIs output:\n{}==end of forbidden APIs==", forbiddenApisOutput);
}

private void extractJars(Set<File> jars) {
/**
* Extract project jars to build directory as specified by getJarExpandDir.
* Handle multi release jars by keeping versions closest to `targetCompatibility` version.
* @param jars to extract to build dir
* @return File set of extracted jars
*/
private Set<File> extractJars(Set<File> jars) {
Set<File> extractedJars = new TreeSet<>();

Check warning on line 310 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L310 

Added line #L310 was not covered by tests
File jarExpandDir = getJarExpandDir();
// We need to clean up to make sure old dependencies don't linger
getProject().delete(jarExpandDir);

jars.forEach(jar -> {
String jarPrefix = jar.getName().replace(".jar", "");
File jarSubDir = new File(jarExpandDir, jarPrefix);
extractedJars.add(jarSubDir);

Check warning on line 318 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L316-L318 

Added lines #L316 - L318 were not covered by tests
FileTree jarFiles = getProject().zipTree(jar);
getProject().copy(spec -> {
spec.from(jarFiles);
spec.into(jarExpandDir);
spec.into(jarSubDir);

Check warning on line 322 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L322 

Added line #L322 was not covered by tests
// exclude classes from multi release jars
spec.exclude("META-INF/versions/**");
});
Expand All @@ -329,14 +338,16 @@
Integer.parseInt(targetCompatibility.get().getMajorVersion())
).forEach(majorVersion -> getProject().copy(spec -> {
spec.from(getProject().zipTree(jar));
spec.into(jarExpandDir);
spec.into(jarSubDir);

Check warning on line 341 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L341 

Added line #L341 was not covered by tests
String metaInfPrefix = "META-INF/versions/" + majorVersion;
spec.include(metaInfPrefix + "/**");
// Drop the version specific prefix
spec.eachFile(details -> details.setPath(details.getPath().replace(metaInfPrefix, "")));
spec.setIncludeEmptyDirs(false);
}));
});

return extractedJars;

Check warning on line 350 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L350 

Added line #L350 was not covered by tests
}

private void assertNoJarHell(Set<String> jdkJarHellClasses) {
Expand Down Expand Up @@ -398,7 +409,12 @@
return forbiddenApisOutput;
}

private Set<String> runJdkJarHellCheck() throws IOException {
/**
* Execute java with JDK_JAR_HELL_MAIN_CLASS against provided jars with OpenSearch core in the classpath.
* @param jars to scan for jarHell violations.
* @return standard out of jarHell process.
*/
private Set<String> runJdkJarHellCheck(Set<File> jars) throws IOException {
ByteArrayOutputStream standardOut = new ByteArrayOutputStream();
InjectedExecOps execOps = getProject().getObjects().newInstance(InjectedExecOps.class);
ExecResult execResult = execOps.getExecOps().javaexec(spec -> {
Expand All @@ -407,9 +423,8 @@
getRuntimeConfiguration(),
getProject().getConfigurations().getByName(CompileOnlyResolvePlugin.RESOLVEABLE_COMPILE_ONLY_CONFIGURATION_NAME)
);

spec.getMainClass().set(JDK_JAR_HELL_MAIN_CLASS);
spec.args(getJarExpandDir());
spec.args(jars);

Check warning on line 427 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L427 

Added line #L427 was not covered by tests
spec.setIgnoreExitValue(true);
if (javaHome != null) {
spec.setExecutable(javaHome + "/bin/java");
Expand Down
Loading