Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scripts to publish javadoc and doc to cloud-rad #1317

Merged
merged 9 commits into from
Nov 10, 2022
Merged

scripts to publish javadoc and doc to cloud-rad #1317

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6bf966a
add kokoro script to upload docs.
zhumin8 Oct 26, 2022
457f07a
add docfx profile to pom for doc upload.
zhumin8 Oct 27, 2022
ea94229
fix: do not remove snapshot version changelog, instead only remove ta…
zhumin8 Nov 1, 2022
739c372
try adding blank lines after tables.
zhumin8 Nov 4, 2022
51e316c
replace table wit md format.
zhumin8 Nov 7, 2022
db6ee6e
replace table with md format.
zhumin8 Nov 7, 2022
9b4f960
use requirement.txt for docuploader.
zhumin8 Nov 9, 2022
a8e0357
fix dir location.
zhumin8 Nov 9, 2022
564c06a
try with python 3.9.5
zhumin8 Nov 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .kokoro/publish_javadoc11.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Format: //devtools/kokoro/config/proto/build.proto

build_file: "spring-cloud-gcp/.kokoro/publish_javadoc11.sh"

# cloud-rad dev
env_vars: {
key: "STAGING_BUCKET_V2"
value: "docs-staging-v2-dev"
}

before_action {
fetch_keystore {
keystore_resource {
keystore_config_id: 73713
keyname: "docuploader_service_account"
}
}
}
elefeint marked this conversation as resolved.
Show resolved Hide resolved

# Downloads docfx doclet resource. This will be in ${KOKORO_GFILE_DIR}/<doclet name>
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/docfx"
133 changes: 133 additions & 0 deletions .kokoro/publish_javadoc11.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
#!/bin/bash
set -eov pipefail

if [[ -z "${CREDENTIALS}" ]]; then
CREDENTIALS=${KOKORO_KEYSTORE_DIR}/73713_docuploader_service_account
fi

if [[ -z "${STAGING_BUCKET_V2}" ]]; then
echo "Need to set STAGING_BUCKET_V2 environment variable"
exit 1
fi

# switch to java 11
sudo update-java-alternatives --set java-1.11.0-openjdk-amd64
$JAVA_HOME/bin/javac -version
export JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64
$JAVA_HOME/bin/javac -version

pyenv global 3.7.2

# install pandoc for combining md files
sudo apt install -q -y pandoc

# Install docuploader package
python3 -m pip install --upgrade six
python3 -m pip install --upgrade protobuf
python3 -m pip install gcp-docuploader==0.6.2

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The public_javadoc scripts in googleapis/ were recently migrated to use a requirements.txt file with hash checking to securely install dependencies. Maybe we could do the same here? Here's an example: googleapis/synthtool#1588

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated with changes. Test run succeeded.
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

python3 -m docuploader --version

# Get into the spring-cloud-gcp repo directory
dir=$(dirname "$0")
pushd $dir/../

# change to release version
./mvnw versions:set --batch-mode -DremoveSnapshot -DprocessAllModules

# Compute the project version.
PROJECT_VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout)
echo ${PROJECT_VERSION}

# Build the javadocs
./mvnw clean javadoc:aggregate -Drelease=true -P docFX

# remove the trailing `-SNAPSHOT` for the current version
sed -i 's/-SNAPSHOT$//' CHANGELOG.md

# print 20 lines to verify
head -20 CHANGELOG.md
# end remove BUILD-SNAPSHOT contents from changelog

# copy CHANGELOG
cp CHANGELOG.md target/docfx-yml/history.md

# combine all doc to documentation.md
sudo pandoc \
docs/src/main/md/first-page.md \
docs/src/main/md/getting-started.md \
docs/src/main/md/core.md \
docs/src/main/md/storage.md \
docs/src/main/md/sql.md \
docs/src/main/md/pubsub.md \
docs/src/main/md/spring-integration.md \
docs/src/main/md/spring-stream.md \
docs/src/main/md/spring-cloud-bus-pubsub.md\
docs/src/main/md/trace.md \
docs/src/main/md/logging.md \
docs/src/main/md/metrics.md \
docs/src/main/md/spanner.md \
docs/src/main/md/datastore.md \
docs/src/main/md/firestore.md \
docs/src/main/md/memorystore.md \
docs/src/main/md/bigquery.md \
docs/src/main/md/security-iap.md \
docs/src/main/md/vision.md \
docs/src/main/md/secretmanager.md \
docs/src/main/md/kms.md \
docs/src/main/md/config.md \
docs/src/main/md/cloudfoundry.md \
docs/src/main/md/kotlin.md \
docs/src/main/md/configuration.md \
docs/src/main/md/migration-guide-1.x.md \
-t markdown_github -o docs/src/main/md/documentation.md

# copy and replace {project-version} documentation
sed "s/{project-version}/${PROJECT_VERSION}/g" docs/src/main/md/documentation.md > target/docfx-yml/documentation.md
# copy appendix.md
cp docs/src/main/md/appendix.md target/docfx-yml/appendix.md

# check change to documentation.md -- remove after verified
head -20 target/docfx-yml/documentation.md

# Move into generated yml directory
pushd target/docfx-yml

### manual changes to generated toc.yml,
# group all javadocs to dir and add documentation to fron

# insert after line starting with
function insertAfter
{
local file="$1" line="$2" newText="$3"
sed -i -e "/^$line$/a"$'\\\n'"$newText"$'\n' "$file"
}
# From line 4-2000, add 2 spaces to the front.
sed -i '4,2000 s/^/ /' toc.yml
# Add Javadocs dir
insertAfter toc.yml \
" items:" " - name: \"JavaDocs\"\n items:"
# add changelog to toc
insertAfter toc.yml \
" items:" " - name: \"Version history\"\n href: \"history.md\""
# add documentation.md to toc (after the first ` items:`)
insertAfter toc.yml \
" items:" " - name: \"Documentation\"\n href: \"documentation.md\""

# check change to toc.yml -- remove after verified
head -20 toc.yml

python3 -m docuploader create-metadata \
--name spring-cloud-gcp \
--version ${PROJECT_VERSION} \
--language java \
--stem "/java/docs/spring/reference"


python3 -m docuploader upload . \
--credentials ${CREDENTIALS} \
--staging-bucket ${STAGING_BUCKET_V2}\
--destination-prefix docfx

popd
popd
71 changes: 16 additions & 55 deletions docs/src/main/md/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,61 +132,20 @@ Spring Cloud GCP provides sample applications which demonstrate how to
use every integration in the library. The table below highlights several
samples of the most commonly used integrations in Spring Cloud GCP.

<table>
<colgroup>
<col style="width: 50%" />
<col style="width: 50%" />
</colgroup>
<thead>
<tr class="header">
<th>GCP Integration</th>
<th>Sample Application</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><p>Cloud Pub/Sub</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-sample">spring-cloud-gcp-pubsub-sample</a></p></td>
</tr>
<tr class="even">
<td><p>Cloud Spanner</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-data-spanner-repository-sample">spring-cloud-gcp-data-spanner-repository-sample</a></p>
<p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-data-spanner-template-sample">spring-cloud-gcp-data-spanner-template-sample</a></p></td>
</tr>
<tr class="odd">
<td><p>Datastore</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-data-datastore-sample">spring-cloud-gcp-data-datastore-sample</a></p></td>
</tr>
<tr class="even">
<td><p>Cloud SQL (w/ MySQL)</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-sql-mysql-sample">spring-cloud-gcp-sql-mysql-sample</a></p></td>
</tr>
<tr class="odd">
<td><p>Cloud Storage</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-storage-resource-sample">spring-cloud-gcp-storage-resource-sample</a></p></td>
</tr>
<tr class="even">
<td><p>Cloud Logging</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-logging-sample">spring-cloud-gcp-logging-sample</a></p></td>
</tr>
<tr class="odd">
<td><p>Trace</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-trace-sample">spring-cloud-gcp-trace-sample</a></p></td>
</tr>
<tr class="even">
<td><p>Cloud Vision</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-vision-api-sample">spring-cloud-gcp-vision-api-sample</a></p></td>
</tr>
<tr class="odd">
<td><p>Cloud Security - IAP</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-security-iap-sample">spring-cloud-gcp-security-iap-sample</a></p></td>
</tr>
<tr class="even">
<td><p>Cloud Security - Firebase</p></td>
<td><p><a href="https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-security-firebase-sample">spring-cloud-gcp-security-firebase-sample</a></p></td>
</tr>
</tbody>
</table>
| GCP Integration | Sample Application |
|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Cloud Pub/Sub | [spring-cloud-gcp-pubsub-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-sample) |
| Cloud Spanner | [spring-cloud-gcp-data-spanner-repository-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-data-spanner-repository-sample) <br/> [spring-cloud-gcp-data-spanner-template-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-data-spanner-template-sample) |
| Datastore | [spring-cloud-gcp-data-datastore-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-data-datastore-sample) |
| Cloud SQL (w/ MySQL) | [spring-cloud-gcp-sql-mysql-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-sql-mysql-sample) |
| Cloud Storage | [spring-cloud-gcp-storage-resource-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-storage-resource-sample) |
| Cloud Logging | [spring-cloud-gcp-logging-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-logging-sample) |
| Trace | [spring-cloud-gcp-trace-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-trace-sample) |
| Cloud Vision | [spring-cloud-gcp-vision-api-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-vision-api-sample) |
| Cloud Security - IAP | [spring-cloud-gcp-security-iap-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-security-iap-sample) |
| Cloud Security - Firebase | [spring-cloud-gcp-security-firebase-sample](https://github.com/GoogleCloudPlatform/spring-cloud-gcp/tree/main/spring-cloud-gcp-samples/spring-cloud-gcp-security-firebase-sample) |



Each sample application demonstrates how to use Spring Cloud GCP
libraries in context and how to setup the dependencies for the project.
Expand All @@ -197,6 +156,8 @@ AppEngine](https://codelabs.developers.google.com/codelabs/cloud-app-engine-spri
and [to Google Kubernetes
Engine](https://codelabs.developers.google.com/codelabs/cloud-springboot-kubernetes/index.html).



#### Codelabs

For a more hands-on approach, there are several guides and codelabs to
Expand Down
Loading