Adjust to move Semantic Conventions out of OTel spec

[jsuereth]

As part of OTEP #227 we're moving the semantic conventions out of the Specification and into a new repository.

As part of this:

• We're leaving the Markdown files for semconv in the specification repository with links to the new location warnings.
• Semantic Conventions will be released with their own version number from this repository: https://github.com/open-telemetry/semantic-conventions

We may be re-architecting the structure of this repository, but we can negotiate how best to do so for the website to easily pull it in.

---

Tasks

• Semconv pages: don't warn about external links to spec #2785
• Add semantic-conventions as a submodule and get schemas from there, with one exception #2918
• Schema files should now be served from the Semantic Conventions repository #2907
• Update semantic-conventions #2982
• Only adjust semconv titles in semconv pages #2987
• [editorial] docs/rpc/json-rpc: fix title and in-page hypenation semantic-conventions#170
• Ensure every section has a README
  • [editorial] Add missing README.md for Resource Cloud Provider semconv semantic-conventions#171
• [editorial] Fix doc page titles and add Hugo front matter semantic-conventions#175
• [editorial] Rename docs files and folders: replace "_" by "-"; add check semantic-conventions#177
• Semconv: adjust doc-page titles #2984
• Update semantic-conventions to c6efa4b, and drop normalization step #2998
• Ensure that semconv pages are preview only, not in production + add watermark #3008
  • Better DRAFT watermark position for all screens #3010
• [CI] Report link-check error when external URL used for local doc page semantic-conventions#187
• Ensure that links to OTel spec from semconv pages refer to website pages (/docs/specs/otel)
  • Waiting on Bump to latest version of the specification. semantic-conventions#185
  • Adjust-pages: link to local OTel spec for main and current vers + other fixes #3012
• Update semantic-conventions to 1.21.0, publish all schemas, drop draft status, enable auto-update #3033
• Ensure that links into semconv from other specs refer to website pages:
  • Make spec links to semconv local #3037
• Add semconv version to landing page title - Update semantic-conventions to 1.21.0, publish all schemas, drop draft status, enable auto-update #3033
• Add version update to autoupdate script - Update semantic-conventions to 1.21.0, publish all schemas, drop draft status, enable auto-update #3033
• Fix links in semconv pages, and drop workaround in this repo: |(/resource/faas\.md)#function-as-a-service|$1|; - [editorial] Link directly to page, not to page's title semantic-conventions#180
• Ensure "View page source" and other links are working, esp. for index files (originally README.md files)
• Address:
  • [editorial] Spec: replace links to semantic_conventions by semconv section of OTel website  opamp-spec#150 -- until then, this causes a build warning; the warning is skipped by an entry in .warnings-skip-list.txt
  • Drop skip pattern from .warnings-skip-list.txt (after CI: check-links: enable caching and check for warnings #3026 is merged)

Other to-dos from #2908:

• Add semantic conventions to ToC / index
• Ensure semantic-conventions -> specification links are accurately internalized to the website
• Ensure specification -> semantic-conventions links are accurately internalized to the website.
• Remove specification/semantic_conventions/*.md from being pulled in OpenTelemetry specification.
• Figure out if supplementary guidelines belong on the website --> [editorial] Move supplementary-guidelines to be under docs semantic-conventions#1240

[chalin]

Note that I've closed the following PR since the semconv pages have moved

• [editorial] Normalize links in semconv/http-metrics opentelemetry-specification#3498

[jsuereth]

@chalin Let me know how best we can start moving this issue forward. Happy to have you at the semconv working group to explain what you need to us, or we can attend whatever SiG/WGs meetings you attend.

[chalin]

@jsuereth - I was OOO. Catching up now. I didn't have the time to work on this before I left. I see that you've submitted a PR. I'll take a look as soon as I can.

[chalin]

Oh, I remember that I had issues regarding the structure of the semconv repo docs. I'll write more about that soon.

[joaopgrassi]

Hi @chalin, I volunteered to take up this work on the semconv repo side. Let me know how we can best work together. I'm available on Slack as well.

[chalin]

@jsuereth @joaopgrassi - I see from the following that you'll be restructuring the spec:

• Feature Request: Restructure Semantic Convention markdown to be topical vs. by signal semantic-conventions#137

Do you want to wait for that restructuring to land before starting to publish the semconv spec on the website?

[chalin]

Hi @joaopgrassi @jsuereth: I see that part of the work has already been done in

• Fixes #2907 - Schema files are served from semantic-conventions. #2908

My suggestion would be to bring in the changes incrementally and delay publishing anything to the production server until work on open-telemetry/semantic-conventions#137 is complete. That way we can avoid having to introduce a bunch of redirect rules. WDYT?

[chalin]

In terms of bringing things in incrementally, here's a first PR:

• Add semantic-conventions as a submodule and get schemas from there, with one exception #2918

PTAL

[joaopgrassi]

Do you want to wait for that restructuring to land before starting to publish the semconv spec on the website?

Yeah I think that would be best, to avoid double work. I will be helping out in the restructuring so should be "quick"

[chalin]

Questions from #2920 (review):

• How do we get left-hand outline like the spec has for sections?

That will happen automatically once the semconv reorg lands.

• Can we change all links to specification in semconv to point at the otel.io copy of the specification?

Yes. I didn't address this yet because the semconv pages currently link directly to v1.21.0 of the OTel spec, whereas the website is now hosting v1.22.0.

• Can we change all links to semconv from the specification to point at the otel.io copy of the semconv?

Yes, once we're closer to publishing the new semconv spec, I can work on addressing this.

[jsuereth]

Thanks @chalin !

I can bump semconv to the 1.22.0 version of the specification prior to release if that helps.

Let me know if there's any of this I can help out with! I'm glad my PR helped outline something useful, should I close it at this point?

[chalin]

If you are ok with the semconv pages linking to the OTel spec on the website, regardless of the spec version specified in the link reference, then I can handle that with the adjustment script. No need to bump the version to 1.22.0.

Yes you can close the other PR.

[joaopgrassi]

Hi @chalin the restructure is now done. How should we tackle the website work now?

[chalin]

We need to wait for the following, right?

• Renamed specification and semantic_conventions directories for more clarity semantic-conventions#161

[jsuereth]

@chalin Looks like we merged that change. I'm working on something local to fix the import, but running into some issues figuring out hugo. Maybe we sync in the (EDT) morning?

[jsuereth]

@chalin Do you think enough work has been done on the Semantic Convention repository that we could cut a release tag to continue the work on the website? (Showing the version and bumping to the release version).

I'm assuming that's the case from your last PR, but wanted to confirm before we cut a release.

[chalin]

Hi @jsuereth - oh, I missed your request to sync last Friday because I ended early. I'm catching up on notifications now. I'd like to bring in the latest semconv, clean up the scripts and make sure that all is good. Cutting a release is seems reasonable.

[jsuereth]

@chalin Great. The goal is to bump the spec version and cut our 1.22.1 release of semantic conventions wed-ish timeframe. Will ping this thread when the release is cut.

[jsuereth]

The v1.12.1 tag has been cut.

[jsuereth]

FYI - With this tag, we need to start serving the 1.21.0 file under the schemas/ directory.

[chalin]

Does this mean that all schemas in that directory are to be served? If so, I'll submit a PR doing that now.

[chalin]

Does this also mean that we can publish the semconv pages to the production server?

[chalin]

@jsuereth - while there will be ongoing work to support semconv and other specs, can we close this issue considering that the bulk of the work has been done?

[chalin]

We're well beyond the original migration effort. Closing. Feel free to comment here if you'd like this to stay open.