Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creates component registries in the OpenApiWorkspace for $ref resolution #1609

Merged
merged 37 commits into from
Apr 5, 2024
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
ce5c724
Add methods to set and retrieve components registries
irvinesunday Mar 5, 2024
94028ae
Create new property that will capture unique document ID or base Uri
irvinesunday Mar 5, 2024
e27a4e8
Update workspace
irvinesunday Mar 18, 2024
95710e5
Update reference resolver tests
irvinesunday Mar 25, 2024
76512a2
Register components in document deserializers
irvinesunday Mar 25, 2024
4bd1685
Add document to its workspace when created via parameterless constructor
irvinesunday Mar 25, 2024
5cea77e
Use document Workspace instance
irvinesunday Mar 27, 2024
351baec
Use the document Workspace instance already created in the ctor
irvinesunday Mar 27, 2024
a5af833
Don't register components directly
irvinesunday Mar 27, 2024
8acb0d2
Remove ref resolution from doc.; update ctor
irvinesunday Mar 27, 2024
2cfaf7a
Update workspace; add new methods
irvinesunday Mar 27, 2024
9bdbac6
Update tests
irvinesunday Mar 27, 2024
74e4b4d
Merge remote-tracking branch 'origin/release/2.0.0' into is/component…
irvinesunday Mar 27, 2024
dd3f13c
Add JsonSchema reference resolution to OpenApiDocument class
irvinesunday Mar 28, 2024
0e3bf94
Remove unnecessary code; revert BaseUrl value
irvinesunday Mar 28, 2024
276e5ee
Update tests
irvinesunday Mar 28, 2024
14597de
Remove unused code
irvinesunday Mar 28, 2024
ea186d8
Revert deleted code
irvinesunday Mar 28, 2024
6ebc492
Update reference test
irvinesunday Mar 29, 2024
4f09473
Update test
irvinesunday Mar 29, 2024
4e480ed
Update test file and test
irvinesunday Mar 29, 2024
1b286e5
Update models
irvinesunday Apr 2, 2024
e7ff360
Update tests
irvinesunday Apr 2, 2024
ef633cf
Update Public Api
irvinesunday Apr 2, 2024
dbfcf0a
Resolve public Api
irvinesunday Apr 2, 2024
464bd21
Register local refs within external documents with unique GUID
irvinesunday Apr 2, 2024
77d1e49
Refactor functions
irvinesunday Apr 2, 2024
9a9827a
Use unique document ids in URIs in components registry and ref resolu…
irvinesunday Apr 3, 2024
feb674f
Merge branch release/2.0.0
irvinesunday Apr 3, 2024
6954790
Remove unnecessary code
irvinesunday Apr 3, 2024
f6912ab
Resolve merge conflicts with release/2.0.0 branch
irvinesunday Apr 4, 2024
33236ad
Merge method resolving JsonSchemas with SetHostDocument
irvinesunday Apr 4, 2024
977e8c6
Update XML summary
irvinesunday Apr 4, 2024
21bed6b
Update PublicAPI
irvinesunday Apr 4, 2024
d744e64
Merge remote-tracking branch 'origin/release/2.0.0' into is/component…
irvinesunday Apr 4, 2024
54712c1
Update Public API
irvinesunday Apr 4, 2024
11b3399
Update comment
irvinesunday Apr 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 33 additions & 67 deletions src/Microsoft.OpenApi/Models/OpenApiDocument.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Json.Schema;
using Microsoft.OpenApi.Exceptions;
using Microsoft.OpenApi.Interfaces;
using Microsoft.OpenApi.Reader;
using Microsoft.OpenApi.Services;
Expand Down Expand Up @@ -94,8 +94,12 @@
/// <summary>
/// Parameter-less constructor
/// </summary>
public OpenApiDocument() { }

public OpenApiDocument()
{
Workspace = new OpenApiWorkspace();
Workspace.AddDocument("/", this);
}

/// <summary>
/// Initializes a copy of an an <see cref="OpenApiDocument"/> object
/// </summary>
Expand Down Expand Up @@ -124,7 +128,7 @@

writer.WriteStartObject();

// openApi;

Check warning on line 131 in src/Microsoft.OpenApi/Models/OpenApiDocument.cs

View workflow job for this annotation

GitHub Actions / Build

Remove this commented out code. (https://rules.sonarsource.com/csharp/RSPEC-125)
writer.WriteProperty(OpenApiConstants.OpenApi, "3.1.0");

// jsonSchemaDialect
Expand Down Expand Up @@ -478,6 +482,29 @@
return ResolveReference(reference, false);
}

/// <summary>
/// Resolves JsonSchema refs
/// </summary>
/// <param name="referenceUri"></param>
/// <returns>A JsonSchema ref.</returns>
public JsonSchema ResolveJsonSchemaReference(Uri referenceUri)
{
if (referenceUri == null) return null;

OpenApiReference reference = new OpenApiReference()
{
ExternalResource = referenceUri.OriginalString,
Id = referenceUri.OriginalString.Split('/').Last(),
Type = ReferenceType.Schema
};

JsonSchema resolvedSchema = reference.ExternalResource.StartsWith("#")
? (JsonSchema)Workspace.ResolveReference<IBaseDocument>(reference.Id, reference.Type, Components) // local ref
: Workspace.ResolveReference<JsonSchema>(reference); // external ref

return resolvedSchema;
}

/// <summary>
/// Takes in an OpenApi document instance and generates its hash value
/// </summary>
Expand Down Expand Up @@ -522,14 +549,14 @@
return null;
}

// Todo: Verify if we need to check to see if this external reference is actually targeted at this document.

Check warning on line 552 in src/Microsoft.OpenApi/Models/OpenApiDocument.cs

View workflow job for this annotation

GitHub Actions / Build

Complete the task associated to this 'TODO' comment. (https://rules.sonarsource.com/csharp/RSPEC-1135)
if (useExternal)
{
if (this.Workspace == null)
if (Workspace == null)
{
throw new ArgumentException(Properties.SRResource.WorkspaceRequredForExternalReferenceResolution);
}
return this.Workspace.ResolveReference(reference);
return Workspace.ResolveReference<IOpenApiReferenceable>(reference);
}

if (!reference.Type.HasValue)
Expand All @@ -552,68 +579,7 @@
return null;
}

if (this.Components == null)
{
throw new OpenApiException(string.Format(Properties.SRResource.InvalidReferenceId, reference.Id));
}

try
{
switch (reference.Type)
{
case ReferenceType.PathItem:
var resolvedPathItem = this.Components.PathItems[reference.Id];
resolvedPathItem.Description = reference.Description ?? resolvedPathItem.Description;
resolvedPathItem.Summary = reference.Summary ?? resolvedPathItem.Summary;
return resolvedPathItem;

case ReferenceType.Response:
var resolvedResponse = this.Components.Responses[reference.Id];
resolvedResponse.Description = reference.Description ?? resolvedResponse.Description;
return resolvedResponse;

case ReferenceType.Parameter:
var resolvedParameter = this.Components.Parameters[reference.Id];
resolvedParameter.Description = reference.Description ?? resolvedParameter.Description;
return resolvedParameter;

case ReferenceType.Example:
var resolvedExample = this.Components.Examples[reference.Id];
resolvedExample.Summary = reference.Summary ?? resolvedExample.Summary;
resolvedExample.Description = reference.Description ?? resolvedExample.Description;
return resolvedExample;

case ReferenceType.RequestBody:
var resolvedRequestBody = this.Components.RequestBodies[reference.Id];
resolvedRequestBody.Description = reference.Description ?? resolvedRequestBody.Description;
return resolvedRequestBody;

case ReferenceType.Header:
var resolvedHeader = this.Components.Headers[reference.Id];
resolvedHeader.Description = reference.Description ?? resolvedHeader.Description;
return resolvedHeader;

case ReferenceType.SecurityScheme:
var resolvedSecurityScheme = this.Components.SecuritySchemes[reference.Id];
resolvedSecurityScheme.Description = reference.Description ?? resolvedSecurityScheme.Description;
return resolvedSecurityScheme;

case ReferenceType.Link:
var resolvedLink = this.Components.Links[reference.Id];
resolvedLink.Description = reference.Description ?? resolvedLink.Description;
return resolvedLink;

case ReferenceType.Callback:
return this.Components.Callbacks[reference.Id];

default:
throw new OpenApiException(Properties.SRResource.InvalidReferenceType);
}
}
catch (KeyNotFoundException)
{
throw new OpenApiException(string.Format(Properties.SRResource.InvalidReferenceId, reference.Id));
}
return Workspace.ResolveReference<IOpenApiReferenceable>(reference.Id, reference.Type, Components);
}

/// <summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ private OpenApiCallback Target
/// <param name="hostDocument">The host OpenAPI document.</param>
/// <param name="externalResource">Optional: External resource in the reference.
/// It may be:
/// 1. a absolute/relative file path, for example: ../commons/pet.json
/// 1. an absolute/relative file path, for example: ../commons/pet.json
/// 2. a Url, for example: http://localhost/pet.json
/// </param>
public OpenApiCallbackReference(string referenceId, OpenApiDocument hostDocument, string externalResource = null)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license.

using System;
using System.Collections.Generic;
using Microsoft.OpenApi.Interfaces;
using Microsoft.OpenApi.Models;
Expand All @@ -14,9 +15,10 @@
internal class OpenApiRemoteReferenceCollector : OpenApiVisitorBase
{
private readonly Dictionary<string, OpenApiReference> _references = new();
private Guid _guid = new();

/// <summary>
/// List of external references collected from OpenApiDocument
/// List of all internal and external references collected from OpenApiDocument
/// </summary>
public IEnumerable<OpenApiReference> References
{
Expand All @@ -32,19 +34,28 @@
/// <param name="referenceable"></param>
public override void Visit(IOpenApiReferenceable referenceable)
{
AddReference(referenceable.Reference);
AddReferences(referenceable.Reference);
}

/// <summary>
/// Collect external reference
/// Collect internal and external references
/// </summary>
private void AddReference(OpenApiReference reference)
private void AddReferences(OpenApiReference reference)
{
// External refs
if (reference is {IsExternal: true} &&
!_references.ContainsKey(reference.ExternalResource))
{
_references.Add(reference.ExternalResource, reference);
}

// Local refs
if (reference is { IsExternal: false } &&
!_references.ContainsKey(reference.ReferenceV3))
{
reference.ExternalResource = _guid.ToString();
_references.Add(reference.ReferenceV3, reference);
}
}
}
}
29 changes: 20 additions & 9 deletions src/Microsoft.OpenApi/Reader/Services/OpenApiWorkspaceLoader.cs
Original file line number Diff line number Diff line change
Expand Up @@ -39,21 +39,32 @@ internal async Task<OpenApiDiagnostic> LoadAsync(OpenApiReference reference,
// Walk references
foreach (var item in referenceCollector.References)
{

// If not already in workspace, load it and process references
if (!_workspace.Contains(item.ExternalResource))
{
var input = await _loader.LoadAsync(new(item.ExternalResource, UriKind.RelativeOrAbsolute));
var result = await OpenApiDocument.LoadAsync(input, format, _readerSettings, cancellationToken);
// Merge diagnostics
if (result.OpenApiDiagnostic != null)
if (!Guid.TryParse(item.ExternalResource, out _))
{
diagnostic.AppendDiagnostic(result.OpenApiDiagnostic, item.ExternalResource);
var input = await _loader.LoadAsync(new(item.ExternalResource, UriKind.RelativeOrAbsolute));
var result = await OpenApiDocument.LoadAsync(input, format, _readerSettings, cancellationToken);
// Merge diagnostics
if (result.OpenApiDiagnostic != null)
{
diagnostic.AppendDiagnostic(result.OpenApiDiagnostic, item.ExternalResource);
}
if (result.OpenApiDocument != null)
{
var loadDiagnostic = await LoadAsync(item, result.OpenApiDocument, format, diagnostic, cancellationToken);
diagnostic = loadDiagnostic;
}
}
if (result.OpenApiDocument != null)
else // local ref in an external file, add this to the documents registry
{
var loadDiagnostic = await LoadAsync(item, result.OpenApiDocument, format, diagnostic, cancellationToken);
diagnostic = loadDiagnostic;
}
if (!_workspace.Contains(item.ExternalResource))
{
_workspace.AddDocument(reference.ExternalResource, document);
}
}
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,12 @@
MakeServers(openApidoc.Servers, openApiNode.Context, rootNode);

FixRequestBodyReferences(openApidoc);
RegisterComponentsSchemasInGlobalRegistry(openApidoc.Components?.Schemas);

// Register components
//if (openApidoc.Components != null)

Check warning on line 268 in src/Microsoft.OpenApi/Reader/V2/OpenApiDocumentDeserializer.cs

View workflow job for this annotation

GitHub Actions / Build

Remove this commented out code. (https://rules.sonarsource.com/csharp/RSPEC-125)
//{
// openApidoc.Workspace.RegisterComponents(openApidoc.BaseUri, openApidoc.Components);
//}

return openApidoc;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,6 @@ public static OpenApiComponents LoadComponents(ParseNode node)
var components = new OpenApiComponents();

ParseMap(mapNode, components, _componentsFixedFields, _componentsPatternFields);

foreach (var schema in components.Schemas)
{
var refUri = new Uri(OpenApiConstants.V3ReferenceUri + schema.Key);
SchemaRegistry.Global.Register(refUri, schema.Value);
}

return components;
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,11 @@

ParseMap(openApiNode, openApidoc, _openApiFixedFields, _openApiPatternFields);

//if (openApidoc.Components != null)

Check warning on line 54 in src/Microsoft.OpenApi/Reader/V3/OpenApiDocumentDeserializer.cs

View workflow job for this annotation

GitHub Actions / Build

Remove this commented out code. (https://rules.sonarsource.com/csharp/RSPEC-125)
//{
// openApidoc.Workspace.RegisterComponents(openApidoc);
//}

return openApidoc;
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,6 @@ public static OpenApiComponents LoadComponents(ParseNode node)

ParseMap(mapNode, components, _componentsFixedFields, _componentsPatternFields);

foreach (var schema in components.Schemas)
{
var refUri = new Uri(OpenApiConstants.V3ReferenceUri + schema.Key);
SchemaRegistry.Global.Register(refUri, schema.Value);
}

return components;
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ namespace Microsoft.OpenApi.Reader.V31
/// runtime Open API object model.
/// </summary>
internal static partial class OpenApiV31Deserializer
{
{
private static readonly FixedFieldMap<OpenApiDocument> _openApiFixedFields = new()
{
{
Expand Down
5 changes: 2 additions & 3 deletions src/Microsoft.OpenApi/Services/OpenApiReferenceResolver.cs
Original file line number Diff line number Diff line change
Expand Up @@ -251,9 +251,8 @@
/// <param name="summary">The schema's summary.</param>
/// <returns></returns>
public JsonSchema ResolveJsonSchemaReference(Uri reference, string description = null, string summary = null)
{
var refUri = $"https://registry{reference.OriginalString.Split('#').LastOrDefault()}";
var resolvedSchema = (JsonSchema)SchemaRegistry.Global.Get(new Uri(refUri));
{
var resolvedSchema = _currentDocument.ResolveJsonSchemaReference(reference);

if (resolvedSchema != null)
{
Expand Down Expand Up @@ -410,7 +409,7 @@
}
// The concept of merging references with their target at load time is going away in the next major version
// External references will not support this approach.
//else if (_resolveRemoteReferences == true)

Check warning on line 412 in src/Microsoft.OpenApi/Services/OpenApiReferenceResolver.cs

View workflow job for this annotation

GitHub Actions / Build

Remove this commented out code. (https://rules.sonarsource.com/csharp/RSPEC-125)
//{
// if (_currentDocument.Workspace == null)
// {
Expand All @@ -424,7 +423,7 @@
// }
// var target = _currentDocument.Workspace.ResolveReference(reference);

// // TODO: If it is a document fragment, then we should resolve it within the current context

Check warning on line 426 in src/Microsoft.OpenApi/Services/OpenApiReferenceResolver.cs

View workflow job for this annotation

GitHub Actions / Build

Complete the task associated to this 'TODO' comment. (https://rules.sonarsource.com/csharp/RSPEC-1135)

// return target as T;
//}
Expand Down
Loading
Loading