Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

img decoding="async" with blazor #47449

Closed
wrharper-AASP opened this issue Mar 27, 2023 · 43 comments
Closed

img decoding="async" with blazor #47449

wrharper-AASP opened this issue Mar 27, 2023 · 43 comments
Labels
area-blazor Includes: Blazor, Razor Components ✔️ Resolution: Answered Resolved because the question asked by the original author has been answered. question Status: Resolved

Comments

@wrharper-AASP
Copy link

When doing async with img src for a base64 string I get a deadlock. I have to use configureawait(false) on all tasks and according to sourceforge and sites like it, everyone is doing this to get around it.

I have recently learned about (decoding="async") inside of the img html object. Is it possible to use this in some way so that you can truly get images with async? this would be a huge performance increase for the project I am doing.

I use a custom REST API I have created to get the images and they come in as base64 strings.

@ghost ghost added the untriaged label Mar 27, 2023
@wrharper-AASP
Copy link
Author

I am a bit confused about who supports what. This project says asp.net core does blazor, when going to asp.net core, it links to this. There appears to be a loop on who does what? If this is the wrong place, let me know

@davidwengier davidwengier transferred this issue from dotnet/razor Mar 27, 2023
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-blazor Includes: Blazor, Razor Components label Mar 27, 2023
@javiercn
Copy link
Member

javiercn commented Mar 28, 2023

@wrharper-AASP thanks for contacting us.

I am a bit confused about who supports what. This project says asp.net core does Blazor, when going to asp.net core, it links to this. There appears to be a loop on who does what? If this is the wrong place, let me know.

This repo is for Blazor, which is part of ASP.NET Core. Hope that helps clarify things.

When doing async with img src for a base64 string I get a deadlock. I have to use configureawait(false) on all tasks and according to sourceforge and sites like it, everyone is doing this to get around it.

I have recently learned about (decoding="async") inside of the img html object. Is it possible to use this in some way so that you can truly get images with async? this would be a huge performance increase for the project I am doing.

I use a custom REST API I have created to get the images and they come in as base64 strings.

I imagine you are doing something like <img src="{{base64Url}}" />?

First of all, we recommend against that. You are allocating very large strings that the system needs to hold on to, and that is going to perform badly.

If you already have data on an API, we suggest you avoid base64 URL encoding the image altogether, serve the bits raw from the endpoint and reference the endpoint in the link tag.

@wrharper-AASP
Copy link
Author

@wrharper-AASP thanks for contacting us.

I am a bit confused about who supports what. This project says asp.net core does Blazor, when going to asp.net core, it links to this. There appears to be a loop on who does what? If this is the wrong place, let me know.

This repo is for Blazor, which is part of ASP.NET Core. Hope that helps clarify things.

When doing async with img src for a base64 string I get a deadlock. I have to use configureawait(false) on all tasks and according to sourceforge and sites like it, everyone is doing this to get around it.
I have recently learned about (decoding="async") inside of the img html object. Is it possible to use this in some way so that you can truly get images with async? this would be a huge performance increase for the project I am doing.
I use a custom REST API I have created to get the images and they come in as base64 strings.

I imagine you are doing something like <img src="{{base64Url}}" />?

First of all, we recommend against that. You are allocating very large strings that the system needs to hold on to, and that is going to perform badly.

If you already have data on an API, we suggest you avoid base64 URL encoding the image altogether, serve the bits raw from the endpoint and reference the endpoint in the link tag.

I actually can't in my case. I can't really explain why without exposing too much of my goals, but it comes down to the useragent. The images have to be a specific useragent to be viewed by the URL so that is why they are converted to base64.

@wrharper-AASP
Copy link
Author

The other Idea I have had has been creating a List, Stack, or Queue cache, but that would increase memory usage tremendously if tons of people use the site. It can't be stored on the client for compliance reasons either.

@javiercn
Copy link
Member

javiercn commented Mar 28, 2023

The other Idea I have had has been creating a List, Stack, or Queue cache, but that would increase memory usage tremendously if tons of people use the site. It can't be stored on the client for compliance reasons either.

The images are already "stored" in the client if they are being rendered in a browser.

The best you can do is something like this which is what we recommend doing.

I actually can't in my case. I can't really explain why without exposing too much of my goals, but it comes down to the useragent. The images have to be a specific useragent to be viewed by the URL so that is why they are converted to base64.

Not sure what the reason is for this, but I would say that relying on the useragent for any type of access restriction is not correct, as it's really easy for any app to change it/fake it. You'd be better of using some sort of authorization to prevent unauthorized access to the resource.

You can prevent the browser from caching anything by setting the appropriate response headers.

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

The images are already "stored" in the client if they are being rendered in a browser.

The best you can do is something like this which is what we recommend doing.

Sort of, but it isn't the same as a directly stored .jpg. It's just a string in the code. It's not as easy for those common guys out there and I have found no other working alternative yet.

Not sure what the reason is for this, but I would say that relying on the useragent for any type of access restriction is not correct, as it's really easy for any app to change it/fake it. You'd be better of using some sort of authorization to prevent unauthorized access to the resource.

You can prevent the browser from caching anything by setting the appropriate response headers.

Oh, it does Auth as well. I don't know why they designed it that way and it isn't something I can change unfortunately since it's not on our end. I couldn't agree more though.

@javiercn
Copy link
Member

Sort of, but it isn't the same as a directly stored .jpg. It's just a string in the code. It's not as easy for those common guys out there and I have found no other working alternative yet.

It's not clear what you mean by this. If it gets downloaded into "a browser" it's trivial for someone to save the image. If the image is displayed on a client machine (browser allegedly) the bits are already there.

As to "holding" on to the image in memory as a string, it's easy to do the math, and if each image you have is 5 mb, with 100 you are getting 500mb. If this is Blazor Server, then each user that does that, takes 500mb of your server memory.

It's not much better in webassembly either, other than the fact that the memory is on the client machine.

The general issue here is that allocating large blobs of string data is going to wreck the performance of your app, as the GC automatically puts the strings in the LOH heap, which means they wont be garbage collected unless the system is pretty much running out of memory.

Not to mention that if you use those strings directly within Blazor, we must hold a reference to them continuously up until you decide to stop rendering them.

If this is the approach you are taking, there's nothing we can do about it. This is going to perform badly pretty much in every framework, that's why blob urls exist for.

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

ok, the last thing I can think of is adding the useragent somehow to all clients. Is that possible?

Also, yes, the user can always just use something like the snipping tool, but someone who has access to another person's drive would have no way to get the image?

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

Based on your solution, it looks like it tries to use a button for the async but this needs to be done automatically in a for loop:

if (GetMedia(i).Result) <--- works
                            {
                                <img id=@SetId(i) />
                                @SetImageAsync(i).Result <--- won't work even with configure false on everything.
                            }

I set it the following way:

    async Task<Stream> GetImageStreamAsync(int i)
    {
        return await Http.GetStreamAsync(Info.MessageInfo[i].mediaid).ConfigureAwait(false);
    }

    async Task<string> SetImageAsync(int i)
    {
        var imageStream = await GetImageStreamAsync(i).ConfigureAwait(false);
        var dotnetImageStream = new DotNetStreamReference(imageStream);
        await JsRuntime.InvokeVoidAsync("setImage", SelectedI.ToString() + "_" + i.ToString(), dotnetImageStream).ConfigureAwait(false);
        return "";
    }

The return empty string is just a trick so it will process the info even though it is a asynced task.
I added the user agent into the program.cs so that it would be part of the standard injection:
builder.Services.AddHttpClient("WebClient", client => { client.DefaultRequestHeaders.UserAgent.ParseAdd("CUSTOMAGENT"); client.Timeout = TimeSpan.FromSeconds(600); });

@javiercn
Copy link
Member

@wrharper-AASP thanks for the additional details.

Rendering in Blazor is synchronous, it does not work because the result from your task is being ignored (It's really hard to tell what you are exactly doing without the concrete snippets).

You need to load any data you need within OnInitializedAsync. It will render twice, the first time before any async work happens, and a second time after it has completed, and at that point you can use the data to render

foreach(var image in Images)
{
   <img src="" ... />
}

override OnInitializedAsync()
{
    Images = await LoadImages();
}

@wrharper-AASP
Copy link
Author

right, that's what the configureawait(false) is for and using .Result. it basically makes it synchronous. There is a timer where new images are checked, and a refresh will need to happen if new images need to be added. It can't just be ran one time and that's it. It needs to be ran on every timer check when something new is detected. So OnInitializedAsync won't work and this is in a for loop.

@wrharper-AASP
Copy link
Author

I have been using .Result to bypass these problems for many things and it works great, but for some reason it deadlocks with the solution you mentioned.

@javiercn
Copy link
Member

@wrharper-AASP You can't do that within Blazor.

Blazor uses a synchronizationcontext and by doing .ConfigureAwait your callbacks won't run within it, which is sure to cause problems. ConfigureAwait is for libraries, not for app code.

@wrharper-AASP
Copy link
Author

@wrharper-AASP You can't do that within Blazor.

Blazor uses a synchronizationcontext and by doing .ConfigureAwait your callbacks won't run within it, which is sure to cause problems. ConfigureAwait is for libraries, not for app code.

https://stackoverflow.com/questions/67516887/blazor-server-side-async-deadlock

@javiercn
Copy link
Member

@wrharper-AASP that question/answer is wrong.

Not sure what you are trying to imply with that stackoverflow question/response, but the only comment from Stephen Cleary is correct. Do not block on async code.

The way you are trying to approach a solution is in direct conflict with how the framework works.

Rendering is synchronous, and if you want to do async work, you need to use a lifecycle method and do it there. Blocking the rendering threads will cause deadlocks at worst and thread starvation at best.

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

@wrharper-AASP that question/answer is wrong.

Not sure what you are trying to imply with that stackoverflow question/response, but the only comment from Stephen Cleary is correct. Do not block on async code.

The way you are trying to approach a solution is in direct conflict with how the framework works.

Rendering is synchronous, and if you want to do async work, you need to use a lifecycle method and do it there. Blocking the rendering threads will cause deadlocks at worst and thread starvation at best.

There have been tons of stackoverflows that say the same thing, that was just one example. How would you do a lifecycle to make it possible then?
Keep in mind, this needs to be able to be updated during new changes being detected. So, it isn't a (one and done) thing.

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

Ok I did the proper lifecycle, but it will not work due to the useragent issue. The blob just shows up as broken images.

I do OnParam set to set the values (before the render) for the HTML and then OnAfterRender without first render so it keeps happening for setting the ID (after the render).

@javiercn
Copy link
Member

@wrharper-AASP there is a complete sample in the link I provided https://learn.microsoft.com/en-us/aspnet/core/blazor/images?view=aspnetcore-7.0#stream-image-data

The only thing you need to do is define the setImage function within your app, so that it's reachable via JS interop

@wrharper-AASP
Copy link
Author

It isn't valid because you don't use an async button push. It should be happening on load and update on changes. No buttons.

@wrharper-AASP
Copy link
Author

also, i have it working by doing what i mentioned, but the images are still broken:
image

@javiercn
Copy link
Member

@wrharper-AASP change SetImageAsync and instead override OnInitializedAsync() to load the data and then do the JS interop call after the data has arrived.

You can use a boolean flag to know when the data is ready, and have a check on the render to avoid rendering anything until all the data is there

@javiercn
Copy link
Member

That src attribute looks wrong. How did you create that

@wrharper-AASP
Copy link
Author

With the solution you gave, it creates that.

@javiercn
Copy link
Member

@wrharper-AASP what do the browser console logs say?

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

The HTML is just <img id=@images[i] /> which is basically the same thing as the example setting an id. The src is generated from await JsRuntime.InvokeVoidAsync("setImage", SelectedI.ToString() + "_" + i.ToString(), dotnetImageStream); everything is like the example shows just with my values instead.

@wrharper-AASP
Copy link
Author

@wrharper-AASP what do the browser console logs say?

nothing because there is actually no errors.

@wrharper-AASP
Copy link
Author

I have tried this 2 ways and both results are the same thing.

  1. converting base64 to a memory stream byte array
  2. tried to mimic the custom user agent with the httpclient the example shows.

@wrharper-AASP
Copy link
Author

I think this comes down to an auth issue. Because I cannot get that image without also doing Auth, I can't do Auth without exposing a key that the client side should never see. This is another reason why I used base64.

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

Again, with OnInitialized, this cannot be used because it only runs once. These values and this component will change constantly, it cannot be used due to update changes.

image

@wrharper-AASP
Copy link
Author

What really needs to exist is a proper before and after. That doesn't seem to be possible in blazor. What I used was OnParametersSet and then OnAfterRender

image
image

@javiercn
Copy link
Member

@wrharper-AASP here is a working sample https://github.com/javiercn/ImageBlobSample

@wrharper-AASP
Copy link
Author

You can't use the direct https:// without exposing the key. That was the whole point of base64 to begin with. Too much security is exposed. Your examples involves commonly exposed public images. That isn't the case...

@javiercn
Copy link
Member

@wrharper-AASP The code is running on the server, where you get to do whatever you want.

You can replace the code that I used to get the image with your own code that does whatever it needs.

@wrharper-AASP
Copy link
Author

No, not without the client seeing this:

private async Task<DotNetStreamReference> GetImage(int i)
    {
        var client = new HttpClient();
client.DefaultRequestHeaders.Add("Authorization", SHOULDNOTBESEEN);
        var stream = await client.GetStreamAsync($"https://dummyimage.com/450x2{i}0/f00/fff");
        return new DotNetStreamReference(stream);
    }

@javiercn
Copy link
Member

@wrharper-AASP the c# code in Blazor Server runs on the server. Only the JS code runs on the browser.

@javiercn
Copy link
Member

@wrharper-AASP
Copy link
Author

it is more complicated than that. I can't explain. I cannot hide that key if that is on the server. We are working backwards further and further; this will not be a solution.

@javiercn
Copy link
Member

@wrharper-AASP there doesn't seem to be a framework issue here.

It's not clear to us what problem you have, and we do not have enough details to offer a meaningful suggestion beyond the sample we provided.

Unless you want to provide a publicly available minimal repro project (not your codebase, something equivalent that replicates the problem) we can't be of further assistance.

While we try our best to help everyone that comes with questions, this tracker is dedicated to product issues and it seems you have an architectural/layering problem to solve. We recommend you post a question on Stackoverflow with the aspnetcore-blazor tag to see if other members on the community can step up to give you guidance.

@wrharper-AASP
Copy link
Author

wrharper-AASP commented Mar 28, 2023

I have had to create a custom REST API for many reasons, and this is one of them. The images cannot be stored anywhere else due to compliance, and they cannot be seen without a specific key and that key would need to be dynamically distributed if it's on the server. It is more complicated than what is being shown. It is clear that it is just not possible at this point and base64 is the only option.

@wrharper-AASP
Copy link
Author

I have found a way to make the reference in a way that works by converting the base64 and using your solution. Will this help performance?
DotNetStreamReference dotnetstream = new DotNetStreamReference(new MemoryStream(Convert.FromBase64String(base64string));

@wrharper-AASP wrharper-AASP reopened this Mar 29, 2023
@wrharper-AASP
Copy link
Author

Thank you, it does look like this is actually resolved with the resolution you provided now. It was just a matter of going about it a different way for the stream.

@likquietly
Copy link

likquietly commented Apr 4, 2023

here is a working sample https://github.com/javiercn/ImageBlobSample

@javiercn, great sample, but how to see the image arriving progressively?
Do you recommend stream image using JS interop instead of placing the image src at URL(img src="api/image/@imageNo") with api method like this at Blazor Server App:

[HttpGet("images/{imageNo:int}")]
public async Task<IActionResult> GetImageAsync(int imageNo)
{
var path = $"C:\\image_{imageNo}";
var data = await System.IO.File.ReadAllBytesAsync(path);

return File(data, "image/jpeg");
}

@wrharper-AASP
Copy link
Author

I recommend doing ID instead of SRC like the original solution shows with a JS invoke:
https://learn.microsoft.com/en-us/aspnet/core/blazor/hosting-models?view=aspnetcore-7.0
His concept/idea is right though.

@ghost ghost locked as resolved and limited conversation to collaborators May 4, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-blazor Includes: Blazor, Razor Components ✔️ Resolution: Answered Resolved because the question asked by the original author has been answered. question Status: Resolved
Projects
None yet
Development

No branches or pull requests

3 participants