RFC: Override HTTP response headers locally with DevTools

[jecfish]

Author: Wolfgang Beyer (@wolfib)
Posted: Feb 13, 2023
Status: Complete
Chrome bug tracker: https://crbug.com/1288023

The goal of this RFC is to validate the design with the community, and to solicit feedback on open questions.

Complete: This RFC is now complete. See a summary here.

Motivation

HTTP headers let the client and the server pass additional information with an HTTP request or response. This meta-information is important for a website’s functionality and security. Until now, editing HTTP headers required having access to the actual web server, which often made it difficult to quickly experiment with changes to HTTP headers.

With the new Header Overrides feature in DevTools, you can specify response headers locally. When Chrome (with DevTools open and header overrides enabled) loads a page for which header overrides are defined, Chrome behaves as if these overridden HTTP headers are coming from the remote server. This allows you to locally experiment with different values for HTTP response headers.

Goals and Non-Goals

Goals:

• Modifying existing HTTP response headers
• Adding new response headers
• Override response headers for multiple requests

Non-Goals:

• Overriding HTTP request headers
• Fully removing a response header (for now it is only possible to set its value to “”)

Try out the Prototype

The prototype is available from Chrome 111 onwards. To check your current browser version, type chrome://version in your address bar (Install Chrome Beta, Dev, or Canary if your current browser version is lower.)

Click on the Settings icon > Experiments tab > enable the Local overrides for response headers experiment.

Modify HTTP Response Headers in the Network Panel

Follow the steps below to modify HTTP Response Headers:

1. Getting started. Click on a request in the Network panel. In the Headers pane, scroll to the Response Headers section. Hover over a response header and click on the edit button.
   

2. One-Time setup. This setup process is only necessary for your first visit. Pick a folder on your local disk to store local overrides in, and allow DevTools to access this folder. If you have already set up local overrides before, these steps will be skipped and you can continue right away. Watch this video for complete setup walkthrough.
   

3. Edit an existing response header. You can now edit the value of any response header by simply clicking on it. The edited headers have a green background.
   

4. Add a custom header. Scroll to the end of the Response Headers section. Click on + Add header button to add a new header. It has a green background.
   

5. Delete or Reset a header. Delete any new custom headers you added with the trash icon button. Deleted headers will be highlighted in a red background colour. If you have made changes to an existing header, clicking the trash icon button means revert it back to its original value. DevTools does not support completely removing an existing response header for now (But you can set a header to an empty string “”). Let us know in the comments if you have any use-case for that.
   

6. Reload the page. Your edits are automatically saved. To apply your changes, simply reload the page.

Advanced: Override response headers for multiple requests in the Sources panel

You might encounter scenarios where you want to modify the response headers of multiple requests. Instead of editing the headers of each request individually, there is a way to create header overrides which automatically apply to multiple requests.

1. Getting started. Open a network request with override headers. In the Headers pane, click on the Header Overrides link in the Response Headers section to open the .headers file in the Sources panel.
   

2. Add override rules. In the .headers file, click Add override rule to add a new rule. In this example, I added a new rule that applies x-custom-header: hello to all resources under the same domain. The rule supports wildcards characters (e.g. '*' for zero or more characters, '?' for exactly one character). For example, a *.jpg applies to all resource locations which end with .jpg.
   

3. Save the changes and reload the page. You need to manually save the changes with Command/Control + S. Reload the page to apply the changes.

4. Create a new headers file manually. Alternatively, you can create a .headers file manually in the Overrides pane under the Sources panel

Example use cases

These are just a few examples of what you can do with header overrides. We would love to hear about your own use cases. Feel free to share them in the comments below.

Cross-Origin Resource Sharing (CORS) Headers

The Cross-Origin Resource Sharing standard relies on headers, which let servers describe which origins are permitted to read their responses. When those headers are missing or mis-configured, you encounter error messages such as the one in the image below.

You can add the suggested Access-Control-Allow-Origin header as an override, to see if the proposed solution is working as intended. Once you have verified this locally, you still need to upload the changes to the server, so that other users see those changes as well. Try it yourself with this CORS demo or watch this video on overriding the CORS header.

Permissions-Policy Headers

The HTTP Permissions-Policy header provides a mechanism to allow and deny the use of browser features. The screenshots below are showing an example in which a Permissions-Policy header is overridden to allow the geolocation API to be used. Try it yourself with this geolocation demo.

Cross-Origin Isolation Headers

Cross-origin isolation is a requirement for unlocking powerful features such as the SharedArrayBuffer. To make your website cross-origin isolated, COOP (Cross-Origin-Opener-Policy) and COEP (Cross-Origin-Embedder-Policy) headers need to be configured correctly. Here is an example in which iframes are not loading due to a mis-configured CORP-header. Using a header override we can confirm that setting the header value to cross-origin fixes the issue:

Questions for Discussion

In addition to general feedback, we would like to collect feedback on the following specific questions:

1. What use cases are you seeing for header overrides? While we have plenty of ideas ourselves of scenarios in which header overrides could be useful, we are very interested in knowing about your actual use cases.

2. When using header overrides, are you experiencing any bugs? Please report any bugs you are discovering when using header overrides. Since this is a completely new feature, the chances of encountering something unforeseen are higher than usual. Reporting bugs helps us improve the experience as fast as possible.

FAQ

Can I edit header names in the Network Panel?
You can only edit the header name of a header you just created with the + Add header button. After refreshing the page, you cannot edit the header name anymore. Just delete the header entirely and add a new header with the desired name.

Why does my edited header still show up even after I deleted it?
If you edit the .headers file directly and change the Apply to, you cannot delete these overrides from the Network Panel anymore. To remove these headers, go to the .headers file and make your changes there.

[pankajparashar]

Overall it looks fantastic! Looking forward to using this for some of my use-cases at work, especially during frontend development when I don't want to wait for the API changes to be ready, I can mock the response headers like cache-control etc, as per my requirement.

I haven't tried this yet, but got few questions,

1. Are the header overrides constrained to just response, or can we also override request headers?
2. Is there an equivalent of a Network request blocking drawer, which lists all the overrides, and we can individually enable/disable overrides when dealing with large number of overrides?
3. Maybe we need some kind of a flag/indicator in the Network panel (at each request level) to indicate that a particular request/response has some kind of overrides applied?
4. Would this work out of the box for GraphQL, gRPC calls?

I can give more feedback after I try this feature. Amazing work guys, as always!

[wolfib]

1. Are the header overrides constrained to just response, or can we also override request headers?
   Only response headers for now. We believe that this is the more important one of the two to start with, since client-side manipulation of response headers was not possible before. For request headers you already have some amount of control, and - even though less convenient - you can use tools such as curl/postman/insomnia to modify them and analyse the results.

2. Is there an equivalent of a Network request blocking drawer, which lists all the overrides, and we can individually enable/disable overrides when dealing with large number of overrides?
   Not exactly. You can globally enable/disable all overrides via the checkbox in the sources panel -> overrides tab. You can also get an overview of all existing override definitions there. As a workaround, instead of deleting a block of overrides and having to type everything again when recreating them, you can try modifying the 'applyTo' property to something which doesn't exist such that the overrides is still there, but doesn't apply to any actual request.

3. Maybe we need some kind of a flag/indicator in the Network panel (at each request level) to indicate that a particular request/response has some kind of overrides applied?
   Yes, this has been suggested before and is on our ToDo list (https://crbug.com/1383096).

4. Would this work out of the box for GraphQL, gRPC calls?
   I definitely hope so. I just quickly tried it on a GraphQL playground and it seemed to work as expected.
   In cases where the URL query params are relevant, you might want to go to the .headers file and add a * wildcard to the applyTo. Otherwise you'd need to define overrides for each URL+query param combo individually.

[wolfib]

3. has now been implemented by adding purple circles to the status column.

[Jxck]

I'd love to have override Request headers too.
There are many use cases requires header modification in development workflow.

• user-agent, accept-language for debug
• adding feature / debug flag for Origin Server / CDN
• trying new features (ex. AMP SXG required "AMP-Cache-Transform: google;v="1..5" in request for debug)

some cases are covered by chrome://flags or cli args etc. but modify request could solve every specific / general purpose and increase debugability & DX since devtools is handy for devs rather than restarting chrome or installing additional toos.

[wolfib]

Noted, unfortunately we probably don't have the bandwidth to go for request header overrides very soon, but receiving this feedback from multiple sides will push it higher up on our backlog.

[sachinjain024]

Hey @wolfib @jecfish, Thanks for putting this all together. I am Sachin - Founder & CEO, of Requestly - An Open Source developer tool for Intercepting & Modifying all parts of HTTP requests & responses like Headers, API responses, API request Body, URL redirects, Request throttles, Injecting scripts, Recording HTTP Sessions, etc.

Requestly has ~175K installs in Chrome Store & I'd love to offer help in building this right from my experience in interacting with these users and understanding their use cases.

As @Jxck mentioned, there are plenty of use cases for modifying HTTP request headers. I don't have complete context if there's a technical constraint/limitation to building it but Ideally, the support should be provided for modifying both HTTP request & response headers.

Here are some use cases that I know of people use modify request headers for

• Adding Authorization Headers. An example is accessing K8s dashboard by adding an Authorization header. Source
• Adding custom Request headers to enable some features (We have seen companies following this practice where you can enable a particular feature by passing a custom HTTP request header)
• Changing content-type

You can't add the Authorization request header from Postman/Insomnia because it is a part of your workflow, you are not sending this request in Isolation. You need some capability like a browser extension (like Requestly) or In-built dev tools capability.

Another thing is we should also provide the capability to remove a response header. A couple of examples

• Removing X-Frame-Options
• Removing the Content-Security-Policy header

Not sure if changing the value to empty string works in these cases though.

Another learning is we should provide an indicator of when a header is modified on the page. Even if the devtools is open, we should let the users know that some override has happened be it - Headers or files. Many users reported that once they configure any header modification in Requestly, it is very difficult to know if any request is modified in the background. We solved it by changing the extension Icon to green.

Default Requestly Icon

The icon when any rule is applied on the page

Thought to share this Idea in case we can do something like this to let people know when an override happens. Hope this is helpful and Cheers to making web development, testing & debugging more easier.

[wolfib]

Oh, looks like a great extension! I wasn't aware of it before.

Thanks for providing use cases for request header overrides. As mentioned above, we probably don't have the bandwidth to go for this very soon, but so far we were not aware of how desirable this would be.

Completely removing response headers is something we thought about, but it didn't make the cut for this first version. If it's important, it would be less effort to add than request header overrides.

Regarding the indicator, are you thinking of something that is visible from everywhere? Our plan was to add some kind of indication to the list of requests in the network panel.

[sachinjain024]

@wolfib Makes sense to start with Overriding response headers first if it is just a prioritization problem. Regarding the Indicator, we have learned that users need two things

1. Indicator (If we show at the request level in the Network tab, it should work)
2. What configuration was applied

To give you an example, when the Requestly Icon turns green we also show the list of rules that were executed on the page in the extension popup

Not sure what could be the right place but if users have defined configuration in the .headers file, not sure how easy it would be to make this easy within dev tools, probably that's why the extension ecosystem exists ;-)

[wolfib]

Thanks! My reading of this is that an indicator in the network panel would be a good place to start :-).

[jljohnson001]

Great work team 🎉! Some use cases I would have for overriding http headers:

• Mocking response data

  • Back-end is not built yet, Stress/performance testing the UI with a large N of response data

• Modifying request headers (echoing others sentiments here)

  • Working with an API that is in progress or has poor documentation, Testing changes between UI <> API (success flow, error flows, feature flags, etc).

[sachinjain024]

@ jljohnson001 Curious to know what do you mean by Stress testing the UI with a large N of response data. Do you mean like changing the actual response to a large response object and test how the frontend code handles the large response returned from server?

[tjeufoolen]

Looks promising! I was actually working on a task last week were I wanted to override the response data instead of the response headers. Hopefully this is something that is going to be supported as well in the near future! I already see a lot of potential use cases for it that would speed up development & testing a lot. 😍

[wolfib]

That sounds like something for which local overrides (https://developer.chrome.com/blog/new-in-devtools-65/#overrides) might be helpful already.

[tjeufoolen]

After playing with it for a few seconds it indeed looks like it! I was testing a REST api and this would indeed be able to override its data for me. Thanks for the informing me of the existence of this feature, I guess I just totally overlooked this feature...

[guest271314]

I couldn't get Local Overrides to work as a local server using this https://medium.com/@jmatix/using-chrome-as-a-local-web-server-af04baffd581 except for in an Incognito window.

We can, in general, override headers in an extension with chrome.declarativeNetRequest.

It is possible to use an extension with Fetch.enable to intercept fetch POST requests and respond with user-defined post data in the form of base64 with something like below.

const encoder = new TextEncoder();
chrome.debugger.onEvent.addListener(async ({ tabId }, message, params) => {
  console.log(tabId, message, params);
  if (
    message === 'Fetch.requestPaused' &&
    /^chrome-extension|ext_stream/.test(params.request.url)
  ) {
    await chrome.debugger.sendCommand({ tabId }, 'Fetch.fulfillRequest', {
      responseCode: 200,
      requestId: params.requestId,
      requestHeaders: params.request.headers,
      body: bytesArrToBase64(
        encoder.encode(
          JSON.stringify([...Uint8Array.from({ length: 1764 }, () => 255)])
        )
      ),
    });
  } else {
    await chrome.debugger.sendCommand({ tabId }, 'Fetch.continueRequest', {
      requestId: params.requestId,
    });
  }
});

The limitation is using base64 response. So we can stream with a single request. So for an indefinite stream from the server using that particular approach we have to make multiple requests and pipe the response to a single ReadableStream, something like

let controller;
const readable = new ReadableStream({
  start(_) {
    console.log(_);
    return (controller = _);
  },
  async pull(c) {
    try {
      const request = await fetch(url + '?ext_stream', {
        method: 'post',
        body: '',
        cache: 'no-store',
      });
      c.enqueue(await request.arrayBuffer());
    } catch (e) {
      console.log('catch stream', e);
      c.close();
    }
  },
  close() {
    console.log('rs close');
  },
});

readable
  .pipeThrough(new TextDecoderStream())
  .pipeThrough(
    new TransformStream({
      transform(v, c) {
        c.enqueue(new Uint8Array(JSON.parse(v)));
      },
      flush() {
        console.log('flush');
      },
    })
  )
  .pipeTo(
    new WritableStream({
      start(_) {
        console.log(_);
      },
      write(v) {
        console.log(v.length);
      },
      close() {
        console.log('close');
      },
      abort(reason) {
        console.log('WritableStream', reason);
      },
    })
  )
  .catch((e) => {
    console.log('catch pipeTo()', e);
  });
await new Promise((r) => setTimeout(r, 1000));
await Promise.allSettled([controller.close()]);

What I would prefer is a way to register an origin as a WindowClient or otherwise set an origin or specific Tab to be intercepted by the ServiceWorker onfetch handler, then we can set headers and response body directly and respond with a ReadableStream using respondWith().

[wolfib]

If I understand correctly you would like to be able to write an extension which overrides content and possibly headers for a stream. This feels like it's beyond the scope of this proposal.

[stephenwil]

Sounds a great idea, and really useful to have it directly in the browser. I've been using tools like Charles Proxy to do similar, but our development macs are getting so locked down, using third party software like Charles is becoming less possible, so having this feature will be great.
Like others, I'd use it for mocking data and the edge cases of where existing and new Apis aren't working as expected

[guest271314]

I concur. I wrote up this explainer https://github.com/guest271314/requestClient for the ability to dual opt-in of ServiceWorker and arbitrary Web sites to register the site as a WindowClient, then we can make use of ServiceWorker and respondWith() in extension or user-defined ServiceWorker to set/override both headers and respond data (e.g., for streaming a radio station to media broadcast to peers over HTTP(s) and other use cases).

[CRC32EX]

The rule supports wildcards characters

Regex support more convenient i think.

[wolfib]

Agreed. The reason for using wildcards the way we are doing it, is that we are reusing existing functionality already offered by the Chrome DevTools protocol (which DevTools is using for communicating with the Chromium backend).

[jecfish]

After several weeks of discussion, the RFC for Response Header Overrides is now closed. Here is our attempt at summarizing the feedback we received both here and on other channels (@ChromeDevTools Twitter, Mastodon, LinkedIn, in person).
We are very happy that the overall feedback has been positive and our users see header overrides as a useful feature to have. We therefore plan to move forward with our proposed design.

We have implemented these feature requests:

• Overridden Indicator. Show an indicatorIn the Network panel of which requests contain overridden headers. (https://crbug.com/1383096)
• Auto-save on edit. Save changes automatically when users edit header overrides in the sources panel. This aligned the behavior with editing in the Network panel. (https://crbug.com/1416334)

Apart from that, we would like to highlight a few nice to have, but non-blocking feature requests we received repeatedly:

• Overriding request headers. This is the most common feature request we heard. While we unfortunately do not have the resources to tackle this immediately, your collective voice will influence our future planning decisions. (https://crbug.com/1425435)
• Completely remove a header (not just setting to empty string) (https://crbug.com/1363135). If you have a use case for which this is absolutely necessary and overriding the header value with a different value or with an empty string does not work, please post on the bug.

There are other UX improvement suggestions as well, we’ll consider adding them, possibly in follow-up releases.
We would like to thank everyone for your feedback and for your help in shaping Chrome DevTools!