Fix tensor parallelism

[mreso]

Description

This PR fixes tensor parallelism which was broken due to backend caching mechanism. It also introduces a simple test to detect this kind of issue in the future.

Fixes #(issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?

[codecov]

Codecov Report

Merging #2741 (81d994c) into master (f3d3bb3) will not change coverage.
Report is 1 commits behind head on master.
The diff coverage is n/a.

❗ Current head 81d994c differs from pull request most recent head 749e41a. Consider uploading reports for the commit 749e41a to get more accurate results

@@           Coverage Diff           @@
##           master    #2741   +/-   ##
=======================================
  Coverage   72.44%   72.44%           
=======================================
  Files          85       85           
  Lines        3963     3963           
  Branches       58       58           
=======================================
  Hits         2871     2871           
  Misses       1088     1088           
  Partials        4        4           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more