Improve our approach for testing auth (part 1) #9681
Conversation
Change auth tests to include all shields of the base class. The code is formated to be used in more general cases and increases code reuseability.
|
Here is an example for auth test of stackexchange as a case study for the rest of the project. |
jNullj
added
core
|
Sorry I haven't managed to look at this one yet. I will make sure to follow this one up over the weekend. |
There was a problem hiding this comment.
Thanks for working on this. I've left some comments. I think having seen this implementation, I'm now actually leaning towards the second option I mentioned in #9493 (comment) but it is really useful to see what you've done here.
We already test all existing classes, no need for a dummy
Add getBadgeExampleCall to extract the first OpenAPI example then reformat it for service invoke function.
Add the testAuth function which tests auth of a service (badge) using a provided dummy response.
|
I did everything we talked about so far in the latest commits but i do have a third option to consider.
The only catch is that I will have to find a way to craft dummy responses automatically. Do you think that's over-complicating or should i give that a try? Would that fit well with the project style and code philosophy? |
services/test-helpers.js
Outdated
|
|
||
| cleanUpNockAfterEach() | ||
|
|
||
| const config = { private: { stackapps_api_key: 'fake-key' } } |
There was a problem hiding this comment.
This is specific to the stackexchange badges. If this is going to be a completely generic helper we can use for all services, we'll need to pass this into testAuth as a param. I'd suggest we make the config object the first or second param to testAuth(). This one is small, so lets address this one before merging.
There was a problem hiding this comment.
So i went a bit overboard and just made everything generic and support all auth methods.
So now i have a function to generate a fake config object based on the service class.
See ffc7800
services/test-helpers.js
Outdated
|
|
||
| const scope = nock(authOrigin) | ||
| .get(/.*/) | ||
| .query(queryObject => queryObject.key === 'fake-key') |
There was a problem hiding this comment.
Again, this is a bit specific to this service. This will work for services that pass auth in the query string, but won't for services where we pass auth in a header, for example. We're probably going to need a few different variants of this function for different auth methods (basic auth, header, query string, etc). I don't think that will be too hard - there's a few different ways we can do this.
I think I'd be fine just merging this how it is and saying at the moment it only works for querystring auth. Then we work out how to generalise this as we expand more services to use this helper. I usually find it easier to think about this stuff once I have a concrete problem I am trying to apply it to.
There was a problem hiding this comment.
See ffc7800, its all more generic now and should handle most service classes
|
Nice. Latest iteration of this looks cool.
What you've suggested would work, but I also think the latest iteration of this PR looks great. Feels like the direction you are already heading is a big improvement. As you note, making a valid example of a fake response in a generic way could be non-trivial for some service families. Maybe for some services families where we've just got a lot of badges that all call the exact same API endpoint and report a different value, we could define a single example response shared by all test cases. Maybe it is good to think of this as a pattern that is available to us when it makes sense, rather than one to apply in all situations. |
Add all auth methods used to testAuth to be generic and used by all services. Add helper functions to make testAuth more readable
|
Oh nvm, i will add another fix tomorrow |
jNullj
changed the title
chris48s
changed the title
|
Thanks for looking at all the additional auth methods. I think the approach of converting our own source code to text and attempting to parse javascript with regex is not a good solution to this though. It is going to create a situation where changes to the Broadly, there are 2 ways we can handle this:
Option 1 is kind of less nice because there will be a little bit of duplication between the service class and the tests. We specify the auth method twice, and in some cases we might specify the header name twice too. I'd be in favour of option 1, at least to start with. |
|
@chris48s edit: we posted nearly at the same time, so i just got the above response after posting. I think option 1 would work best to test out this new approach for auth testing without making too much changes to the codebase. |
Use apiHeaderKey & bearerHeaderKey as function params rather then extracting them with regex from function strings. Those options are now part of an options object param joined with the contentType that replaces header. header was originaly added for setting content type of the reply, so it makes more sense to directly set the content type
|
Will fix this tomorrow .query(queryObject => queryObject.key === fakeSecret) |
Before this commit the QueryStringAuth would only work for the key of stackexchange. This commit makes the testAuth function generic and allows passing user and pass keys.
|
Ok, the last thing i want to do is add en example for each auth method in thie PR |
Might set wrong header for jwt login request. This commit fixes that.
Some services might have more then one authOrigin. This commit makes sure we test for redundent authOrigins as well as support requests to them if needed.
Prior to this change, JwtAuth testing would lead to erros due to the absence of a specified login endpoint, Nock would be dumplicated for both login and non login hosts and indicate a missing request. This commit enforces the requirement for a new jwtLoginEndpoint argument when testing JwtAuth. The argument seperates the endpoint nock scope from the behavior of the request nock.
jNullj
changed the title
|
Fixed a few things and made sure we got at least 1 example for each authentication method. |
|
Thanks. The latest iteration of this looks really good. Only one minor comment |
services/test-helpers.js
Outdated
| if (!fakeauthorizedOrigins || typeof fakeauthorizedOrigins !== 'object') { | ||
| throw new TypeError('Invalid fakeauthorizedOrigins: Must be an array.') | ||
| } |
There was a problem hiding this comment.
The check and exception text don't seem to match here
chris48s
changed the title
|
Cool. This looks great. Just FYI, if you want to follow up more of these #9493 (comment) we can skip running the service tests on these PRs. Putting the service names in square brackets runs the service tests - the ones in files called The tests in files called So if we're not actually modifying the service code, it is ok to skip the service tests. |
This PR refs #9493
As mentioned in the issue, this is a first attempt to improve auth testing with stackexchange as a study case and hopefully a first step.
I tried to stick with making everything testable from within the same file while reusing the same code.
If
testAuthis very reusable it might be better to move it intotest-helpers.jsto save a few lines of code, i suspect we might be able to use it for most tests saving a few lines of code per test.I think this could be improved if i had a way to generate the
dummyResponseautomatically from the oneapi object (should be possible in theory, there are some unmaintained modules that could do that, should i make my own to only cover what is needed by shields?)