Pin botocore dependency to a minor version range (like boto3) rather than an exact patch version

[javulticat]

Is your feature request related to a problem? Please describe.
awscli pins its botocore dependency to an exact patch version. For example, awscli==1.18.143 requires botocore==1.18.2. However, boto3 handles its botocore dependency in a different (and, seemingly, better) way, by pinning it to a range representing the current patch version up to the next minor version. For example, boto3==1.15.2 requires botocore>=1.18.2,<1.19.0. The effect that this ends up having is that for projects which depend on both awscli and boto3, awscli must always be updated before boto3 can be updated, for what seems like no necessary technical reason, in order for dependency resolution using common tools like pipenv (21k stars) to succeed.

And while this can be somewhat worked around in certain projects (by explicitly pinning versions of awscli and boto3, and making sure to update awscli before updating boto3), it becomes much more arduous if your project also relies on a 3rd party dependency which does not explicitly pin a version of boto3. For example, the latest version of the popular (4k stars) library moto==1.3.16 requires boto3>=1.9.201, which causes the transitive boto3 dependency identified by a tool like pipenv to always be the latest version of boto3. As we've established, attempting to resolve dependencies using a newer version of boto3 than awscli will always fail, so the end result of this is that anytime a new release of boto3/awscli is released (which is nearly every day), if using a 3rd party which has an unpinned transitive dependency on boto3, all dependency resolution using a tool like pipenv begins failing, even for completely unrelated dependencies, until awscli has been updated.

For example, here is the dependency resolution error we began receiving from pipenv yesterday upon the release of new boto3/awscli versions:

There are incompatible versions in the resolved dependencies:
  botocore<1.19.0,>=1.18.2 (from boto3==1.15.2->moto==1.3.16)
  botocore<2.0a.0,>=1.12.36 (from s3transfer==0.3.3->awscli==1.18.142)
  botocore==1.18.1 (from awscli==1.18.142)
  botocore>=1.11.3 (from aws-xray-sdk==2.6.0->moto==1.3.16)
  botocore>=1.12.201 (from moto==1.3.16)

And, for clarity, here is what our dependencies looked like in our Pipfile yesterday:

awscli==1.18.142
boto3==1.15.1
moto==1.3.16

And while manually updating our dependency pin to awscli==1.18.143 resolved the issue for the day, it will return when 1.18.144 is released, presumably, later today.

Describe the solution you'd like
For awscli to pin its botocore dependency the same way boto3 does - to a range including all future patch versions of the same minor version. This will at least limit this occurrence from happening nearly every day, to only happening when a new minor release of botocore is created.

So instead of:

• awscli==1.18.143 requiring botocore==1.18.2
• awscli==1.18.143 should require botocore>=1.18.2,<1.19.0 (which is the same botocore requirement that the corresponding boto3==1.15.2 release has)

Describe alternatives you've considered
As mentioned, in the case outlined, the alternative is to manually update awscli every time there is a new patch release (which is almost every day) before being able to successfully resolve dependencies via common dependency resolution tools such as pipenv.

[tim-finnigan]

I think this issue overlaps with #5173 and should maybe be closed in favor of that issue which has more activity. As mentioned there, #6854 was also opened to gauge interest in aligning CLI/botocore minor versions but it was closed to a lack of engagement. (There is another related issue, #5943, involving third party dependencies). Please let us know if there are any distinctions you'd like to make regarding this feature request and those other issues.

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.