Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'mod_wsgi.Input' object has no attribute 'tell' #545

Closed
sakshigupta3 opened this issue Mar 19, 2020 · 5 comments
Closed

'mod_wsgi.Input' object has no attribute 'tell' #545

sakshigupta3 opened this issue Mar 19, 2020 · 5 comments

Comments

@sakshigupta3
Copy link

I'm running the below versions of httpd, mod_wsgi and latest google-cloud-storage code

  • httpd-2.4.41-12.fc31.x86_64
  • python3-mod_wsgi-4.6.6-2.fc31.x86_64
  • python3.7
    I'm getting the below traceback sometimes, the issue is intermittent and that's why more confusing ( like once in an hour ). We upgraded from python2.7 to python3.7, could this be a dependency issue ? Is so, why is it intermittent ?

[2020-03-19 07:23:01,337] [ERROR] GCPStorage: PutRequest: stest.2020-03-19/DynamicPolicyRouteChangeTest.py-stest131.sjc-16045647-403435829.log.gz. Unexpected error: (<class 'AttributeError'>) 'mod_wsgi.
Input' object has no attribute 'tell'
[2020-03-19 07:23:01,340] [ERROR] Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/LogLake/Server.py", line 477, in handlePutRequest
requestBody, requestHeaders )
File "/usr/lib/python3.7/site-packages/LogLake/Storage.py", line 79, in putObject
self.bucketName( bucketName ), objectName, content, headers )
File "/usr/lib/python3.7/site-packages/LogLake/GCPStorage.py", line 221, in doPutObject
size=fileSize, client=self.gcpStorageClient )
File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1261, in upload_from_file
client, file_obj, content_type, size, num_retries, predefined_acl
File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1171, in _do_upload
client, stream, content_type, size, num_retries, predefined_acl
File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1114, in _do_resumable_upload
predefined_acl=predefined_acl,
File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1063, in _initiate_resumable_upload
stream_final=False,
File "/usr/local/lib/python3.7/site-packages/google/resumable_media/requests/upload.py", line 343, in initiate
stream_final=stream_final,
File "/usr/local/lib/python3.7/site-packages/google/resumable_media/_upload.py", line 414, in _prepare_initiate_request
if stream.tell() != 0:
AttributeError: 'mod_wsgi.Input' object has no attribute 'tell'

@GrahamDumpleton
Copy link
Owner

The wsgi.input in WSGI applications although it is sort of file like, is not guaranteed, nor required to support all operations that a Python file object supports. In particular it isn't required to support asking where in the stream the file pointer is located, nor can the stream be rewound.

The package you are using appears to be trying to apply such file like operations and so strictly speaking probably isn't going to work on various WSGI servers. Even if such operations did exist for wsgi.input in some WSGI servers, changing the location of the file pointer would break the operation of the WSGI servers handling of how much data has been read.

The only way around this would be to read in the complete input as a string, and then wrapt it in a StringIO object.

This could then be supplied to the library.

Alternatively, read the input and save it to a file, and then operate on the file.

@sakshigupta3
Copy link
Author

Thanks for the quick response. I'll try that, but things were working fine in the prior version ( mod_wsgi 3.4 ). Would reverting to an older version compatible with python3 solve this ?

@GrahamDumpleton
Copy link
Owner

There would have been no difference in mod_wsgi. The difference must be with how that third party package works in Python 3 vs Python 2.7.

@GrahamDumpleton
Copy link
Owner

FWIW, the only methods wsgi.input has ever supported in mod_wsgi is:

static PyMethodDef Input_methods[] = {
    { "close",     (PyCFunction)Input_close,     METH_NOARGS, 0 },
    { "read",      (PyCFunction)Input_read,      METH_VARARGS, 0 },
    { "readline",  (PyCFunction)Input_readline,  METH_VARARGS, 0 },
    { "readlines", (PyCFunction)Input_readlines, METH_VARARGS, 0 },
    { NULL, NULL}
};

@sakshigupta3
Copy link
Author

Thanks for the help. Using BytesIO worked.

sf-project-io pushed a commit to redhat-cip/dci-control-server that referenced this issue Sep 6, 2021
To avoid copying the data in-memory or locally we used to try to pass
`flask.request.stream` instead of `flask.request.data` when receiving
files (specially when using Swift backend).

It appears that mod_wsgi never supported the use of `.tell()` [1] and
this maybe has never worked as we thought on the production environment
at least.

In addition since we're using DCIAuth as authentication method 99.9% of
the time, the stream is already consumed anyway during the content
checksum, making this optimization attempt useless. [2]

This change removes the `upload_component_file()` function and replaces
its calls with `flask.request.data`.

[1]: GrahamDumpleton/mod_wsgi#545 (comment)
[2]: from production logs, in August 2021:
  - `request.stream` was returned 2 times
  - `request.data` was returned 743300 times

Change-Id: I4797c18498237a297351fd4c40a8de69a9780180
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants