allow path to be an StringIO object and skip extension check for that case

[fennnec]

this is to allow in-memory creation of excel-files from pandas

[jreback]

what is the use case for this?

[fennnec]

i have a large django database and i do my exports for reporting in csv and excel formats via django-pandas (and thus via pandas)
i don't need to store the file anywhere on my server but just use a stringbuffer to stream the file to the client requesting it via wrapping it in a django HTTPResponse

xlsxwriter provides this functionality and only the file-extension check inside pandas prevented me from using the in-memory-only version of xlsxwriter

[jreback]

ahh...makes sense, ok need some tests for this then (simple as create a frame, save it to a buffer, prob save the buffer, then read it back and compare); I don't think that you can READ from a buffer as not sure that makes any sense (you would have to have xlrd stream it to you, not even sure if that is possible).

[fennnec]

here is the code i use currently:

response = HttpResponse(content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet') # set contenttype
stringio = StringIO()
ew = ExcelWriter(stringio, engine='xlsxwriter', **{'options': {'in-memory': True}})
dataframe.to_excel(ew, sheet_name='keywordanalyse', engine='xmlxwriter')
ew.save()
response.content = stringio.getvalue()
return response

i don't know why the explicit call to save() is necessary but without it the xslx comes back empty.

I'll try to provide what you asked for, too

[fennnec]

here is a simple test i just did:

 from pandas import DataFrame
 from pandas.io.excel import ExcelWriter
 from StringIO import  StringIO
 buf = StringIO()
 data = {'a': [1,2,3], 'b':[4,5,6], 'c':['wassup', 'wassuuuuuuuup', 'wassuuuuuuuuuuuuuuuuuuup']}
 df = DataFrame(data)
 df

Out[6]:
<> a b c
0 1 4 wassup
1 2 5 wassuuuuuuuup
2 3 6 wassuuuuuuuuuuuuuuuuuuup

 ew = ExcelWriter(buf, engine='xlsxwriter', **{'options': {'in-memory': True}})
 df.to_excel(ew, sheet_name='test', engine='xmlxwriter')
 ew.save()
 xlsxstring = buf.getvalue()
 f = open('text.xlsx', 'w')
 f.write(xlsxstring)
 f.close()

then i verified the outcome via opening the file in excel (see screenshot):

[jreback]

what I mean is pls add the tests to the PR

their are a lot of existing tests in pandas/io/tests/test_excel.py

[fennnec]

ok, I'll try my best although i have never written tests in python

[jreback]

@fennnec just copy paste the test format and modify to use your change.

The ideal way is to write the test first, have it fail, then apply your change and have it fix the issue (and not break anything else!)

see here:
https://github.com/pydata/pandas/wiki/Testing

[fennnec]

i tested it in my application by hand

[jreback]

@fennnec we can only accept patches that have associated tests
and that pass Travis

your patch won't work even with local testing as their is a syntax error

[fennnec]

well, it works locally here

test_stringio_writer (__main__.XlsxWriterTests) ... ok
test_stringio_writer (__main__.XlsxWriterTests_NoMerge) ... ok

so i don't know what the problem with another setup is

[jreback]

you need to do: from pandas.compat import StringIO in order to pass python3

[fennnec]

could you perhaps point me in the right direction for the python 3 errors, since i don't really understand what's going wrong there on the save method

[jreback]

IIRC you have to use io.BytesIO (and not StringIO) as its opened in binary mode. @jtratner ?

[fennnec]

passes travis now

[jreback]

gr8, pls add a release note (reference this PR) in 0.13.1, API changes (edit doc/source/release.rst)

then squash to a single commit

see here: https://github.com/pydata/pandas/wiki/Using-Git

@jtratner / @y-p ok with this?

[ghost]

This was done in a half-assed manner and has all the appearence of a thrown-over-the-wall patch.

👎

[ghost]

@jreback, don't let me put you in a difficult position. If you're satisfied that this
is solid after the review, Merge away.

[jreback]

@fennnec

• needs additional tests for each of the different excel writers; if it doesn't work with the other writers then needs to say this in the docstring
• docstring update
• io.rst update
• whatsnew entry

[cpcloud]

There's also a ton of merge commits.

[jreback]

needs to be rebased on current master

[jreback]

@fennnec needs a rebase

[jreback]

closing this. @fennnec if you want to resubmit this we can take a look again