Chat Exporting Error

[jdl96]

Hello, I am getting an error when I am trying to export part of a Skype chat and I am unsure of what is causing the problem. I am using a freshly installed copy of Skyperious. Here is the error I am getting:

    Traceback (most recent call last):
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skyperious", line 3294, in on_filterexport_chat
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\export", line 118, in export_chats
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\export", line 226, in export_chat_template
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\third_party.step", line 80, in stream
      File "<string>", line 31, in <module>
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1542, in parse
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1809, in dom_to_html
    AttributeError: 'NoneType' object has no attribute 'text'

EDIT: I narrowed down the problem to a specific day in the logs I was trying to export, and when I try to export just that day only, I get the following error:

    Traceback (most recent call last):
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skyperious", line 3294, in on_filterexport_chat
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\export", line 118, in export_chats
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\export", line 226, in export_chat_template
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\third_party.step", line 80, in stream
      File "<string>", line 44, in <module>
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1533, in parse
      File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1654, in parse_message_dom
    UnboundLocalError: local variable 'b' referenced before assignment

[Gorbush]

I have the second problem as well.

Error merging chats:Traceback (most recent call last): File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\workers", line 487, in work_diff_merge_left File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\workers", line 792, in get_chat_diff_left File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1533, in parse File "C:\stuff\projektid\skyperious\packaging\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1654, in parse_message_dom UnboundLocalError: local variable 'b' referenced before assignment

[suurjaak]

A new Skyperious version, 3.3 has just been released, where the "local variable 'b'" error should be fixed. Can you confirm?
Does the first error still occur in the new version?

[Gorbush]

It Works! Owesome!

[suurjaak]

Glad to hear that :)

[jdl96]

I have updated and can now export the Skype chats I wanted.. Well, sort of. One of them that I wasn't able to export is now fixed, and I can export it to html. However, I have another chat that won't work in html. It still gives me the AttributeError: 'NoneType' object has no attribute 'text' error. I can export it to .txt though.

[suurjaak]

jdl96: can you give the full error you're getting?

[jdl96]

This is the error I'm getting:

    Traceback (most recent call last):
      File "C:\stuff\projektid\skyperious\dist\build\pyinstaller\out00-PYZ.pyz\skyperious", line 3384, in on_filterexport_chat
      File "C:\stuff\projektid\skyperious\dist\build\pyinstaller\out00-PYZ.pyz\export", line 121, in export_chats
      File "C:\stuff\projektid\skyperious\dist\build\pyinstaller\out00-PYZ.pyz\export", line 198, in export_chat_template
      File "C:\stuff\projektid\skyperious\dist\build\pyinstaller\out00-PYZ.pyz\third_party.step", line 80, in stream
      File "<string>", line 35, in <module>
      File "C:\stuff\projektid\skyperious\dist\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1547, in parse
      File "C:\stuff\projektid\skyperious\dist\build\pyinstaller\out00-PYZ.pyz\skypedata", line 1865, in dom_to_html
    AttributeError: 'NoneType' object has no attribute 'text'

[suurjaak]

jdl96: can you try this pre-release version, and see if any error still occurs?

[jdl96]

With your pre-release I am able to successfully export the chat in html! Thank you!
However I think I found another glitch. I tried to do a test to see if I could export a chat from the first of this month to yesterday the 18th, but it only exports the first day, and not even all of it. All messages and days after a picture file was sent are cut off. If I try skipping the 1st and try the 2nd to the 18th, the same thing happens. There was a picture sent on the 2nd too and all messages/days after that are missing.

[suurjaak]

Oho, that is a weird one. And the incomplete HTML export displays in browser properly? What happens if you try the same export to text or Excel?
Are there any messages in the log window (from menu Help -> Show log window)?

[jdl96]

Yes, the incomplete html export displays properly in the browser, but the messages beyond the file sending are missing. I tested in Chrome, Firefox, and IE.
All the messages are there when I export to text, exel, and csv so it must be an issue with html exports. There are no messages/errors in the log window.

EDIT: Okay so I just discovered that the messages actually are there, it's just you can't see them. I looked at the html file's source in a text editor and the missing messages are there, but they're not being displayed when viewing the file in a browser for some reason.

[suurjaak]

Aha, that sheds some light. It sounds like some messages result in invalid HTML, probably to do with something new in Skype.

Could you paste a section of the HTML, from the start of the picture message to the start of the next message, or send it to my e-mail? You can replace any personal information in there with random text.

[jdl96]

I pasted the code on pastebin since it didn't want to show up correctly here, I hope you don't mind: http://pastebin.com/HpGxrjHt

[suurjaak]

Thank you, figured out the problem now.

Will have a new version for you to try out soon, if you don't mind.

[jdl96]

Sure, I don't mind.

[suurjaak]

jdl96: here is an updated version. How does that one work?

[jdl96]

It works great! All the messages appear now and everything seems to be exporting properly without any errors.

[suurjaak]

Excellent. I will release an official update in a short while.

[suurjaak]

jdl96, can you confirm if these errors are fixed in the new v3.4 release?

[jdl96]

Just tried exporting some things and haven't had any errors, but I think I did find another bug.
Today's the 16th of April for me, and for some reason I can select up to April 17th on the export filter thing. I re-exported the month of march that I've exported before just to compare the two files and see if everything is alright, and I noticed something strange with the timing of messages.. For example, a message is posted on March 29 at 23:00 (11:00PM) on the original export, but with the update you released, the message now appears on March 31st at 0:00 (12:00AM) on the new exported file. It looks as if the program sees everything 1 day and 1 hour ahead.

[suurjaak]

This update did contain a change in handling timestamps. Previously message timestamps were interpreted using local computer timezone, now they take into account the timezone setting in Skype profile. But at the moment it's unclear how can this result in a shift of more than a day..

Can you tell me the value of the timezone field in table Accounts (under "Data tables" tab)? And what is your computer's current date-time?

Also, can you tell me the value of the timestamp field in table Messages for this one example message? Easiest way to see this is probably going to the "SQL window" tab and executing a query like

SELECT * FROM Messages WHERE body_xml LIKE '%text from your message%'

[jdl96]

Timezone is set to "72000".
Timestamp for the message, if I did this correctly, is "1427684438".
My computer's date and time is April 16th, 2015, 10:54AM.

[suurjaak]

Thank you. Another question - what datetime does Skype show for this message?

[jdl96]

In Skype itself? If I hover over the time the tooltip says "Sunday, March 29, 2015 11:00:38 PM".

[suurjaak]

Thank you, I think I've figured out the bug now.

Do you happen to remember if the actual message time was 23:00, or could it have been 00:00? Skype can actually show wrong time in message history, as it does not account for daylight saving time. Or do you reside in a timezone where clocks are not adjusted?

[jdl96]

I have Skype set to go by my computer's time. Daylight savings was on March the 8th, and the computer automatically changes the time so that's probably not the issue. I wasn't online at the time the message was posted so I don't know for sure, but it was probably at 23:00.

[SzaraHill]

Hi! I've been using Skyperious for over a year now to log an ongoing conversation in an Excel database, and it's been a godsend. This timestamp issue showed up for me with a previous release (either 3.4 or 3.3 - I've been a bit behind in exporting my messages so I can't be absolutely sure).

I can easily compensate for the time offset with a formula, and I'm not in any way complaining about this minor error in such a magnificent program that's free! However, I thought you'd appreciate the additional detail in case you wanted to troubleshoot further.

I can confirm that this is not accounted for simply by daylight saving time, as I'm quite aware of when the messages are sent and received in local time on my computer or device via Skype. There's definitely a difference between this and the timestamp in Skyperious.

For example, this is how I discovered the discrepancy: I'd previously exported the dates April 9-10 of this year into Excel. Today I exported April 10 onward and discovered the following conversations were duplicated:

April 9, 08:13-14:26 duplicated as April 10, 09:13-15:26
April 10, 11:33 - 19:01 duplicated as April 11, 12:33-20:01

It doesn't seem to correspond to any existing timezone either. My computer's regional settings are set to UTC-07:00 Mountain Time (US & Canada), currently observing DST; according to timeanddate.com there is no time zone 13 hours ahead of this.

Please let me know if there's anything I can do to help you pinpoint the source of the error, but if it's not a priority I completely understand.

[suurjaak]

SzaraHill, thank you for the additional information. I think I have found a solution.

jdl96 and SzaraHill: can you try this pre-release version and see if the times are correct now?

[jdl96]

I re-exported a March 2015 chat to test your pre-release version, and the timing is still a bit off.
The original export shows a message on March 1st at 00:00, while the pre-release export shows the same message on the same day, but at an hour later instead (01:00).

[suurjaak]

Well, dammit :)

Can you try this 3.4.1c version? I added a temporary "Timezone setting" toggle on the database page. If you change the setting, it will change timestamps accordingly and reload the current chat. Can you report which setting results in correct datetimes?

[SzaraHill]

Success on the date! Default timezone setting (DST shift) is still one hour ahead; I made the mistake of attempting to change the setting with a 70000+ message chat loaded... program seemed to freeze and no change happened. Re-started and tried again with a much smaller chat selected, and found success!

The window listing "all chat entries in database" doesn't change, but when I switch to "no DST shift" or "local time", the lower window with the actual chat now shows the correct date and time for all chats.

Well done and way to go!

[jdl96]

No DST Shift and Local Time show the correct date and time in Skyperious itself, but when exporting in HTML, the only one that shows the correct time for me is Local Time... No DST Shift and DST Shift are both still an hour ahead.

[SzaraHill]

Dang, I hadn't tried doing an export. Fixed that now.
For me, timestamp is correct on both .html and .xlsx export with "No DST Shift" AND "local time" selected.

(By the way, I used to find that having Skyperious open, even if it wasn't reading the main database, led to an error logging in to Skype. That isn't happening anymore. Whatever you did, or whatever the Skype people did - props!)

[suurjaak]

This is a pretty pickle.. Because in my own database, "DST shift" works 100%, but "no DST shift" shows history wrong in winter time. "local time" works, of course, but the sole aim of this change was NOT to rely on local time :) I noticed that when opening a database on another computer using a different timezone, all messages were displayed in that timezone and not at the originally sent time.

Looks like I will have to roll back this change altogether.

SzaraHill: the Skype+Skyperious simultaneous usage issue was cleared by Skype, formerly they used to lock the database file exclusively.

[suurjaak]

Can you please check if the time issue has been fully fixed in the new v3.4.1 release?

[jdl96]

Yes, it looks fixed from here in html export. Thanks!