改成bin/python-lib.bypy.frozen了 #2

pfnsnp · 2020-10-27T11:12:38Z

修改后重新编译https://github.com/kovidgoyal/calibre/blob/master/bypy/README.rst吧！

YR1044 · 2021-07-28T08:33:32Z

请问是否有计划支持5.2版本自动化？

snomiao · 2021-08-01T05:17:13Z

请问是否有计划支持5.2版本自动化？

暂无……目前 release 里面的 5.1.0 安装包+修改包套件对我还够用，5.2.0之后需要编译calibre，然后编译方法之前试了一晚上没能把环境配出来于是暂时放弃了（不排除啥时候发现calibre有新功能我很想要了再折腾一遍……）……要是哪位dalao给5.2.0以上搞成功的话，欢迎把链接贴到这个issue里面，贴过来！

snomiao · 2021-12-07T12:55:29Z

说起来有没有什么能在Docker上把那套东西编译出来的办法……有人知道相关项目吗
查到了以下几个项目，可惜都不是编译用的。只是些web server

wyyqyl · 2022-11-08T14:17:15Z

使用开发者模式修改backend.py这个文件后都不需要编译他就可以实现这个功能了

c0ur1er · 2022-12-16T19:40:28Z

使用开发者模式修改backend.py这个文件后都不需要编译他就可以实现这个功能了

哇确实🍗🍗，这个方法应该对未来的新版本都有效。实测Windows 10 Calibre v6.10.0有效。
正常安装后下载源码然后添加环境变量以及修改源码就可以了！

但是毕竟是开发者模式下运行的，与正常使用相比还是存在一些问题的

kurikomoe · 2022-12-30T10:15:36Z

https://github.com/kurikomoe/calibre-utf8-path
使用开发者模式，把东西打包了一下。同时改了一下，现在可以正常的通过批量重命名的方式刷新旧 calibre 保存的文件的文件名了。

同时编译了一个 6.10 的 Linux 版本。

Issues-translate-bot · 2022-12-30T10:15:48Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

https://github.com/kurikomoe/calibre-utf8-path
Using the developer mode, I packaged things up. At the same time, it was changed, and now the file names of the files saved by the old caliber can be refreshed normally through batch renaming.

At the same time, a 6.10 Linux version was compiled.

kurikomoe · 2022-12-30T10:16:49Z

说起来有没有什么能在Docker上把那套东西编译出来的办法……有人知道相关项目吗查到了以下几个项目，可惜都不是编译用的。只是些web server

linuxserver/calibre - Docker Image | Docker Hub

kurron/docker-calibre: A Docker image that can run the Calibre ebook manager.

现在的编译方法是利用 kvm 虚拟机跑编译的，Linux 上编译改改编译脚本还是比较方便的编译一份的。Windows 目前咱自己没啥需求就没有尝试编译。

Issues-translate-bot · 2022-12-30T10:17:01Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Speaking of which, is there any way to compile that set of things on Docker... Does anyone know the relevant projects? I found the following projects, but unfortunately they are not used for compilation. just some web servers

linuxserver/calibre - Docker Image | Docker Hub

kurron/docker-calibre: A Docker image that can run the Caliber ebook manager.

The current compilation method is to use the kvm virtual machine to run and compile. It is more convenient to compile a copy of the compiled script on Linux. Windows At present, we have no need to try to compile.

rogeryoungh · 2023-01-28T05:40:47Z

现在的编译方法是利用 kvm 虚拟机跑编译的，Linux 上编译改改编译脚本还是比较方便的编译一份的。Windows 目前咱自己没啥需求就没有尝试编译。

@kurikomoe 请教一下如何编译？我正在尝试改造成 github action，但是他这个文档讲的太不清楚了。

Issues-translate-bot · 2023-01-28T05:40:58Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

The current compilation method is to use the kvm virtual machine to run the compilation. It is more convenient to compile a copy of the compilation script on Linux. Windows At present, we have no need to try to compile.

@kurikomoe Please tell me how to compile? I'm trying to transform it into a github action, but his documentation is too unclear.

kurikomoe · 2023-01-30T07:37:32Z

现在的编译方法是利用 kvm 虚拟机跑编译的，Linux 上编译改改编译脚本还是比较方便的编译一份的。Windows 目前咱自己没啥需求就没有尝试编译。

@kurikomoe 请教一下如何编译？我正在尝试改造成 github action，但是他这个文档讲的太不清楚了。

如果用 github action 的话，直接用官方的编译方法应该是不行的，github action（非 hosted）不支持嵌套虚拟化，因此无法在免费机子上启动 kvm。只能自己看一下 calibre 的 bypy repo [https://github.com/kovidgoyal/bypy.git]，把 python 发往 kvm 的编译命令拉出来重写一个编译脚本。（感觉修改bypy 里面的命令发送函数，直接提取那堆 shell 指令还是可行的）

另一个想法是改 libcalibre-launcher.so 这个文件，stirngs 看了一下，感觉最后的 python scripts 似乎是编译到这里了，理论上把这玩意的编译抓出来直接替换 so 就够了

Issues-translate-bot · 2023-01-30T07:37:43Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

The current compilation method is to use the kvm virtual machine to run and compile. It is more convenient to compile a copy of the compiled script on Linux. Windows At present, we have no need to try to compile.

@kurikomoe Please tell me how to compile? I'm trying to transform it into a github action, but his documentation is too unclear.

If you use github action, it should not be possible to directly use the official compilation method. github action (non-hosted) does not support nested virtualization, so you cannot start kvm on a free machine. I can only look at caliber's bypy repo [https://github.com/kovidgoyal/bypy.git], pull out the compilation command sent by python to kvm, and rewrite a compilation script. (I feel that it is still feasible to modify the command sending function in bypy and directly extract the pile of shell instructions)

kurikomoe · 2023-02-02T03:26:22Z

昨天有空的时候跟了一下编译脚本。
TL;DR
执行 CALIBRE-ROOT/bypy/linux/main.py 的话可以绕过虚拟机编译出 frozen 文件。但是需要自己处理 python packages 依赖。

BYPY_ROOT=$PWD/.. python3 bypy/linux/__main__.py  
sudo ln -sf /tmp/t/frozen-*/lib/libcalibre-launcher.so /opt/calibre/lib/libcalibre-launcher.so 
sudo ln -sf /tmp/t/frozen-*/lib/calibre-extensions/python-lib.bypy.frozen  /opt/calibre/lib/calibre-extensions/

（bypy.git 和 calibre.git 在同一个目录下，在 calibre.git 下执行，需要手动修改 bypy.git/**/constants.py 的一些常量定义）

分析：

frozen 文件实际上是把各种依赖的 pyc 文件打包起来，之后 bypy-frozen-importer 负载 hook python import 机制从 frozen 文件解包 pyc 执行
frozen 在打包的时候会生成一个一次性的 bypy-byte-index.h 的文件用于记录 pyc 文件在 frozen 里面的偏移，但是编译完之后由于 strip 的问题， header 文件里面的常量定义拿不到了，因此想解包 frozen 替换文件后再重新打包回去会比较难（可能分析 pyc 文件头能做到之类的）。
打包同时也会打包对应的 python dep 文件。在 manjaro 上用 python3.10.9 的话，手动按报错一个一个装 dep 能拿到一个勉强能跑的版本（一堆 import 没处理，但是 calibre 启动，转换 epub->mobi，设备识别正常）
需要替换 .frozen 文件和 .so 文件，同时提供/复用原本的 lib

相关 repo（注意，需要手动 bypy/linux/__main__.py 里面的一堆路径，同时 bypy.git 需要根据 0001 patch 改动一下）
https://github.com/kurikomoe/calibre-utf8-path/tree/poc

编译好的文件：
https://github.com/kurikomoe/calibre-utf8-path/blob/poc/calibre-patch-poc.txz

Issues-translate-bot · 2023-02-02T03:26:36Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

When I was free yesterday, I followed the compilation script.
TL;DR
If you execute CALIBRE-ROOT/bypy/linux/main.py, you can bypass the virtual machine and compile frozen files. But you need to handle python packages dependencies yourself.

BYPY_ROOT=$PWD/.. python3 bypy/linux/__main__.py
sudo ln -sf /tmp/t/frozen-*/lib/libcalibre-launcher.so /opt/calibre/lib/libcalibre-launcher.so
sudo ln -sf /tmp/t/frozen-*/lib/calibre-extensions/python-lib.bypy.frozen /opt/calibre/lib/calibre-extensions/

(bypy.git and calibre.git are in the same directory and executed under calibre.git, some constant definitions of bypy.git/**/constants.py need to be manually modified)

analyze:

The frozen file actually packs various dependent pyc files, and then bypy-frozen-importer loads the hook python import mechanism to unpack the pyc from the frozen file for execution
When frozen is packaged, it will generate a one-time bypy-byte-index.h file to record the offset of the pyc file in frozen, but after compiling, due to the problem of strip, the constant definition in the header file takes Not anymore, so it will be more difficult to unpack the frozen replacement file and then repack it back (maybe it can be done by analyzing the pyc file header).
Packaging will also package the corresponding python dep file. If you use python3.10.9 on manjaro, you can manually install dep one by one according to the error report to get a version that can barely run (a bunch of imports are not processed, but caliber starts, converts epub->mobi, and the device recognition is normal)
Need to replace .frozen files and .so files while providing/reusing the original lib

Relevant repo (note that a bunch of paths in bypy/linux/__main__.py need to be manually edited, and bypy.git needs to be changed according to 0001 patch)
https://github.com/kurikomoe/calibre-utf8-path/tree/poc

Cirn09 · 2023-03-22T15:22:22Z

我们不用很麻烦很累就可以用上中文路径： #6 (comment)

Issues-translate-bot · 2023-03-22T15:22:34Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

We can use the Chinese path without much trouble and fatigue: #6 (comment)

Cirn09 · 2023-03-28T11:52:58Z

新版的 Patch 方案：
https://github.com/Cirn09/calibre-do-not-translate-my-path

Issues-translate-bot · 2023-03-28T11:53:08Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

The new version of the Patch scheme:
https://github.com/Cirn09/calibre-do-not-translate-my-path

snomiao · 2023-03-28T12:11:49Z

great job！ 2023年3月28日(火) 20:53 Cirn09 ***@***.***>:

…

新版的 Patch 方案： https://github.com/Cirn09/calibre-do-not-translate-my-path — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABX33FTJQ7DEAZR46BWWHIDW6LGKNANCNFSM4TAVM3KA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

pfnsnp closed this as completed Nov 1, 2020

pfnsnp reopened this Nov 1, 2020

pfnsnp closed this as completed Nov 11, 2020

snomiao reopened this Apr 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

改成bin/python-lib.bypy.frozen了 #2

改成bin/python-lib.bypy.frozen了 #2

pfnsnp commented Oct 27, 2020

YR1044 commented Jul 28, 2021

snomiao commented Aug 1, 2021 •

edited

Loading

snomiao commented Dec 7, 2021

wyyqyl commented Nov 8, 2022

c0ur1er commented Dec 16, 2022 •

edited

Loading

kurikomoe commented Dec 30, 2022

Issues-translate-bot commented Dec 30, 2022

kurikomoe commented Dec 30, 2022

Issues-translate-bot commented Dec 30, 2022

rogeryoungh commented Jan 28, 2023

Issues-translate-bot commented Jan 28, 2023

kurikomoe commented Jan 30, 2023 •

edited

Loading

Issues-translate-bot commented Jan 30, 2023

kurikomoe commented Feb 2, 2023 •

edited

Loading

Issues-translate-bot commented Feb 2, 2023

Cirn09 commented Mar 22, 2023

Issues-translate-bot commented Mar 22, 2023

Cirn09 commented Mar 28, 2023

Issues-translate-bot commented Mar 28, 2023

snomiao commented Mar 28, 2023 via email

改成bin/python-lib.bypy.frozen了 #2

改成bin/python-lib.bypy.frozen了 #2

Comments

pfnsnp commented Oct 27, 2020

YR1044 commented Jul 28, 2021

snomiao commented Aug 1, 2021 • edited Loading

snomiao commented Dec 7, 2021

wyyqyl commented Nov 8, 2022

c0ur1er commented Dec 16, 2022 • edited Loading

kurikomoe commented Dec 30, 2022

Issues-translate-bot commented Dec 30, 2022

kurikomoe commented Dec 30, 2022

Issues-translate-bot commented Dec 30, 2022

rogeryoungh commented Jan 28, 2023

Issues-translate-bot commented Jan 28, 2023

kurikomoe commented Jan 30, 2023 • edited Loading

Issues-translate-bot commented Jan 30, 2023

kurikomoe commented Feb 2, 2023 • edited Loading

Issues-translate-bot commented Feb 2, 2023

Cirn09 commented Mar 22, 2023

Issues-translate-bot commented Mar 22, 2023

Cirn09 commented Mar 28, 2023

Issues-translate-bot commented Mar 28, 2023

snomiao commented Mar 28, 2023 via email

snomiao commented Aug 1, 2021 •

edited

Loading

c0ur1er commented Dec 16, 2022 •

edited

Loading

kurikomoe commented Jan 30, 2023 •

edited

Loading

kurikomoe commented Feb 2, 2023 •

edited

Loading