Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: 词库文件名不支持中文 #1080

Closed
ivysrono opened this issue Jan 10, 2024 · 10 comments · Fixed by rime/librime#806
Closed

Bug: 词库文件名不支持中文 #1080

ivysrono opened this issue Jan 10, 2024 · 10 comments · Fixed by rime/librime#806

Comments

@ivysrono
Copy link

操作系统信息

  • OS 详细版本: Windows 11 23H2
  • 小狼毫版本: 0.15.0.0

描述遇到的问题
测试了很久,最后发现 *.dict.yaml 这个文件名中不能存在中文。

复现步骤
重现问题的步骤,如:

rime/librime#792 以及 #576 (comment) 确认该问题为小狼毫独有,librime无此问题。

正如@Qeynos 所说,毕竟我们是个主要用于输入中文的输入法,文件名不能完整支持中文稍有怪异。

@Techince
Copy link

新版本已经支持中文路径了,在Actions里可以下载最新的构建版本。

@Qeynos
Copy link

Qeynos commented Jan 10, 2024 via email

@Techince
Copy link

Techince commented Jan 10, 2024

但最新nightly被windows defender报木马 Techince @.>于2024年1月10日 周三21:43写道:

新版本已经支持中文路径了,在Actions里可以下载最新的构建版本。 — Reply to this email directly, view it on GitHub <#1080 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG3ZW4JJLFNUG4WODJAPQTTYN2LIRAVCNFSM6AAAAABBUWE2NWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBUHA3TMMBUGY . You are receiving this because you were mentioned.Message ID: @.
>

添加到信任区域就行了,没有病毒的。
我本机编译时也遇到过,编译即将完成时被杀软拦截了,添加到信任区域就可以使用了。

@fxliang
Copy link
Contributor

fxliang commented Jan 25, 2024

1,中文路径去年已修复
2,所述方案建议上传供评估
3,关于报毒,请参考下图(主页下载页截图)
Screenshot_20240125_231819

@ivysrono
Copy link
Author

ivysrono commented Jan 28, 2024

@fxliang https://github.com/liuhuanyong/DomainWordsDict

2,所述方案建议上传供评估

@fxliang
Copy link
Contributor

fxliang commented Jan 28, 2024

@fxliang https://github.com/liuhuanyong/DomainWordsDict

2,所述方案建议上传供评估

这些都不是词典格式

@ivysrono
Copy link
Author

@fxliang https://github.com/liuhuanyong/DomainWordsDict

2,所述方案建议上传供评估

这些都不是词典格式

文件内容显然会做相应调整。

@fxliang
Copy link
Contributor

fxliang commented Jan 29, 2024

前端处理的是设置shared data dir 和user data dir,这个是有确认支持中文路径的。其他都是librime处理的过程。

如果没有可测试的完整方案,这个issue就不会有结果。

或者你试试在librime提issue。

更新:试了下,确认是librime的问题

@ivysrono
Copy link
Author

ivysrono commented Feb 2, 2024

感谢,坐等更新。

@ivysrono ivysrono closed this as completed Feb 2, 2024
@fxliang
Copy link
Contributor

fxliang commented Feb 2, 2024

感谢,坐等更新。

之前不了解github actions的一些设置,造成一些反复的尝试,现在修复了。可以看看这个的产出(已带上主仓的release的插件)
https://github.com/fxliang/librime/actions/runs/7754411046

如无问题我晚上再提pr

lotem added a commit to lotem/librime that referenced this issue Feb 3, 2024
Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.
Closes rime#804

Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: `installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.
lotem added a commit to lotem/librime that referenced this issue Feb 3, 2024
Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.
Closes rime#804

Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: `installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.
lotem added a commit to lotem/librime that referenced this issue Feb 3, 2024
Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.
Closes rime#804

Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: `installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.
lotem added a commit to lotem/librime that referenced this issue Feb 3, 2024
Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.
Closes rime#804

Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: `installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.
lotem added a commit to lotem/librime that referenced this issue Feb 3, 2024
Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.
Closes rime#804

Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: `installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.
lotem added a commit to lotem/librime that referenced this issue Feb 4, 2024
Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.
Closes rime#804

Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: `installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.
lotem added a commit to rime/librime that referenced this issue Feb 6, 2024
refactor: convert path to native encoding on Windows

feat(rime_api): provide secure version of path getter functions `RimeApi::get_*_dir_s`. 

Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.

Closes #804
Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: Most `string` filenames in APIs are changed to `path`;
`installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.

Details of the code refactor

Wrap `std::filesystem::path` in a thin wrapper class `rime::path` which calls `std::filesystem::u8path` in the constructor on Windows.

Operator `/=` and `/` are also overloaded to convert the right operand from UTF-8 string to native path.

Follow these rules to apply correct conversion between `string` and `rime::path`:

- construct `rime::path` with UTF-8 encoded string;
- get native string by `path::u8string`;
- to extract UTF-8 string from `path`, for example to find schema ID from file name, call `path::u8string`;
- avoid implicit conversion from string, which results in `std::filesystem::path` without performing UTF-8 to native conversion;
- explicitly construct `rime::path` from `std::filesystem::path` before append operation, to ensure the overloaded operator with string conversion is used.
graphemecluster pushed a commit to TypeDuck-HK/librime that referenced this issue Mar 18, 2024
refactor: convert path to native encoding on Windows

feat(rime_api): provide secure version of path getter functions `RimeApi::get_*_dir_s`.

Follow @fxliang 's PR, use `u8path` on Windows to convert UTF-8 string
to Windows native path.

Closes rime#804
Fixes rime/weasel#576
Fixes rime/weasel#1080

BREAKING CHANGE: Most `string` filenames in APIs are changed to `path`;
`installation.yaml` should be UTF-8 encoded.

Previouly on Windows, the file can be written in local encoding to
enable paths with non-ASCII characters. It should be updated to UTF-8
after this change.

Details of the code refactor

Wrap `std::filesystem::path` in a thin wrapper class `rime::path` which calls `std::filesystem::u8path` in the constructor on Windows.

Operator `/=` and `/` are also overloaded to convert the right operand from UTF-8 string to native path.

Follow these rules to apply correct conversion between `string` and `rime::path`:

- construct `rime::path` with UTF-8 encoded string;
- get native string by `path::u8string`;
- to extract UTF-8 string from `path`, for example to find schema ID from file name, call `path::u8string`;
- avoid implicit conversion from string, which results in `std::filesystem::path` without performing UTF-8 to native conversion;
- explicitly construct `rime::path` from `std::filesystem::path` before append operation, to ensure the overloaded operator with string conversion is used.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants