Offline usage

[akaguny]

Task

Hi, how i may use cheat.sh offline, some of my servers don`t have access to the enternet, but i want to use cheat.sh)

Details

• cheat.sh local installation
• document installation process (installation and standalone usage)
• programming languages cheat sheets support

[chubin]

It's not yet implemented, but it will be implemented soon (on this weekend I hope). It's one of the most wanted features. Stay tuned!

[akaguny]

@chubin you are promise feature, up issue)

[chubin]

That's true. Thank you for upping it

[neilhwatson]

Offline or internal mirror?

[chubin]

I think that offline is better. Internal mirror is another option, but it is possible already. We (me and @akaguny) have meant offline usage here

[giraldeau]

After the ycombinator post, the service returns this:

$ ~/bin/cht.sh <whatever>
/*
 * Sorry, we are experiencing extremely highload now.
 * We are working on the problem and how to get it fixed soon.
 * Please come back in several hours or try some other queries:
 * 
 * For example:
 */

 curl cht.sh/:list               to list available cheat sheets
 curl cht.sh/LANGUAGE/:list      to list available cheat sheets for LANGUAGE
 curl cht.sh/LANGUAGE/:learn     to learn the LANGUAGE

Which means an offline mode would be certainly great.

[chubin]

@giraldeau Yes, it is true. We are trying to mitigate the high load problem currently. I think it will be fixed soon.

[brandonpittman]

I'm sure it won't matter after a few days, but this is the best reason to have it work offline.

However, by the time it works offline, the whole not being able to access it won't even matter.

Until HackerNews sends out its newsletter on Friday and starts the downtime cycle all over again.

[chubin]

@brandonpittman
We will fix the capacity problem (I hope so) and we will be able to handle such a load.

The offline feature is very useful still, for one of the main reasons why cheat.sh was created is to fight the distraction the comes from the web/browser. And the best method to fight the distraction is to go offline. That's why the offline (or semi-offline) mode is extremely important for us.

[brandonpittman]

@chubin I hope I didn't sound negative. 🙇

Love the project's goals and I'm looking forward to later today (Japan time) when maybe the load won't be so heavy and I can play with it a bit.

[Fake4d]

Can i just clone the repository to an internal webserver in my network to use it?

Is there a new plan for the release of an offline feature?

The tool is so useful! Thanks for all the hard work!

[brandonpittman]

Totally forgot about this project until I got the last comment notification.

[chubin]

@brandonpittman @giraldeau
The capacity problem is fixed (hopefully).
The service is still experiencing high load, but now it is working stable.
Please try your queries now, they should work.

Now back to the offline mode.
I would split the issue into two different topics, as proposed by @neilhwatson :

1. Offline mode
2. Mirror

Both of them are really important. I've already written about the offline mode. The mirror feature is very important too.

In the mirror mode, cheat.sh can be installed in Intranet (or even in the Internet) building a cascaded distributed cheat sheets repository:

cheat.sh # for the global cheat sheets
  ^
  |
cht.lan  # for local cheat sheets
 ^
  |
curl cht.lan

The users of cht.lan can access both local cheat sheets and the global cheat sheets.
Local cheat sheets can cover some internal topics such as names of internal services etc.
They local cheat sheets can be located in the root namespace as well as in some dedicated namespace.

This mode can be implemented quite easily. Actually, it is implemented already, the only missing part is the straightforward installation process + documentation.

The offline mode is trickier to implement because of the obvious fact that some cheat sheets should be generated on the fly. But as I said we have several good options here too.

[alphapapa]

I recently noticed the Dash project again, which has mirroring, offline access, and some forms of "cheat sheets" in its libraries. Maybe rather than reinvent the wheel, the content in this project could be added to Dash. Or maybe another instance of Dash's code and infrastructure could be set up with this project's content. Seems a shame to reinvent so many wheels. IIUC this project's primary contribution is how it "ingests" content from sources, and that could be connected with other destinations.

[chubin]

@alphapapa Dash is an interesting project, but it has not so much common with cheat.sh as you probably think

[voodooEntity]

Up ^^ really would love to see this offline

[chubin]

@voodooEntity Do you want to see UNIX/Linux commands cheat sheets offline, or programming languages quesions/answers, or both?

[deathtrip]

I think having both would be great.

[ndavidpsi]

How's the situation? Can cheat.sh be used offline?

[chubin]

@ndavidpsi Yes, you can use cheat.sh offline for UNIX/LInux command cheat sheets, but not yet for coding questions cheat sheets. Do you need the former or the latter?

[ndavidpsi]

I don't know much about cheat.sh yet, so for now my main/only reason for using it is to check programing language syntaxes/tutorials

[chubin]

@ndavidpsi I've documented the installation process in doc/standalone.md; it is pretty straightforward, and there should be no problems with it. UNIX/Linux command line cheat sheets as well as some programming languages tutorials (e.g. /{lang}/:learn) are available offline, but not yet all programming languages cheat sheets; I am working on it though

[ndavidpsi]

Thanks that's wonderful. I actually use Windows but I have access to Linux as well so it isn't an issue. But is there a Windows/cmd version of cheat.sh or will it ever be?

[chubin]

We could easily add it, it won't be hard. Do you want to run it from the terminal or from the editor? In the latter case, what editor/IDE do you use?

[ndavidpsi]

Wow that's great. Both would be ideal, terminal and editor. But if I were to choose only one I'd go with editor support, I use neovim

[chubin]

We currently have no Neovim support, but we will hopefully have it soon
Currently we have support for Vim but not yet for Neovim

[ndavidpsi]

From all the plugins I've tried all those made for Vim work in Neovim so it'll probably work. But actually, if for now you could integrate the feature in Windows Terminal version it would be great, because I can always execute a terminal command within neovim

[chubin]

1. Could you please test out Vim plugin in Neovim? It would be really helpful
2. Do you have some Linux compatibility layers installed in your system? In particular, do you have bash?

[ndavidpsi]

I have access to bash and curl. The plugin for vim works perfectly. Now, what would be the procedure for a standalone installation in vim?

[chubin]

@ndavidpsi The problem is that for the standalone installation you need to much different dependencies, and to install them on Windows is though possible, but I am not sure that it will be straightforward (bash, git, python, python deps). I think the best way here would be to implement a new closs-platform client that could be easily installed in Linux as well as in Windows and in Mac, that would also support the standalone mode. I believe, this approach would be ideal.

[ndavidpsi]

Yeah, that should be the long term solution. But those things you mentioned I have them all, git, python, bash...

[chubin]

Try to go through this, let's see what errors you will get:

https://github.com/chubin/cheat.sh/blob/master/doc/standalone.md

[ndavidpsi]

But that's the Terminal version. I want to do it in Vim version as well

[ndavidpsi]

I'll do it tomorrow morning

[ndavidpsi]

My procedure (on cmd):

• curl https://cht.sh/:cht.sh > some/path
• (didn't use chmod)
• bash.exe cht.sh --standalone-install

Error:

>> Installing python requirements into the virtual environment
    ERROR: Command errored out with exit status 1:
     command: /home/ndavid/.cheat.sh/ve/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-06zi7nqh/PyICU/setup.py'"'"'; __file__='"'"'/tmp/pip-install-06zi7nqh/PyICU/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-q641oh5i
         cwd: /tmp/pip-install-06zi7nqh/PyICU/
    Complete output (53 lines):
    Traceback (most recent call last):
      File "/tmp/pip-install-06zi7nqh/PyICU/setup.py", line 63, in <module>
        ICU_VERSION = os.environ['ICU_VERSION']
      File "/usr/lib/python3.9/os.py", line 679, in __getitem__
        raise KeyError(key) from None
    KeyError: 'ICU_VERSION'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/tmp/pip-install-06zi7nqh/PyICU/setup.py", line 66, in <module>
        ICU_VERSION = check_output(('icu-config', '--version')).strip()
      File "/tmp/pip-install-06zi7nqh/PyICU/setup.py", line 19, in check_output
        return subprocess_check_output(popenargs)
      File "/usr/lib/python3.9/subprocess.py", line 420, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      File "/usr/lib/python3.9/subprocess.py", line 501, in run
        with Popen(*popenargs, **kwargs) as process:
      File "/usr/lib/python3.9/subprocess.py", line 947, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/usr/lib/python3.9/subprocess.py", line 1819, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'icu-config'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/tmp/pip-install-06zi7nqh/PyICU/setup.py", line 69, in <module>
        ICU_VERSION = check_output(('pkg-config', '--modversion', 'icu-i18n')).strip()
      File "/tmp/pip-install-06zi7nqh/PyICU/setup.py", line 19, in check_output
        return subprocess_check_output(popenargs)
      File "/usr/lib/python3.9/subprocess.py", line 420, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      File "/usr/lib/python3.9/subprocess.py", line 501, in run
        with Popen(*popenargs, **kwargs) as process:
      File "/usr/lib/python3.9/subprocess.py", line 947, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/usr/lib/python3.9/subprocess.py", line 1819, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'pkg-config'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-06zi7nqh/PyICU/setup.py", line 71, in <module>
        raise RuntimeError('''
    RuntimeError:
    Please install pkg-config on your system or set the ICU_VERSION environment
    variable to the version of ICU you have installed.

    (running 'icu-config --version')
    (running 'pkg-config --modversion icu-i18n')
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR:
---
Collecting colored
  Using cached colored-1.4.2.tar.gz (56 kB)
Collecting langdetect
  Using cached langdetect-1.0.8.tar.gz (981 kB)
Collecting cffi
  Using cached cffi-1.14.4-cp39-cp39-manylinux1_x86_64.whl (405 kB)
Collecting polyglot
  Using cached polyglot-16.7.4.tar.gz (126 kB)
Collecting PyICU
  Using cached PyICU-2.6.tar.gz (233 kB)
---
See /home/ndavid/.cheat.sh/log/install.log for more
ERROR: Could not install python dependencies into the virtual environment

[ndavidpsi]

I think it would be possible to convert that sh file into a bat file (which is what is used on windows). Then the native console would do all the work. Instead of, in my case, WSL2 bash.exe

[abitrolly]

@ndavidpsi https://github.com/chubin/cheat.sh/blob/master/Dockerfile here are more things that you may have been missing, like PyICU library.

[ndavd]

Thanks (PS: I'm ndavidpsi, using this account from now on)

[ndavd]

Even though I installed pkg-config on bash it is throwing me an error to install it

[chubin]

I think we should disable PyICU for windows, because it will be hard to install it there, and it is needed for a feature that is experimental and not need on Windows anyway.
Can you please remove it from requirements.txt and try again?

[ndavd]

Would it be possible to convert cht.sh to cht.bat? Maybe in another repo or on this one idk. It's just that's what windows uses. That way bash wouldn't be required. It would be the best way to properly implement it on windows

[infokiller]

@sp1n0r can you please open a separate issue and move the Windows discussion there?
I'm subscribed to this issue because I'm interested in offline usage and it just spams my mailbox

[ndavd]

@infokiller of course. I'll open an offline usage for Windows separate issue rn

[mcallaway]

I also saw the above error condition when trying a local install on Mac:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/private/tmp/pip-install-xxqgi8yo/pyicu_6ef8b7f9e76c451f8604abb68cf4752e/setup.py", line 71, in <module>
    raise RuntimeError('''
RuntimeError:
Please install pkg-config on your system or set the ICU_VERSION environment
variable to the version of ICU you have installed.

I solved this via:

brew install icu4c

And updating my environment as directed.

I also use pyenv for Python environments. I will create a new Issue to detail what I did to make local install work when pyenv is present.

[chubin]

@mcallaway Matt, we will make this ICU Module optional; see #272. We already discussed it with @abitrolly: this module is needed only for multi-language, non-English queries support, which is (1) highly experimental (2) not used by a typical cheat.sh user, let alone standalone installations

[abitrolly]

#259 is the issue about removing ICU if anyone wants to help. Maybe it worth to pin it.

[mcallaway]

Until ICU is optional, you might consider some text in the "install" section that explains the current requirement.

[abitrolly]

@mcallaway I believe the problem with ICU is Windows specific and doesn't directly affect offline usage once you install appropriate toolchain to build it.

[ahillio]

I just got a Are you offline? message suggesting I post some words of encouragement here. I'm very often offline wondering something about some program language (how to check if file exists in python/lua etc). Fully offline usage would be so good <3