Support Mac OS. Help wanted 🙏🍎 #178

kantord · 2023-09-07T12:01:03Z

At all steps, SeaGOAT is developed with guaranteeing Mac OS support. That being said, as I don't have a Mac OS device and I don't have expertise regarding Mac OS development workflows, I have not been able to verify that it works properly.

Help wanted!

Here is a rough checklist of the current status of Mac OS support:

Enable CI checks for Mac OS
Make sure that all CI tests pass for Mac
Manually test SeaGOAT on Mac
Document installation steps on Mac
Make sure that any required Mac-specific features are implemented
Make sure that it's easy to install on Mac using common tools (homebrew?)

Currently the main problem is that I have not been able to try and run SeaGOAT manually on OSX.

Also I have discovered that some of the tests are failing, so I created a CI file specifically for OSX that only checks some of the tests are are passing without issue:

SeaGOAT/.github/workflows/test-mac-os.yml

Lines 56 to 58 in 1edae8e

    
                       - name: Run pytest 
        
                         run: | 
        
                             poetry run pytest tests/test_repository.py -vvs --timeout=60

For comparison the Linux tests look like this:

SeaGOAT/.github/workflows/test.yml

Lines 56 to 58 in 1edae8e

    
                       - name: Run pytest 
        
                         run: | 
        
                             poetry run pytest . -vvs --timeout=60

kantord · 2023-09-12T17:12:17Z

update, now all the tests pass on Mac OS in the pipeline ✔️ 🚀 still hasn't been able to confirm by manually testing, but now everything should work well in Mac

YussufSassi · 2023-09-20T13:16:40Z

Can confirm it's working on OSX

Here's what I tested it on:
Device: MacBook Pro 2023, M2 Max, 32GB RAM
MacOS Version: Ventura 13.5.2 (22G91)

Mac-specific installation steps:

brew install pipx
brew install ripgrep

After that I followed the installation steps in the docs (including bat, which I installed via brew)

Tested on a small project, ran quickly and there were no issues

On a large project with about 160,000 tasks in the indexing process it doesn't seem to use the full power of the cpu (not sure if that's intentional). It's sitting at around 40-43% usage while indexing the project.

forest-d · 2023-09-20T13:32:16Z

👍 can also confirm it's working on OSX.

Device: MacBook Pro 2020, M1, 8GB RAM
MacOS: Ventura 13.5.1

Here's what I did:

Created a new conda env conda create -n seagoat python=3.11
Installed dependencies conda install ripgrep && brew install bat && pip install seagoat
Ran seagoat server start . on one of my repos
Ran gt "my query" and observed the expected bat output

kantord · 2023-09-20T14:15:56Z

Can confirm it's working on OSX

Here's what I tested it on: Device: MacBook Pro 2023, M2 Max, 32GB RAM MacOS Version: Ventura 13.5.2 (22G91)

Mac-specific installation steps:
brew install pipx
brew install ripgrep
After that I followed the installation steps in the docs (including bat, which I installed via brew)

Tested on a small project, ran quickly and there were no issues

On a large project with about 160,000 tasks in the indexing process it doesn't seem to use the full power of the cpu (not sure if that's intentional). It's sitting at around 40-43% usage while indexing the project.

Thank you for the feedback, I'm glad to hear that it's working!

Regarding the CPU use that is intentional, since the process can take a long time for a large project, I designed it in such a way that it avoids blocking your computer while processing to avoid any disruption

kantord · 2023-09-20T14:16:15Z

👍 can also confirm it's working on OSX.

Device: MacBook Pro 2020, M1, 8GB RAM MacOS: Ventura 13.5.1

Here's what I did:

Created a new conda env conda create -n seagoat python=3.11

Installed dependencies conda install ripgrep && brew install bat && pip install seagoat

Ran seagoat server start . on one of my repos

Ran gt "my query" and observed the expected bat output

Thanks! Glad to hear it works both on M1 and M2

monopoly-db · 2023-09-20T20:50:34Z

I have a Intel Mac and I get this from seagoat-server on my C++ git repo:

2023-09-20 16:40:46,448 Creating server...
2023-09-20 16:40:46,449 Starting worker thread...
2023-09-20 16:40:46,456 Serving on http://0.0.0.0:52647
2023-09-20 16:40:47,972 Checking repository for new changes
2023-09-20 16:41:00,631 Task queue depth is 1
warning: exhaustive rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 33109 and retry the command.
Analyzing source code: 0it [00:00, ?it/s]
2023-09-20 16:41:33,563 Analyzed the minimum number of chunks needed to operate. 
2023-09-20 16:41:33,563 Analyzed all chunks!
2023-09-20 16:41:33,564 Handling task: query
Exception in thread Thread-1 (_worker_function):
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.11/site-packages/seagoat/queue/base_queue.py", line 79, in _worker_function
    self._handle_task(context, task)
  File "/usr/local/lib/python3.11/site-packages/seagoat/queue/base_queue.py", line 66, in _handle_task
    result = handler(context, *task.args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/seagoat/queue/task_queue.py", line 83, in handle_query
    results = context["seagoat_engine"].get_results(kwargs["limit_clue"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/seagoat/engine.py", line 208, in get_results
    sorted(
  File "/usr/local/lib/python3.11/site-packages/seagoat/engine.py", line 212, in <lambda>
    + 0.3 * normalize_file_position(top_files[x.path])
                                    ~~~~~~~~~^^^^^^^^
KeyError: 'ver/version.cc'

Hardware:

  Model Name:	MacBook Pro
  Model Identifier:	MacBookPro16,1
  Processor Name:	8-Core Intel Core i9
  Processor Speed:	2.3 GHz
  Number of Processors:	1
  Total Number of Cores:	8
  L2 Cache (per Core):	256 KB
  L3 Cache:	16 MB
  Hyper-Threading Technology:	Enabled
  Memory:	16 GB
  System Firmware Version:	1968.140.7.0.0 (iBridge: 20.16.6072.0.0,0)
  OS Loader Version:	540.120.3~22

Running MacOS Monterey version 12.6.9

kantord · 2023-09-25T07:18:17Z

Is this a Mac specific issue? Did any of you face something similar? #243 (comment)

cschubiner · 2024-09-25T16:53:06Z

getting this issue on mac:

(base) ~ ❯❯❯ seagoat-server start ~/canal
2024-09-25 12:52:30,832 Creating server...
2024-09-25 12:52:30,833 Starting worker thread...
2024-09-25 12:52:30,835 Serving on http://0.0.0.0:49815
2024-09-25 12:52:31,434 Checking repository for new changes
Exception in thread Thread-1 (_worker_function):
Traceback (most recent call last):
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/queue/base_queue.py", line 78, in _worker_function
    task = self._task_queue.get(timeout=0.1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/queue/base_queue.py", line 83, in _worker_function
    self.handle_maintenance(context)
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/queue/task_queue.py", line 68, in handle_maintenance
    remaining_chunks_to_analyze = context["seagoat_engine"].analyze_codebase(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/engine.py", line 77, in analyze_codebase
    fetcher["cache_repo"]()
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 111, in cache_repo
    memory_cache.rebuild()
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 82, in rebuild
    self._build_cache_file()
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 60, in _build_cache_file
    for formattted_cache_line in self._generate_cache_lines():
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 52, in _generate_cache_lines
    for file, line_number, line in self._iterate_lines_to_cache():
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 41, in _iterate_lines_to_cache
    for file in self._iterate_files_to_cache():
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 37, in _iterate_files_to_cache
    for file, _ in self.repository.top_files():
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/repository.py", line 119, in top_files
    (self.get_file(filename), score)
     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/repository.py", line 134, in get_file
    self.get_file_object_id(filename),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/repository.py", line 45, in get_file_object_id
    subprocess.check_output(
IndexError: list index out of range

kantord · 2024-09-30T13:59:43Z

getting this issue on mac:

(base) ~ ❯❯❯ seagoat-server start ~/canal
2024-09-25 12:52:30,832 Creating server...
2024-09-25 12:52:30,833 Starting worker thread...
2024-09-25 12:52:30,835 Serving on http://0.0.0.0:49815
2024-09-25 12:52:31,434 Checking repository for new changes
Exception in thread Thread-1 (_worker_function):
Traceback (most recent call last):
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/queue/base_queue.py", line 78, in _worker_function
    task = self._task_queue.get(timeout=0.1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/queue.py", line 179, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/queue/base_queue.py", line 83, in _worker_function
    self.handle_maintenance(context)
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/queue/task_queue.py", line 68, in handle_maintenance
    remaining_chunks_to_analyze = context["seagoat_engine"].analyze_codebase(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/engine.py", line 77, in analyze_codebase
    fetcher["cache_repo"]()
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 111, in cache_repo
    memory_cache.rebuild()
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 82, in rebuild
    self._build_cache_file()
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 60, in _build_cache_file
    for formattted_cache_line in self._generate_cache_lines():
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 52, in _generate_cache_lines
    for file, line_number, line in self._iterate_lines_to_cache():
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 41, in _iterate_lines_to_cache
    for file in self._iterate_files_to_cache():
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/sources/ripgrep.py", line 37, in _iterate_files_to_cache
    for file, _ in self.repository.top_files():
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/repository.py", line 119, in top_files
    (self.get_file(filename), score)
     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/repository.py", line 134, in get_file
    self.get_file_object_id(filename),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/canal/Library/Application Support/pipx/venvs/seagoat/lib/python3.12/site-packages/seagoat/repository.py", line 45, in get_file_object_id
    subprocess.check_output(
IndexError: list index out of range

seems like it might be a generic bug, not mac related, but I'm not 100% sure. Is the repo you are using it on public?

kantord added help wanted Extra attention is needed up for grabs labels Sep 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Mac OS. Help wanted 🙏🍎 #178

Support Mac OS. Help wanted 🙏🍎 #178

kantord commented Sep 7, 2023 •

edited

Loading

kantord commented Sep 12, 2023

YussufSassi commented Sep 20, 2023 •

edited

Loading

forest-d commented Sep 20, 2023 •

edited

Loading

kantord commented Sep 20, 2023

kantord commented Sep 20, 2023

monopoly-db commented Sep 20, 2023

kantord commented Sep 25, 2023

cschubiner commented Sep 25, 2024

kantord commented Sep 30, 2024

Support Mac OS. Help wanted 🙏🍎 #178

Support Mac OS. Help wanted 🙏🍎 #178

Comments

kantord commented Sep 7, 2023 • edited Loading

kantord commented Sep 12, 2023

YussufSassi commented Sep 20, 2023 • edited Loading

forest-d commented Sep 20, 2023 • edited Loading

kantord commented Sep 20, 2023

kantord commented Sep 20, 2023

monopoly-db commented Sep 20, 2023

kantord commented Sep 25, 2023

cschubiner commented Sep 25, 2024

kantord commented Sep 30, 2024

kantord commented Sep 7, 2023 •

edited

Loading

YussufSassi commented Sep 20, 2023 •

edited

Loading

forest-d commented Sep 20, 2023 •

edited

Loading