Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset hot fix #1109

Closed
wants to merge 6 commits into from
Closed

Dataset hot fix #1109

wants to merge 6 commits into from

Conversation

iamzoltan
Copy link
Contributor

No description provided.

@iamzoltan
Copy link
Contributor Author

iamzoltan commented Jul 5, 2024

@AdiBish

we are getting some odd errors while trying to process your changes.

------------------
data_storage_directory = "/temp"  # Note: this path must exist on your local drive
cache = VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=data_storage_directory)
============================== Failure ==============================
------------------
---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
Cell In[3], line 2
      1 data_storage_directory = "/temp"  # Note: this path must exist on your local drive
----> 2 cache = VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=data_storage_directory)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/brain_observatory/behavior/behavior_project_cache/project_cache_base.py:77, in ProjectCacheBase.from_s3_cache(cls, cache_dir, bucket_name_override)
     50 @classmethod
     51 def from_s3_cache(
     52         cls,
     53         cache_dir: Union[str, Path],
     54         bucket_name_override: Optional[str] = None
     55 ) -> "ProjectCacheBase":
     56     """instantiates this object with a connection to an s3 bucket and/or
     57     a local cache related to that bucket.
     58 
   (...)
     74 
     75     """
---> 77     fetch_api = cls.cloud_api_class().from_s3_cache(
     78 cache_dir,
     79 bucket_name=(
     80 bucket_name_overrideifbucket_name_overrideisnotNone
     81 elsecls.BUCKET_NAME),
     82 project_name=cls.PROJECT_NAME,
     83 ui_class_name=cls.__name__)
     85     return cls(fetch_api=fetch_api)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/brain_observatory/behavior/behavior_project_cache/project_apis/data_io/project_cloud_api_base.py:108, in ProjectCloudApiBase.from_s3_cache(cls, cache_dir, bucket_name, project_name, ui_class_name)
     78 @classmethod
     79 def from_s3_cache(cls, cache_dir: Union[str, Path],
     80                   bucket_name: str,
     81                   project_name: str,
     82                   ui_class_name: str) -> "ProjectCloudApiBase":
     83     """instantiates this object with a connection to an s3 bucket and/or
     84     a local cache related to that bucket.
     85 
   (...)
    106 
    107     """
--> 108     cache = S3CloudCache(cache_dir,
    109 bucket_name,
    110 project_name,
    111 ui_class_name=ui_class_name)
    112     return cls(cache)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/api/cloud_cache/cloud_cache.py:1066, in S3CloudCache.__init__(self, cache_dir, bucket_name, project_name, ui_class_name)
   1063 self._manifest = None
   1064 self._bucket_name = bucket_name
-> 1066 super().__init__(cache_dir=cache_dir,project_name=project_name,
   1067 ui_class_name=ui_class_name)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/api/cloud_cache/cloud_cache.py:391, in CloudCacheBase.__init__(self, cache_dir, project_name, ui_class_name)
    390 def __init__(self, cache_dir, project_name, ui_class_name=None):
--> 391     super().__init__(cache_dir=cache_dir,project_name=project_name,
    392 ui_class_name=ui_class_name)
    394     # what latest_manifest was the last time an OutdatedManifestWarning
    395     # was emitted
    396     self._manifest_last_warned_on = None
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/api/cloud_cache/cloud_cache.py:63, in BasicLocalCache.__init__(self, cache_dir, project_name, ui_class_name)
     57 def __init__(
     58     self,
     59     cache_dir: Union[str, Path],
     60     project_name: str,
     61     ui_class_name: Optional[str] = None
     62 ):
---> 63     os.makedirs(cache_dir,exist_ok=True)
     65     # the class users are actually interacting with
     66     # (for warning message purposes)
     67     if ui_class_name is None:
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/os.py:225, in makedirs(name, mode, exist_ok)
    223         return
    224 try:
--> 225     mkdir(name,mode)
    226 except OSError:
    227     # Cannot rely on checking for EEXIST, since the operating system
    2[28](https://github.com/NeuromatchAcademy/course-content/actions/runs/9813753170/job/27100280023#step:10:29)     # could give priority to other errors like EACCES or EROFS
    229     if not exist_ok or not path.isdir(name):
PermissionError: [Errno 13] Permission denied: '/temp'
Error: Process completed with exit code 1.

I do think this is unrelated to your changes, but it is preventing our pipeline from running. Any ideas?

@marius10p
Copy link
Collaborator

Let's wait on this one. The original creator of the notebook is supposed to upload a fixed version very soon...

@steevelaquitaine
Copy link
Contributor

@iamzoltan @marius10p @yavorska-iryna, I think that @yavorska-iryna has addressed the issues in her pull request (if she is the original creator of the notebook you mentioned @marius10p). @iamzoltan, do you think we can now accept the pull request? What about the other pull requests ?

@iamzoltan
Copy link
Contributor Author

Alright, I will close this PR.

@iamzoltan iamzoltan closed this Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants