Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File searching bug #1678

Merged
merged 14 commits into from
Sep 27, 2023
Merged

File searching bug #1678

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d52619c
fix file search
kbwestfall Sep 21, 2023
f99f431
rm glob
kbwestfall Sep 21, 2023
53d5bb7
CHANGES + docs
kbwestfall Sep 21, 2023
a1f9b84
missed release doc
kbwestfall Sep 21, 2023
7260021
name file returned by find_single_file
kbwestfall Sep 25, 2023
bfb1bcd
minor function clean-up
kbwestfall Sep 25, 2023
7784c5a
minor function clean-up
kbwestfall Sep 25, 2023
d224139
list bug fix
kbwestfall Sep 25, 2023
b73d749
sorted glob
kbwestfall Sep 25, 2023
1065f7c
dev doc update
kbwestfall Sep 26, 2023
0e07990
typo
kbwestfall Sep 26, 2023
ecd8619
Merge branch 'develop' into filesearch
kbwestfall Sep 26, 2023
2b6db67
add required argument to find_single_file
kbwestfall Sep 27, 2023
1ab1ef0
comment on checking the doc formatting
kbwestfall Sep 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGES.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@

1.14.1dev
---------

PLEASE ADD TO doc/releases/1.14.1dev.rst!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea!



1.14.0 (18 Sep 2023)
--------------------

Expand Down
2 changes: 1 addition & 1 deletion doc/help/run_pypeit.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
usage: run_pypeit [-h] [-v VERBOSITY] [-r REDUX_PATH] [-m] [-s] [-o] [-c]
pypeit_file

## PypeIt : The Python Spectroscopic Data Reduction Pipeline v1.13.1.dev859+gf736f5f47
## PypeIt : The Python Spectroscopic Data Reduction Pipeline v1.14.0
##
## Available spectrographs include:
## bok_bc, gemini_flamingos1, gemini_flamingos2, gemini_gmos_north_e2v,
Expand Down
29 changes: 29 additions & 0 deletions doc/releases/1.14.1dev.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@

Version 1.14.1dev
=================

Dependency Changes
------------------

Functionality/Performance Improvements and Additions
----------------------------------------------------

Instrument-specific Updates
---------------------------

Script Changes
--------------

Datamodel Changes
-----------------

Under-the-hood Improvements
---------------------------

Bug Fixes
---------
Comment on lines +5 to +24
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having all of these headings for the developer to use is very helpful (rather than the semi-chronological order in CHANGES.rst).


- Fixed bug associated with finding more than one file with the same name (but
presumably different extensions).


4 changes: 4 additions & 0 deletions doc/whatsnew.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ For a detailed log of code edits (including releases earlier than version

----

.. include:: releases/1.14.1dev.rst

----

.. include:: releases/1.14.0.rst

----
Expand Down
10 changes: 4 additions & 6 deletions pypeit/spectrographs/jwst_nirspec.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@

.. include:: ../include/links.rst
"""
import glob
import numpy as np

from pypeit import msgs
from pypeit import telescopes
from pypeit import utils
from pypeit.core import framematch
from pypeit import io
from pypeit.par import pypeitpar
Expand Down Expand Up @@ -288,11 +288,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil)))

fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
msgs.error(f'No files matching pattern: {raw_file}')
kbwestfall marked this conversation as resolved.
Show resolved Hide resolved

# Read
msgs.info("Reading JWST/NIRSpec file: {:s}".format(fil[0]))
Expand Down
18 changes: 8 additions & 10 deletions pypeit/spectrographs/keck_kcwi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@
.. include:: ../include/links.rst
"""

import glob

from IPython import embed

import numpy as np
Expand All @@ -16,8 +14,10 @@
from astropy.time import Time
from astropy.coordinates import SkyCoord, EarthLocation
from scipy.optimize import curve_fit

from pypeit import msgs
from pypeit import telescopes
from pypeit import utils
from pypeit import io
from pypeit.core import parse
from pypeit.core import procimg
Expand Down Expand Up @@ -969,10 +969,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil), raw_file))
fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
msgs.error(f'No files matching pattern: {raw_file}')

# Read
msgs.info("Reading KCWI file: {:s}".format(fil[0]))
Expand Down Expand Up @@ -1282,10 +1281,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil), raw_file))
fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
kbwestfall marked this conversation as resolved.
Show resolved Hide resolved
msgs.error(f'No files matching pattern: {raw_file}')

# Read
msgs.info("Reading KCWI file: {:s}".format(fil[0]))
Expand Down
9 changes: 4 additions & 5 deletions pypeit/spectrographs/keck_lris.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

.. include:: ../include/links.rst
"""
import glob
import os

from IPython import embed
Expand All @@ -19,6 +18,7 @@

from pypeit import msgs
from pypeit import telescopes
from pypeit import utils
from pypeit import io
from pypeit.core import parse
from pypeit.core import framematch
Expand Down Expand Up @@ -433,10 +433,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil)))
fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
msgs.error(f'No files matching pattern: {raw_file}')

# Read
msgs.info("Reading LRIS file: {:s}".format(fil[0]))
Expand Down
9 changes: 4 additions & 5 deletions pypeit/spectrographs/lbt_mods.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@

.. include:: ../include/links.rst
"""
import glob
import numpy as np
from astropy.io import fits

from pypeit import msgs
from pypeit import telescopes
from pypeit import utils
from pypeit import io
from pypeit.core import framematch
from pypeit.par import pypeitpar
Expand Down Expand Up @@ -206,10 +206,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil)))
fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
msgs.error(f'No files matching pattern: {raw_file}')

# Read
msgs.info("Reading LBT/MODS file: {:s}".format(fil[0]))
Expand Down
10 changes: 4 additions & 6 deletions pypeit/spectrographs/mmt_binospec.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,11 @@

.. include:: ../include/links.rst
"""
import glob

import numpy as np

from pypeit import msgs
from pypeit import telescopes
from pypeit import utils
from pypeit import io
from pypeit.core import framematch
from pypeit.spectrographs import spectrograph
Expand Down Expand Up @@ -381,10 +380,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil)), raw_file)
fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
msgs.error(f'No files matching pattern: {raw_file}')

# Read
msgs.info("Reading BINOSPEC file: {:s}".format(fil[0]))
Expand Down
9 changes: 4 additions & 5 deletions pypeit/spectrographs/mmt_bluechannel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@

.. include:: ../include/links.rst
"""
import glob
import numpy as np
from astropy.io import fits
from astropy.time import Time

from pypeit import msgs
from pypeit import telescopes
from pypeit import utils
from pypeit.core import framematch
from pypeit.par import pypeitpar
from pypeit.spectrographs import spectrograph
Expand Down Expand Up @@ -493,10 +493,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil)))
fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
msgs.error(f'No files matching pattern: {raw_file}')

# Read FITS image
msgs.info("Reading MMT Blue Channel file: {:s}".format(fil[0]))
Expand Down
10 changes: 4 additions & 6 deletions pypeit/spectrographs/mmt_mmirs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@

.. include:: ../include/links.rst
"""
import glob

import numpy as np
from scipy.signal import savgol_filter

Expand All @@ -14,6 +12,7 @@

from pypeit import msgs
from pypeit import telescopes
from pypeit import utils
from pypeit import io
from pypeit.core import parse
from pypeit.core import framematch
Expand Down Expand Up @@ -349,10 +348,9 @@ def get_rawimage(self, raw_file, det):
(1-indexed) number of the amplifier used to read each detector
pixel. Pixels unassociated with any amplifier are set to 0.
"""
# Check for file; allow for extra .gz, etc. suffix
fil = glob.glob(raw_file + '*')
if len(fil) != 1:
msgs.error("Found {:d} files matching {:s}".format(len(fil)))
fil = utils.find_single_file(f'{raw_file}*')
if fil is None:
msgs.error(f'No files matching pattern: {raw_file}')

# Read
msgs.info("Reading MMIRS file: {:s}".format(fil[0]))
Expand Down