Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(developer): work around devDependencies bug in npm #6074

Merged
merged 6 commits into from
Dec 22, 2021
Merged

fix(developer): work around devDependencies bug in npm #6074

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a5575e0
fix(developer): work around devDependencies bug in npm
mcdurdin Dec 21, 2021
23a29e2
fix(developer): tweak script for npm fix
mcdurdin Dec 21, 2021
5fccc11
chore(developer): refactor model compiler bundling into bash script
mcdurdin Dec 21, 2021
87f0d80
chore: fix bundle.sh call
mcdurdin Dec 21, 2021
2e13e7f
chore: temp set -x
mcdurdin Dec 22, 2021
984d429
fix(developer): bundle.sh rm
mcdurdin Dec 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions developer/js/.npmignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Ignore files required only in development.
build.sh
bundle.sh
source/*
tests/*
tsconfig.json
104 changes: 104 additions & 0 deletions developer/js/bundle.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
#!/usr/bin/env bash

set -eu
set -x

display_usage() {
echo "Usage: $0 --build-path path"
echo " $0 --help"
echo
echo "This script is called from the Keyman Developer installer build"
echo "and is not intended for standalone use."
echo
echo " --help displays this screen and exits"
echo " --build-path path temporary path where archive will be built"
}

BUILD_PATH=

# Process command-line arguments
while [[ $# -gt 0 ]] ; do
key="$1"
case $key in
-help|-h)
display_usage
exit
;;
--build-path)
shift
BUILD_PATH="$1"
;;
*)
echo "$0: invalid option: $key"
display_usage
exit 64
esac
shift # past the processed argument
done

if [[ -z $BUILD_PATH ]]; then
echo "Parameter --build-path is required"
display_usage
exit 64
fi

KEYMAN_WIX_TEMP_BASE="$BUILD_PATH"
KEYMAN_WIX_TEMP_MODELCOMPILER="$BUILD_PATH/ModelCompiler"
KEYMAN_MODELCOMPILER_ROOT="$KEYMAN_ROOT/developer/js"

# We use `npm pack` to extract only the aspects of the model-compiler actually needed for distribution.
# While we could use npm-bundle or similar, that adds extra, unwanted cruft; our approach gives us more
# control of the set of files distributed with the Keyman Developer installer.
#
# For users on other operating systems, node.js is a dependency and the compiler can be installed with
# `npm install @keymanapp/lexical-model-compiler`.

# We copy the files to a temp folder in order to exclude thumbs.db, .vs, etc from harvesting
rm -rf "$KEYMAN_WIX_TEMP_MODELCOMPILER"

# Step 1 - npm-pack the model compiler package, then unpack it in our bundling directory.
# This automatically strips the package to its barebones.
npm pack
mv keymanapp-lexical-model-compiler*.tgz kmlmc.tgz
mv kmlmc.tgz "$KEYMAN_WIX_TEMP_BASE"

# We extract the npm-packed version of the model compiler in order to reproduce our needed bundle.
cd "$KEYMAN_WIX_TEMP_BASE"
tar xvzf kmlmc.tgz

# Creates the directory referenced by $(KEYMAN_WIX_TEMP_MODELCOMPILER).
mv package ModelCompiler

# Cleans up the npm pack artifacts, which are no longer needed.
rm kmlmc.tgz

# Step 2 - the model compiler has one in-repo dependency that will also need to be packed.
# Managing other in-repo dependencies will be simpler; they install into ModelCompiler's
# extracted bundle.
cd "$KEYMAN_MODELCOMPILER_ROOT/node_modules/@keymanapp/models-types"
npm pack
mv keymanapp-models-types*.tgz kmtypes.tgz
mv kmtypes.tgz "$KEYMAN_WIX_TEMP_MODELCOMPILER"

# Step 3 - install just the bare essentials by using our packed local dependency, followed by
# all external production dependencies.
cd "$KEYMAN_WIX_TEMP_MODELCOMPILER"
# package-lock.json wasn't bundled; this is needed to keep dependency versions consistent.
cp "$KEYMAN_MODELCOMPILER_ROOT/package-lock.json" "$KEYMAN_WIX_TEMP_MODELCOMPILER/"

# as of npm v8.x, even though we are only working with `dependencies`, `devDependencies` is
# still checked, and as these two modules are present in devDependencies but are only
# available when in the repo path, we need to remove them before attempting to continue.
# Yuck! ref: https://github.com/npm/cli/issues/3975#issuecomment-985305678
# ref: https://github.com/npm/cli/issues/2921
# can't use npm uninstall because it depends on @keymanapp/models-types being present!
cat package.json | grep -v "@keymanapp/models-templates" | grep -v "@keymanapp/models-wordbreakers" > package.json

npm install kmtypes.tgz --production --no-optional
npm install --production --no-optional

# Clean up the npm pack artifacts for the dependencies.
rm kmtypes.tgz

# We don't need the unit tests
rm -rf "$KEYMAN_WIX_TEMP_MODELCOMPILER/tests"
53 changes: 2 additions & 51 deletions windows/src/developer/inst/download.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,60 +74,11 @@ heat-model-compiler:

!ifdef GIT_BASH_FOR_KEYMAN
$(GIT_BASH_FOR_KEYMAN) build.sh
$(GIT_BASH_FOR_KEYMAN) bundle.sh --build-path "$(KEYMAN_WIX_TEMP_BASE)"
!else
start /wait .\build.sh
start /wait .\bundle.sh --build-path "$(KEYMAN_WIX_TEMP_BASE)"
!endif
# We use `npm pack` to extract only the aspects of the model-compiler actually needed for distribution.
# While we could use npm-bundle or similar, that adds extra, unwanted cruft; our approach gives us more
# control of the set of files distributed with the Keyman Developer installer.
#
# For users on other operating systems, node.js is a dependency and the compiler can be installed with
# `npm install @keymanapp/lexical-model-compiler`.

# We copy the files to a temp folder in order to exclude thumbs.db, .vs, etc from harvesting
-rmdir /s/q $(KEYMAN_WIX_TEMP_MODELCOMPILER)

# Step 1 - npm-pack the model compiler package, then unpack it in our bundling directory.
# This automatically strips the package to its barebones.
npm pack
ren keymanapp-lexical-model-compiler*.tgz kmlmc.tgz
move kmlmc.tgz $(KEYMAN_WIX_TEMP_BASE)

# We extract the npm-packed version of the model compiler in order to reproduce our needed bundle.
cd $(KEYMAN_WIX_TEMP_BASE)

# Requires use of 7-zip, which we're already depending on elsewhere in the build process.
$(WZZIPPATH) x kmlmc.tgz
$(WZZIPPATH) x kmlmc.tar

# Creates the directory referenced by $(KEYMAN_WIX_TEMP_MODELCOMPILER).
move package ModelCompiler

# Cleans up the npm pack artifacts, which are no longer needed.
del kmlmc.tgz
del kmlmc.tar

# Step 2 - the model compiler has one in-repo dependency that will also need to be packed.
# Managing other in-repo dependencies will be simpler; they install into ModelCompiler's
# extracted bundle.
cd $(KEYMAN_MODELCOMPILER_ROOT)\node_modules\@keymanapp\models-types
npm pack
ren keymanapp-models-types*.tgz kmtypes.tgz
move kmtypes.tgz $(KEYMAN_WIX_TEMP_MODELCOMPILER)

# Step 3 - install just the bare essentials by using our packed local dependency, followed by
# all external production dependencies.
cd $(KEYMAN_WIX_TEMP_MODELCOMPILER)
# package-lock.json wasn't bundled; this is needed to keep dependency versions consistent.
copy $(KEYMAN_MODELCOMPILER_ROOT)\package-lock.json $(KEYMAN_WIX_TEMP_MODELCOMPILER)
npm install kmtypes.tgz --production --no-optional
npm install --production --no-optional

# Clean up the npm pack artifacts for the dependencies.
del kmtypes.tgz

# We don't need the unit tests
-rmdir /s/q $(KEYMAN_WIX_TEMP_MODELCOMPILER)\tests

# Build the .wxs file
cd $(ROOT)\src\developer\inst
Expand Down