Silent YAML parse failure prevents working copy of mobile-hub-project.yaml from being updated #16

aassaad · 2017-12-15T21:09:30Z

TL;DR

lib/aws-operations/mobile-yaml-schema.js is missing a type definition, causing YAML a parse failure which is silently eaten when syncing mobilehub-project.yml from awsmobilejs/#current-backend-info to awsmobilejs/backend
could be the cause of Following Get Started tutorial fails #10 and run erases imported API configuration retrieved with pull #12?
easy to fix the parse failure by adding the missing type definition
don't know if that's a complete fix
a global verbose/debug flag would help identify silent failures

Steps

Create project in the Mobile Hub console, select "web" type and check enable web hosting
From the console, add User Sign-in using Cognito w/ email+password
In an empty directory, run awsmobile init <project id> , leave all values as default when prompted

Comparing mobile-hub-project.yml in awsmobile/backend and awsmobile/#current-backend-info/ gives:

$ git diff -w awsmobilejs/#current-backend-info/mobile-hub-project.yml awsmobilejs/backend/mobile-hub-project.yml
diff --git a/awsmobilejs/#current-backend-info/mobile-hub-project.yml b/awsmobilejs/backend/mobile-hub-project.yml
index 81760fe..aeb1bac 100644
--- a/awsmobilejs/#current-backend-info/mobile-hub-project.yml
+++ b/awsmobilejs/backend/mobile-hub-project.yml
@@ -1,6 +1,4 @@
 --- !com.amazonaws.mobilehub.v0.Project
-apps:
-  web: !com.amazonaws.mobilehub.v0.AppDO {}
 features:
   content-delivery: !com.amazonaws.mobilehub.v0.ContentDelivery
     attributes:
@@ -11,6 +9,5 @@ features:
   mobile-analytics: !com.amazonaws.mobilehub.v0.Pinpoint
     components:
       analytics: !com.amazonaws.mobilehub.v0.PinpointAnalytics {}
-  sign-in: !com.amazonaws.mobilehub.v0.SignIn {}
-name: test5
+name: test5-2017-12-15-13-52-15
 region: us-east-1

From the console, add NoSQL database, select "Locations" example
Hit "Integrate backend"
As instructed, run awsmobile pull, and answer yes when prompted to sync changes from #current-backend-info to the dev copy
diff again:

--- a/awsmobilejs/#current-backend-info/mobile-hub-project.yml
+++ b/awsmobilejs/backend/mobile-hub-project.yml
@@ -1,6 +1,4 @@
 --- !com.amazonaws.mobilehub.v0.Project
-apps:
-  web: !com.amazonaws.mobilehub.v0.AppDO {}
 features:
   content-delivery: !com.amazonaws.mobilehub.v0.ContentDelivery
     attributes:
@@ -8,51 +6,8 @@ features:
       visibility: public-global
     components:
       release: !com.amazonaws.mobilehub.v0.Bucket {}
-  database: !com.amazonaws.mobilehub.v0.Database
-    components:
-      database-nosql: !com.amazonaws.mobilehub.v0.NoSQLDatabase
-        tables:
-          - !com.amazonaws.mobilehub.v0.NoSQLTable
-            attributes:
-            <excluded for space>
   mobile-analytics: !com.amazonaws.mobilehub.v0.Pinpoint
     components:
       analytics: !com.amazonaws.mobilehub.v0.PinpointAnalytics {}
-  sign-in: !com.amazonaws.mobilehub.v0.SignIn
-    attributes:
-      enabled: true
-      optional-sign-in: false
-    components:
-      sign-in-user-pools: !com.amazonaws.mobilehub.v0.UserPoolsIdentityProvider
-        attributes:
-          alias-attributes:
-            - preferred_username
-            - email
-          mfa-configuration: OFF
-          name: userpool
-          password-policy: !com.amazonaws.mobilehub.ConvertibleMap
-            min-length: "8"
-            require-lower-case: true
-            require-numbers: true
-            require-symbols: true
-            require-upper-case: true
-name: test5
+name: test5-2017-12-15-13-52-15

Clearly those files are not synced...

So I poked around for the code prompting whether to sync (lib/backend-operations/backend-info-manager.onSyncComplete), and found that the backendProject object that's conditionally passed to executeSyncToDevBackend is undefined.

Tracing that undefined value upstream through lib/backend-operations/ops-project.syncCurrentBackendInfo led to a call to lib/aws-operations/mobile-yaml-ops.loadYml, and in there I found a try/catch with several console.log statements hidden behind a "verbose" flag, which was eating all the errors.

Setting verbose to true results in the following being printed to console:

$ awsmobile pull

retrieving the latest backend awsmobile project information
awsmobile project's details logged at: awsmobilejs/#current-backend-info/backend-details.json
awsmobile project's access information logged at: awsmobilejs/#current-backend-info/aws-exports.js
awsmobile project's specifications logged at: awsmobilejs/#current-backend-info/mobile-hub-project.yml
error occured in parsing the yaml file:
/Users/alec/git/test5/awsmobilejs/#current-backend-info/mobile-hub-project.yml
{ YAMLException: unknown tag !<!com.amazonaws.mobilehub.v0.AppDO> at line 3, column 44:
     ... .amazonaws.mobilehub.v0.AppDO {}
                                         ^
    at generateError (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:165:10)
    at throwError (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:171:9)
    at composeNode (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1401:7)
    at readBlockMapping (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1062:11)
    at composeNode (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1332:12)
    at readBlockMapping (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1062:11)
    at composeNode (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1332:12)
    at readDocument (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1492:3)
    at loadDocuments (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1548:5)
    at load (/Users/alec/.nvm/versions/node/v8.9.1/lib/node_modules/awsmobile-cli/node_modules/js-yaml/lib/js-yaml/loader.js:1569:19)
  name: 'YAMLException',
  reason: 'unknown tag !<!com.amazonaws.mobilehub.v0.AppDO>',
  mark:
   Mark {
     name: null,
     buffer: '--- !com.amazonaws.mobilehub.v0.Project\napps: \n  web: !com.amazonaws.mobilehub.v0.AppDO {}\nfeatures: \n  content-delivery: !com.amazonaws.mobilehub.v0.ContentDelivery\n    attributes: \n      enabled: true\n      visibility: public-global\n    components: \n      release: !com.amazonaws.mobilehub.v0.Bucket {}\n  database: !com.amazonaws.mobilehub.v0.Database\n    components: \n      database-nosql: !com.amazonaws.mobilehub.v0.NoSQLDatabase\n        tables: \n          - !com.amazonaws.mobilehub.v0.NoSQLTable\n            attributes: \n              email: S\n              org: S\n              userId: S\n              username: S\n            hashKeyName: userId\n            hashKeyType: S\n            indexes: \n              - !com.amazonaws.mobilehub.v0.NoSQLIndex\n                hashKeyName: org\n                hashKeyType: S\n                indexName: org-email\n                rangeKeyName: email\n                rangeKeyType: S\n              - !com.amazonaws.mobilehub.v0.NoSQLIndex\n                hashKeyName: email\n                hashKeyType: S\n                indexName: email\n                rangeKeyName: ""\n                rangeKeyType: ""\n              - !com.amazonaws.mobilehub.v0.NoSQLIndex\n                hashKeyName: username\n                hashKeyType: S\n                indexName: username\n                rangeKeyName: ""\n                rangeKeyType: ""\n            rangeKeyName: ""\n            rangeKeyType: ""\n            tableName: ___DYNAMIC_PREFIX___-users-test4\n            tablePrivacy: protected\n  mobile-analytics: !com.amazonaws.mobilehub.v0.Pinpoint\n    components: \n      analytics: !com.amazonaws.mobilehub.v0.PinpointAnalytics {}\n  sign-in: !com.amazonaws.mobilehub.v0.SignIn\n    attributes: \n      enabled: true\n      optional-sign-in: false\n    components: \n      sign-in-user-pools: !com.amazonaws.mobilehub.v0.UserPoolsIdentityProvider\n        attributes: \n          alias-attributes: \n            - email\n            - phone_number\n          mfa-configuration: OPTIONAL\n          name: userpool\n          password-policy: !com.amazonaws.mobilehub.ConvertibleMap\n            min-length: "8"\n            require-lower-case: true\n            require-numbers: true\n            require-symbols: true\n            require-upper-case: true\nname: test4\nregion: us-east-1\n\u0000',
     position: 90,
     line: 2,
     column: 43 },
  message: 'unknown tag !<!com.amazonaws.mobilehub.v0.AppDO> at line 3, column 44:\n     ... .amazonaws.mobilehub.v0.AppDO {}\n                                         ^' }

I found the YAML schema definition in lib/aws-operations/mobile-yaml-schema.js, where adding a jsyaml.Type definition for AppDO stopped the parse failure and allowed the files to sync successfully:

---lib/aws-operations/mobile-yaml-schema.js
+++lib/aws-operations/mobile-yaml-schema.js
@@ -109,6 +109,16 @@ let APIYamlType = new jsyaml.Type('!com.amazonaws.mobilehub.v0.API', {
     instanceOf: API
 })

+function AppDO(data) {
+    Object.assign(this, data)
+    this[typeProperty] = 'AppDO'
+}
+let AppDOYamlType = new jsyaml.Type('!com.amazonaws.mobilehub.v0.AppDO', {
+    kind: 'mapping',
+    construct: data => new AppDO(data),
+    instanceOf: AppDO
+})
+
 function Bot(data) {
     Object.assign(this, data)
     this[typeProperty] = 'Bot'
@@ -421,6 +431,7 @@ let AWS_MOBILE_YAML_SCHEMA = jsyaml.Schema.create([
     FileUploaderYamlType,
     SharedComponentFactoryYamlType,
     APIYamlType,
+    AppDOYamlType,
     BotYamlType,
     BotsYamlType,
     BucketYamlType,
@@ -467,6 +478,7 @@ function reConstruct(typeName, obj){
         case 'FileUploader': result = new FileUploader(obj); break;
         case 'SharedComponentFactory': result = new SharedComponentFactory(obj); break;
         case 'API': result = new API(obj); break;
+        case 'AppDO': result = new AppDO(obj); break;
         case 'Bot': result = new Bot(obj); break;
         case 'Bots': result = new Bots(obj); break;
         case 'Bucket': result = new Bucket(obj); break;

Since I don't know the broader implications of my workaround (or of the fact that definitions were missing in the first place, for that matter) I'm not comfortable working with the tool before an official fix, but I'm happy to submit a PR with this change plus some tests if it's that simple.

I'd also love to see a --verbose or --debug flag or environment variable to prevent silent failures like this, and could put a PR together for that as well if helpful

The text was updated successfully, but these errors were encountered:

fkattan · 2017-12-19T20:31:30Z

I'm having the same issue.

awsmobile pull

executes uneventfully, but sync is not complete and trying to run:

awsmobile features or

awsmobile database enable --prompt

return with error:

can not locate mobile-hub-project file inside awsmobilejs/backend
local backend specs appear to be corrupted
please restored the local backend specs to valid state before continuing

sindreu · 2017-12-23T23:35:47Z

I also have the same issue.

pcolazurdo · 2017-12-29T15:28:22Z

similar situation

Just running:

awsmobile init
awsmobile publish

gives the following error

can not locate mobile-hub-project file inside awsmobilejs/backend
local backend specs appear to be corrupted
please restored the local backend specs to valid state before continuing

the directory awsmobilejs/backend is empty

# awsmobile -V

1.0.6

mlabieniec · 2017-12-29T20:14:01Z

@pcolazurdo @sindreu can you let us know what your node --version and OS as well please?

typemismatch · 2017-12-30T01:52:48Z

I have the same issue - nothing works, brand new project. awsmobile - v1.0.6, nodejs - v9.2.1, npm - v4.6.1.

I get the same error and actually have nothing in /backend ....

mlabieniec · 2017-12-30T05:26:18Z

@typemismatch we are aware of an issue with v9.2.1. This seems to be directly related to fsevents specifically for v9.

pcolazurdo · 2017-12-30T10:26:16Z

@mlabieniec sure, here it is:

uname -a

Linux ip-172-31-26-64 4.9.58-18.55.amzn1.x86_64 #1 SMP Thu Nov 2 04:38:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

node -v

v8.9.1

npm -v

5.6.0

ryanh-ai · 2018-01-02T17:54:17Z

Having same issue with same versions....

elorzafe · 2018-01-02T19:24:49Z

I couldnt reproduce that issue. I installed on linux and mac and it works fine. I guess it could be a credentials issue. Can you create a project using MobileHub console?
https://console.aws.amazon.com/mobilehub/home#/

elorzafe · 2018-01-02T19:32:05Z

It will be helpful to diagnose the issue if someone post the folder structure where are trying to execute awsmobile push.

elorzafe · 2018-01-02T19:59:46Z

I reproduced this error...

can not locate mobile-hub-project file inside awsmobilejs/backend
local backend specs appear to be corrupted
please restored the local backend specs to valid state before continuing

...

I fix it by executing awsmobile pull and on sync question answer yes.

pcolazurdo · 2018-01-02T20:17:01Z

This is the content of my structure:

.
./src
./src/aws-exports.js
./.gitignore
./awsmobilejs
./awsmobilejs/#current-backend-info
./awsmobilejs/#current-backend-info/mobile-hub-project.yml
./awsmobilejs/#current-backend-info/aws-exports.js
./awsmobilejs/#current-backend-info/backend-details.json
./awsmobilejs/.awsmobile
./awsmobilejs/.awsmobile/info
./awsmobilejs/.awsmobile/info/project-info.json
./awsmobilejs/.awsmobile/backend-build
./awsmobilejs/backend

pcolazurdo · 2018-01-02T20:17:50Z

I was able to create the project using the console with no issue at all (same credentials) but the pull / push process fails

luizdanieldesouza · 2018-01-04T02:33:16Z

I'm also having the same issue.

" can not locate mobile-hub-project file inside awsmobilejs/backend
local backend specs appear to be corrupted
please restored the local backend specs to valid state before continuing"

Any help will be much appreciated..

mlabieniec · 2018-01-04T04:01:02Z

@pcolazurdo I tried the same environment and can't seem to reproduce. Can give a try:

npm uninstall -g awsmobile-cli
npm cache clean --force
npm install -g awsmobile-cli

just to eliminate the possibility of a corrupt npm environment. I'm wondering if perhaps it's a local install of the global module that may have failed....Also, what is your project type i.e. create-react-app or similar?

pcolazurdo · 2018-01-04T16:27:49Z

Done that, still same issue ... when I install I can see an error that says:

[execsync v1.0.2] Attempting to compile native extensions.
[execSync v1.0.2]
Native code compile failed!!

but from what I can see there is no dependency from awsmobile into it (only the code coverage tests) - trying to fix the error with ExecSync seem to indicate that it won't work for node >0.10

Darkade · 2018-01-05T21:56:38Z

I am having the same problem when running awsmobile database enable --prompt

This is what I'm using:

$ uname -a
Darwin Callisto.local 16.7.0 Darwin Kernel Version 16.7.0: Mon Nov 13 21:56:25 PST 2017; root:xnu-3789.72.11~1/RELEASE_X86_64 x86_64
$ awsmobile -V
1.0.6
$ node -v
v9.3.0
$ yarn -v
np1.3.2
$ npm -v
4.6.1

If I copy the awsmobilejs/#current-backend-info/mobile-hub-project.yml to awsmobilejs/backend/mobile-hub-project.yml the error changes to

$ awsmobile cloud-api enable --prompt

local backend specs appear to be corrupted
please restore the local backend specs before continuing

So the can not locate mobile-hub-project file inside awsmobilejs/backend line is gone

UnleashedMind · 2018-01-08T23:18:53Z

Thanks. This issue is fixed by the latest update (1.0.7).

UnleashedMind · 2018-01-08T23:48:02Z

Please follow these steps:

npm update -g awsmobile-cli
awsmobile -V (and make sure the version is 1.0.7)
cd your-project-folder
awsmobile pull

At this point the yaml file in the backend is restored. You should be able to continue your work as usual

aassaad changed the title ~~Silent YAML parse failure prevents working copy of mobilehub-from being updated~~ Silent YAML parse failure prevents working copy of mobile-hub-project.yaml from being updated Dec 15, 2017

mlabieniec added the investigating label Dec 23, 2017

mlabieniec assigned elorzafe Dec 29, 2017

Darkade mentioned this issue Jan 8, 2018

Can't run React sample app - local backend specs appear to be corrupted #33

Closed

UnleashedMind closed this as completed Jan 8, 2018

UnleashedMind reopened this Jan 8, 2018

UnleashedMind closed this as completed Jan 8, 2018

ghost mentioned this issue Jan 9, 2018

run erases imported API configuration retrieved with pull #12

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Silent YAML parse failure prevents working copy of mobile-hub-project.yaml from being updated #16

Silent YAML parse failure prevents working copy of mobile-hub-project.yaml from being updated #16

aassaad commented Dec 15, 2017

fkattan commented Dec 19, 2017

sindreu commented Dec 23, 2017

pcolazurdo commented Dec 29, 2017 •

edited

Loading

mlabieniec commented Dec 29, 2017

typemismatch commented Dec 30, 2017

mlabieniec commented Dec 30, 2017

pcolazurdo commented Dec 30, 2017

ryanh-ai commented Jan 2, 2018

elorzafe commented Jan 2, 2018

elorzafe commented Jan 2, 2018

elorzafe commented Jan 2, 2018

pcolazurdo commented Jan 2, 2018

pcolazurdo commented Jan 2, 2018

luizdanieldesouza commented Jan 4, 2018

mlabieniec commented Jan 4, 2018

pcolazurdo commented Jan 4, 2018

Darkade commented Jan 5, 2018

UnleashedMind commented Jan 8, 2018

UnleashedMind commented Jan 8, 2018

Silent YAML parse failure prevents working copy of mobile-hub-project.yaml from being updated #16

Silent YAML parse failure prevents working copy of mobile-hub-project.yaml from being updated #16

Comments

aassaad commented Dec 15, 2017

TL;DR

Steps

fkattan commented Dec 19, 2017

sindreu commented Dec 23, 2017

pcolazurdo commented Dec 29, 2017 • edited Loading

mlabieniec commented Dec 29, 2017

typemismatch commented Dec 30, 2017

mlabieniec commented Dec 30, 2017

pcolazurdo commented Dec 30, 2017

uname -a

node -v

npm -v

ryanh-ai commented Jan 2, 2018

elorzafe commented Jan 2, 2018

elorzafe commented Jan 2, 2018

elorzafe commented Jan 2, 2018

pcolazurdo commented Jan 2, 2018

pcolazurdo commented Jan 2, 2018

luizdanieldesouza commented Jan 4, 2018

mlabieniec commented Jan 4, 2018

pcolazurdo commented Jan 4, 2018

Darkade commented Jan 5, 2018

UnleashedMind commented Jan 8, 2018

UnleashedMind commented Jan 8, 2018

pcolazurdo commented Dec 29, 2017 •

edited

Loading