Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Custom policies IAM Policies for Lambda and Containers #8068

Merged
merged 27 commits into from
Oct 4, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
fc5fef8
Custom policy implementation
luhanamz Aug 12, 2021
85f7eea
feat: add custom policies file to function and API container
luhanamz Aug 16, 2021
5ae975a
feat: changes for first PR
luhanamz Aug 16, 2021
c6ceb2f
feat: Some changes according to the PR comments
luhanamz Aug 17, 2021
8ddd755
feat: replace env to current env in the resource when checkout and ad…
luhanamz Aug 20, 2021
8966a7d
feat: e2e test and replacing env
luhanamz Aug 26, 2021
83e4155
feat: Minor changes for env replacement
luhanamz Aug 27, 2021
048b1e5
feat: remove changing env between env
luhanamz Aug 27, 2021
a2cd9fa
feat: Add cloudform type for type safety, move validation to provider…
luhanamz Aug 27, 2021
b7038fe
feat: remove some unused function and import, change regex for resource
luhanamz Aug 30, 2021
1f7efc7
feat: Some changes according to the PR comment
luhanamz Aug 30, 2021
c3f4d05
feat: changes according to PR comments
luhanamz Aug 31, 2021
41f242d
feat: remove unused import
luhanamz Aug 31, 2021
1bb1a42
feat: remove previous unused code
luhanamz Aug 31, 2021
28dab7d
feat: Changes according to PR comments
luhanamz Aug 31, 2021
bb85696
feat: some changes according to PR comments
luhanamz Sep 1, 2021
cac4100
feat: work on PR comments
luhanamz Sep 3, 2021
d1da5e8
feat: rebase for conflict
luhanamz Sep 3, 2021
5f9bb10
feat: rebase for failure of hooksmanager test failed
luhanamz Sep 4, 2021
0aa1933
feat: unit test
luhanamz Sep 7, 2021
8a8268c
feat: fix fail test
luhanamz Sep 7, 2021
453b8b2
feat: change default template of custom policies
luhanamz Sep 8, 2021
d9ed2e4
feat: fix failed test
luhanamz Sep 8, 2021
b7dda25
feat: PR comments
luhanamz Sep 9, 2021
c0c01d5
feat: pr comments
luhanamz Sep 9, 2021
64096b9
feat: fix failed test
luhanamz Sep 9, 2021
22c178e
feat: PR comments from ED
luhanamz Sep 9, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { DEPLOYMENT_MECHANISM } from './base-api-stack';
import { GitHubSourceActionInfo } from './pipeline-with-awaiter';
import { API_TYPE, IMAGE_SOURCE_TYPE, ResourceDependency, ServiceConfiguration } from './service-walkthroughs/containers-walkthrough';
import { ApiResource, generateContainersArtifacts } from './utils/containers-artifacts';
import { createDefaultCustomPoliciesFile, pathManager } from 'amplify-cli-core';

export const addResource = async (
serviceWalkthroughPromise: Promise<ServiceConfiguration>,
Expand Down Expand Up @@ -96,6 +97,10 @@ export const addResource = async (

}

createDefaultCustomPoliciesFile(category, resourceName);

const customPoliciesPath = pathManager.getCustomPoliciesPath(category, resourceName);

context.print.success(`Successfully added resource ${resourceName} locally.`);
context.print.info('');
context.print.success('Next steps:');
Expand All @@ -111,6 +116,7 @@ export const addResource = async (
context.print.info(
`- Amplify CLI infers many configuration settings from the "docker-compose.yaml" file. Learn more: docs.amplify.aws/cli/usage/containers`,
);
context.print.info(`- To access AWS resources outside of this Amplify app, edit the ${customPoliciesPath}`);
context.print.info('- Run "amplify push" to build and deploy your image');

return resourceName;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import {
saveMutableState,
updateLayerArtifacts,
} from './utils/storeResources';
import { createDefaultCustomPoliciesFile } from 'amplify-cli-core';

/**
* Entry point for creating a new function
Expand Down Expand Up @@ -112,18 +113,23 @@ export async function addFunctionResource(

await createFunctionResources(context, completeParams);

createDefaultCustomPoliciesFile(category, completeParams.resourceName);

if (!completeParams.skipEdit) {
await openEditor(context, category, completeParams.resourceName, completeParams.functionTemplate);
}

const { print } = context;

const customPoliciesPath = pathManager.getCustomPoliciesPath(category, completeParams.resourceName);

print.success(`Successfully added resource ${completeParams.resourceName} locally.`);
print.info('');
print.success('Next steps:');
print.info(`Check out sample function code generated in <project-dir>/amplify/backend/function/${completeParams.resourceName}/src`);
print.info('"amplify function build" builds all of your functions currently in the project');
print.info('"amplify mock function <functionName>" runs your function locally');
print.info(`To access AWS resources outside of this Amplify app, edit the ${customPoliciesPath}`);
print.info('"amplify push" builds all of your local backend resources and provisions them in the cloud');
print.info(
'"amplify publish" builds all of your local backend and front-end resources (if you added hosting category) and provisions them in the cloud',
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import {createDefaultCustomPoliciesFile} from '../customPoliciesUtils'
import { JSONUtilities } from '..';
import { pathManager, PathConstants } from '../state-manager';
import path from 'path';

describe('Custom policies util test', () => {

jest.mock('../state-manager');

const testCategoryName = 'function';
const testResourceName = 'functionTest';
const expectedFilePath = path.join(__dirname, 'testFiles', 'custom-policies-test', testCategoryName, testResourceName, PathConstants.CustomPoliciesFilename);
jest.spyOn(pathManager, 'getCustomPoliciesPath').mockReturnValue(expectedFilePath);

beforeEach(jest.clearAllMocks);

test('Write default custom policy file to the specified resource name', () => {

createDefaultCustomPoliciesFile(testCategoryName, testResourceName);

const data = JSONUtilities.readJson(expectedFilePath);

expect(data).toMatchObject([
{
Action: [],
Resource: []
}
]);

})
})
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ pathManager_mock.getHooksDirPath.mockReturnValue(testProjectHooksDirPath);
stateManager_mock.getHooksConfigJson.mockReturnValueOnce({ extensions: { py: { runtime: 'python3' } } });

jest.mock('execa');
jest.mock('process');
jest.mock('../../state-manager');
jest.mock('which', () => ({
sync: jest.fn().mockImplementation(runtimeName => {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"Action": [],
"Resource": []
}
]
64 changes: 64 additions & 0 deletions packages/amplify-cli-core/src/customPoliciesUtils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import { Fn, IAM } from 'cloudform-types';
import { JSONUtilities, pathManager } from '.';

export type CustomIAMPolicies = CustomIAMPolicy[];

export type CustomIAMPolicy = {
Action: string[];
Effect: string;
Resource: string[];
}

export const CustomIAMPoliciesSchema = {
type : 'array',
minItems: 1,
items: {
type: 'object',
properties: {
Action: { type: 'array', items: { type: 'string' }, minItems: 1, nullable: false },
Resource: { type: 'array', items: { type: 'string' }, minItems: 1, nullable: false }
},
optionalProperties: {
Effect: { type: 'string', enum:['Allow', 'Deny'], default: 'Allow' },
},
required: ['Resource', 'Action'],
additionalProperties: true
},
additionalProperties: false
}

export const customExecutionPolicyForFunction = new IAM.Policy({
PolicyName: 'custom-lambda-execution-policy',
ammarkarachi marked this conversation as resolved.
Show resolved Hide resolved
Roles: [
Fn.Ref('LambdaExecutionRole')
],
PolicyDocument: {
Version: '2012-10-17',
Statement: []
}
}).dependsOn(['LambdaExecutionRole']);

export const customExecutionPolicyForContainer = new IAM.Policy({
PolicyDocument: {
Statement: [
],
Version: '2012-10-17'
},
PolicyName: 'CustomExecutionPolicyForContainer',
ammarkarachi marked this conversation as resolved.
Show resolved Hide resolved
Roles: [
]
});

export function createDefaultCustomPoliciesFile(categoryName: string, resourceName: string) {
const customPoliciesPath = pathManager.getCustomPoliciesPath(categoryName, resourceName);
const defaultCustomPolicies = [
{
Action: [],
Resource: []
}
]
JSONUtilities.writeJson(customPoliciesPath, defaultCustomPolicies);
edwardfoyle marked this conversation as resolved.
Show resolved Hide resolved
}



2 changes: 2 additions & 0 deletions packages/amplify-cli-core/src/errors/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ export class AngularConfigNotFoundError extends Error {}
export class AppIdMismatchError extends Error {}
export class UnrecognizedFrameworkError extends Error {}
export class ConfigurationError extends Error {}
export class CustomPoliciesFormatError extends Error {}

export class NotInitializedError extends Error {
public constructor() {
super();
Expand Down
2 changes: 2 additions & 0 deletions packages/amplify-cli-core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ export * from './cliGetCategories';
export * from './cliRemoveResourcePrompt';
export * from './cliViewAPI';
export * from './hooks';
export * from './cliViewAPI';
export * from './customPoliciesUtils'

// Temporary types until we can finish full type definition across the whole CLI

Expand Down
5 changes: 5 additions & 0 deletions packages/amplify-cli-core/src/state-manager/pathManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ export const PathConstants = {
CLIJsonWithEnvironmentFileName: (env: string) => `cli.${env}.json`,

CfnFileName: (resourceName: string) => `${resourceName}-awscloudformation-template.json`,

CustomPoliciesFilename: 'custom-policies.json',
};

export class PathManager {
Expand Down Expand Up @@ -148,6 +150,9 @@ export class PathManager {

getDotAWSDirPath = (): string => path.normalize(path.join(homedir(), PathConstants.DotAWSDirName));

getCustomPoliciesPath = (category: string, resourceName: string): string =>
path.join(this.getResourceDirectoryPath(undefined, category, resourceName), PathConstants.CustomPoliciesFilename);

getAWSCredentialsFilePath = (): string => path.normalize(path.join(this.getDotAWSDirPath(), PathConstants.AWSCredentials));

getAWSConfigFilePath = (): string => path.normalize(path.join(this.getDotAWSDirPath(), PathConstants.AWSConfig));
Expand Down
19 changes: 16 additions & 3 deletions packages/amplify-cli-core/src/state-manager/stateManager.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
import * as fs from 'fs-extra';
import * as path from 'path';
import _ from 'lodash';
import { $TSAny, $TSMeta, $TSTeamProviderInfo, DeploymentSecrets, HooksConfig, PathConstants } from '..';
import { SecretFileMode } from '../cliConstants';
import { PathConstants, pathManager } from './pathManager';
import { $TSMeta, $TSTeamProviderInfo, $TSAny, DeploymentSecrets, HooksConfig } from '..';
import { JSONUtilities } from '../jsonUtilities';
import { SecretFileMode } from '../cliConstants';
import { HydrateTags, ReadTags, Tag } from '../tags';
import { pathManager } from './pathManager';
import { CustomIAMPolicies } from '../customPoliciesUtils';


export type GetOptions<T> = {
throwIfNotExist?: boolean;
Expand Down Expand Up @@ -76,6 +78,15 @@ export class StateManager {
return this.getData<$TSTeamProviderInfo>(filePath, mergedOptions);
};

getCustomPolicies = (categoryName: string, resourceName: string): CustomIAMPolicies | undefined => {
const filePath = pathManager.getCustomPoliciesPath(categoryName, resourceName);
try{
return JSONUtilities.readJson<CustomIAMPolicies>(filePath);
} catch(err) {
edwardfoyle marked this conversation as resolved.
Show resolved Hide resolved
return undefined;
}
};

localEnvInfoExists = (projectPath?: string): boolean => this.doesExist(pathManager.getLocalEnvFilePath, projectPath);

getLocalEnvInfo = (projectPath?: string, options?: GetOptions<$TSAny>): $TSAny => {
Expand Down Expand Up @@ -368,3 +379,5 @@ export class StateManager {
}

export const stateManager = new StateManager();


Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ import { onCategoryOutputsChange } from './on-category-outputs-change';
import { initializeEnv } from '../../initialize-env';
import { getProviderPlugins } from './get-provider-plugins';
import { getEnvInfo } from './get-env-info';
import { EnvironmentDoesNotExistError, exitOnNextTick, stateManager, $TSAny, $TSContext } from 'amplify-cli-core';
import { EnvironmentDoesNotExistError, exitOnNextTick, stateManager, $TSAny, $TSContext, CustomPoliciesFormatError } from 'amplify-cli-core';
import { printer } from 'amplify-prompts';

export async function pushResources(
context: $TSContext,
Expand Down Expand Up @@ -76,8 +77,9 @@ export async function pushResources(
await onCategoryOutputsChange(context, currentAmplifyMeta);
} catch (err) {
// Handle the errors and print them nicely for the user.
context.print.error(`\n${err.message}`);

if (!(err instanceof CustomPoliciesFormatError)) {
printer.error(`\n${err.message}`);
}
throw err;
}
} else {
Expand Down
26 changes: 26 additions & 0 deletions packages/amplify-e2e-core/src/categories/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -566,3 +566,29 @@ export function addRestContainerApi(projectDir: string) {
});
});
}

export function addRestContainerApiForCustomPolicies(projectDir: string, settings: { name: string }) {
return new Promise<void>((resolve, reject) => {
spawn(getCLIPath(), ['add', 'api'], { cwd: projectDir, stripColors: true })
.wait('Please select from one of the below mentioned services:')
.sendKeyDown()
.sendCarriageReturn()
.wait('Which service would you like to use')
.sendKeyDown()
.sendCarriageReturn()
.wait('Provide a friendly name for your resource to be used as a label for this category in the project:')
.send(settings.name)
.sendCarriageReturn()
.wait('What image would you like to use')
.sendKeyDown()
.sendCarriageReturn()
.wait('When do you want to build & deploy the Fargate task')
.sendCarriageReturn()
.wait('Do you want to restrict API access')
.sendConfirmNo()
.wait('Select which container is the entrypoint')
.sendCarriageReturn()
.wait('"amplify publish" will build all your local backend and frontend resources')
.run((err: Error) => err ? reject(err) : resolve());
});
}
4 changes: 4 additions & 0 deletions packages/amplify-e2e-core/src/utils/projectMeta.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ function getProjectMeta(projectRoot: string) {
const metaFilePath: string = path.join(projectRoot, 'amplify', '#current-cloud-backend', 'amplify-meta.json');
return JSON.parse(fs.readFileSync(metaFilePath, 'utf8'));
}
function getCustomPoliciesPath(projectRoot: string, category: string, resourceName: string): string {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be just used from pathmanager in core?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it failed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I will have to take a look at it

return path.join(projectRoot, 'amplify', 'backend', category, resourceName, 'custom-policies.json');
}

function getProjectTags(projectRoot: string) {
const projectTagsFilePath: string = path.join(projectRoot, 'amplify', '#current-cloud-backend', 'tags.json');
Expand Down Expand Up @@ -167,4 +170,5 @@ export {
getCloudBackendConfig,
setTeamProviderInfo,
getLocalEnvInfo,
getCustomPoliciesPath,
};
12 changes: 12 additions & 0 deletions packages/amplify-e2e-tests/functions/get-ssm-parameter.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
const aws = require('aws-sdk');

exports.handler = async event => {
const { secretName} = event;
const { Parameter } = await new aws.SSM()
.getParameter({
Name: secretName,
WithDecryption: true,
})
.promise();
return Parameter;
};
Loading