User Namespace Authorization
- Herd currently includes endpoint-level security. The Herd team is building towards a more granular security model where operations are authorized at the Namespace level. This feature is planned for released in version 0.13.0
- All access to data in Herd will be controlled by the calling user's authorization to data in that Namespace.
- A user will be granted permissions to READ, WRITE, EXECUTE, or GRANT data in a set of Namespaces.
- These authorizations restrict access to all operations including:
- Meta-data services like reading and writing Business Object Definitions, Business Object Formats, and Business Object Data in a given Namespace
- Other services such as Clusters, Jobs, and Notifications owned by a given Namespace
- See Authorization Scope for more details
- If not authorized, the services will return an appropriate HTTP error response such as 403 Forbidden
- Herd team will delegate administration of granular security to Data Owners for each Namespace. See Administration Model
- Note - this mechanism is currently implemented at the Namespace level but will likely be augmented in the future to work at a lower level such as Namespace + Business Object Definition
An example user's authorization data is below. In this example, J. Smith is a member of the Analytics team that runs Jobs which read data from various Namespaces and creates output in the Analytics Namespace.
- SmithJ (reference to account in AD)
- READ permissions to Namespaces: MARKET, OPTIONS, ETL
- WRITE permissions to Namespaces: ANALYTICS
- EXECUTE permissions to: ANALYTICS
Herd will designate a Data Owner for each Namespace. This Data Owner is responsible for granting READ/WRITE/EXECUTE access to their Namespace to any applicable Users. Here are examples of what the Data Owner will perform:
- Grant a user access to their Namespace by calling User Namespace Authorization Post for that user and indicating which permissions the user should have (READ/WRITE/EXECUTE/GRANT or any combination)
- Modify a user's access to their Namespace by calling User Namespace Authorization Put for an existing user and passing the updated set of permissions (READ/WRITE/EXECUTE/GRANT or any combination).
- This will replace that user's entire set of permissions to the Namespace with the set of permissions included in the PUT request
- Delete a user's access to their Namespace by calling User Namespace Authorization Delete for an existing user.
- This will remove all that user's permissions to the Namespace
- Review a user's access to their Namespace by calling User Namespace Authorization Get for a user Note: Data Owners must have the DATA_OWNER role and they must have GRANT and READ permissions on their Namespace
End users can call Current User Get to view all their permissions in each Namespace where they have any permissions.
The following steps will take place to create a Namespace and delegate responsibility to the Data Owner
- Data Owner creates new Namespace NEW_APP
- Herd Administrator assigns DATA_OWNER role to the Data Owner
- Herd Administrator grants GRANT permissions on the NEW_APP Namespace to the Data Owner
- Data Owner may now perform all responsibilities as listed in Data Owner Responsibilities
Authorization covers the following services as detailed below:
| Service | Permissions |
|---|---|
| Business Object Format CRUD services | READ and/or WRITE on Namespace of Format in request |
| Business Object Definition CRUD services | READ and/or WRITE on Namespace of BDef in request |
| Business Object Data CRUD services including Status update | READ and/or WRITE on Namespace of BData in request |
| Availability and DDL services | READ on Namespace of BData in request |
| Availability and DDL collection services | READ on Namespace, will fail if user is not authorized on all of the request object collection's Namespaces |
| Service | Permissions |
|---|---|
| Upload Single Initiation service | WRITE on Namespace of both BData in request, will fail if user is not authorized for Namespace of either BData in request |
| Download Single Initiation service | READ on Namespace of BData in request |
| Upload and Download Credential services | WRITE and/or READ on Namespace of BData in request (including Herd Uploader/Downloader tools) |
| Credential extension service | READ on Namespace of BData in request |
| Storage Policy services | READ and/or WRITE on Namespace of BData in policy Policies without namespace require OPS role in addition to permissions above |
| Account-Permissions mapping services | * For GET service - READ on Namespace for which the request is granting permissions * For POST, PUT, DELETE services - GRANT on Namespace is required. * Note: WRITE is not required so Data Owners do not necessarily have permission to write to the Namespace for which they are granting permissions. * Note: DATA_OWNER role is also required to access any of these services |
| Service | Permissions |
|---|---|
| Job Definition CRUD services | READ and/or WRITE on Namespace of Job Definition (permissions on BData modified in Job are not required to CRUD Job Definition) |
| Job Execute POST service | EXECUTE permission on Namespace of Job Definition is required to start a Job Note - during job execution, job runs in context of the Account that registered the job, not the Account that executes the job (see Execution context for asynchronous Jobs) |
| Job Execute status service | READ on Namespace of Job Definition |
| Job Status services | READ on Namespace of Job Definition provided in request |
| Notification Registration CRUD services | For GET - READ on Namespace of Notification Registration For POST, PUT, Status PUT - EXECUTE on Namespace of Job Definitions that are triggered For POST, PUT, Status PUT - READ on Namespace of all BData in Notification Filter |
| EMR Cluster Definition CRUD | READ and/or WRITE on Namespace of Cluster Definition |
| Add Cluster Steps | EXECUTE on Namespace of Cluster Definition |
| Get Cluster (starts cluster) | EXECUTE on Namespace of Cluster Definition |
The following services return data from multiple Namespaces and will filter results based on permissions as follows. If the filter removes all results, the service will return an empty list as opposed to an error response.
| Service | Permissions |
|---|---|
| Job Status service | If Namespace is not included in the request, filter to contain entries caller has READ on Namespace of Job |
| Get Business Object Definitions service | If Namespace is not included in the request, filter to contain entries caller has READ on Namespace of Business Object Definition |
All asynchronous Jobs execute in the context of an Account. This applies to both notification-based (Registration Notification) and service-triggered workflow (Job Execution POST). Execution in context of an Account means that all service operations in a Job enforce READ/WRITE Permissions based on Namespaces the Account is authorized.
| Service | Permissions |
|---|---|
| Asynchronous Jobs (service-triggered) | All services in workflow execute in the context of the User that created the Job Definition |
| Asynchronous Jobs (notification-triggered) | All services in workflow execute in the context of the User that created the Job Definition |
- JarJar, a workflow developer on the ETL team:
- READ: MARKET, ETL
- WRITE: ETL, CROSS_MARKET
- EXECUTE: (None)
- Artoo, a manager on the ETL team:
- READ: MARKET
- WRITE: (None)
- EXECUTE: ETL
- Rey, an Operations Manager who has the following permissions:
- READ: MARKET, ETL, CROSS_MARKET
- WRITE: (None)
- EXECUTE: ETL
The example scenario is as follows:
- ETL comment - investigate possibility of RunAs in workflow definition - but who you can assign is somehow restricted by GRANT so can't just always assign super user
- JarJar creates the MarketToCrossMarket workflow that reads from MARKET and ETL Namespaces and write output to the CROSS_MARKET and ETL Namespaces
- JarJar registers a Job Definition in Herd with the MarketToCrossMarket workflow
- JarJar must have WRITE permissions to the ETL Namespace to create a Job Definition in the ETL Namespace
- Artoo registers notifications that respond to status changes on source objects in the MARKET Namespace and trigger the MarketToCrossMarket workflow
- Artoo must have READ permissions on the MARKET and ETL Namespaces as source objects in those Namespace appear in the Notification Filter
- Artoo must have EXECUTE permissions on the ETL Namespace as the notification will start the MarketToCrossMarket workflow which is in the ETL Namespace
- A source object status change occurs and triggers the notification which starts the MarketToCrossMarket workflow The MarketToCrossMarket workflow executes in the context of the JarJar User as that is who registered the workflow definition
- JarJar has permissions to READ and WRITE all the BData in the workflow so it executes without permissions violations
- The next day there is a transient system error that affects the MarketToCrossMarket workflow
- Rey observes the error and determines the workflow should be re-run
- Rey triggers the workflow by calling the Herd Job Execute service
- Rey must have EXECUTE permissions on the ETL Namespace to trigger the workflow
- The MarketToCrossMarket workflow executes in the context of the JarJar User as that is who registered the workflow definition
- JarJar has permissions to READ and WRITE all the BData in the workflow so it executes without permissions violations
- Namespace CRUD
- Data Provider CRUD
- PartitionKeyGroups, ExpectedPartitionValues CRUD
- Storage, StoragePlatforms CRUD
- File Types Read
- Getting Started with herd
- herd Usage Pages
- herd API documentation
- herd Workflow Tasks
- herd Tools