Skip to content
/ ansible-role-nifi Public
forked from cavemandaveman/ansible-role-nifi

An Ansible Role that installs NiFi on Linux. By default, it installs NiFi in a way that makes upgrading painless.

galaxy.ansible.com/cavemandaveman/nifi

License

GPL-3.0 license
0 stars 16 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

agile-lab-dev/ansible-role-nifi

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

81 Commits
defaults
defaults
 
 
handlers
handlers
 
 
meta
meta
 
 
tasks
tasks
 
 
templates
templates
 
 
.gitignore
.gitignore
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 

Repository files navigation

Ansible Role: NiFi

An Ansible Role that installs NiFi on Linux. By default, it installs NiFi in a way that makes upgrading painless.

Requirements

Requires at least Java 8.

Role Variables

See defaults/main.yml for all variables and how to specify them. For a deeper dive, the NiFi System Administrator’s Guide is a great resource.

The following specifies where to download (or look for existing) binaries (tarballs), where to install NiFi, and a home directory which will be symbolically linked to the specified release. Also, a centralized config directory to store files that need not be changed (to avoid copying during upgrades). You can add more artbitrary key/value pairs to this dict and those directories will be created. This might be useful if you need extra directories for things like custom nars, drivers, etc.

nifi_config_dirs:
  binaries: /tmp
  install: /opt/nifi/releases
  home: /opt/nifi/releases/current
  external_config: /opt/nifi/config_resources

By default, this is the directory structure that will be created:

|--opt/
  |--nifi/
    |--releases/
      |--current -> nifi-1.14.0/
      |--nifi-1.14.0/
      |--nifi-1.13.2/
    |--config_resources/
      |--archive/
      |--authorizations.xml
      |--content_repository/
      |--custom_nars/
      |--database_repository/
      |--flow.xml.gz
      |--flowfile_repository/
      |--provenance_repository/
      |--state/
      |--users.xml

Any key/value pair from a config file can be added to the following dicts. Dict names correspond to file names. The current config options for these files can be found here.

nifi_properties:
bootstrap:
nifi_env:
logback:
login_identity_providers:
state_management:
authorizers:
zookeeper:

Dependencies

None.

Example Playbooks

These assume you have hash_behaviour=merge set in your config. If not, please also include the default dict key/values from defaults/main.yml.

Basic single node NiFi instance:

- hosts: nifi_servers
  become: yes
  roles:
    - cavemandaveman.nifi

Basic 3 node NiFi cluster using embedded Zookeeper:

- hosts: nifi_servers
  become: yes
  roles:
    - cavemandaveman.nifi
  vars:
    nifi_properties:
      # HTTP properties must be unset for HTTPS to work
      nifi.web.http.host: ""
      nifi.web.http.port: ""
      nifi.web.https.host: "{{ ansible_fqdn }}"
      nifi.web.https.port: 8443
      nifi.cluster.is.node: true
      nifi.cluster.node.address: "{{ ansible_fqdn }}"
      nifi.cluster.node.protocol.port: 11443
      nifi.cluster.flow.election.max.candidates: 3
      nifi.cluster.load.balance.host: "{{ ansible_fqdn }}"
      nifi.cluster.load.balance.port: 6342
      nifi.state.management.embedded.zookeeper.start: true
      nifi.zookeeper.connect.string: nifi_server1:2181,nifi_server2:2181,nifi_server3:2181
    login_identity_providers:
      /loginIdentityProviders/provider/identifier: single-user-provider
      /loginIdentityProviders/provider/class: org.apache.nifi.authentication.single.user.SingleUserLoginIdentityProvider

    authorizers:
      authorizer:
        /authorizers/authorizer/identifier: single-user-authorizer
        /authorizers/authorizer/class: org.apache.nifi.authorization.single.user.SingleUserAuthorizer
    state_management:
      /stateManagement/cluster-provider/property[@name="Connect String"]: "{{ nifi_properties['nifi.zookeeper.connect.string'] }}"
    # Assuming nifi_server1 = 192.168.1.10, nifi_server2 = 192.168.1.11, nifi_server3 = 192.168.1.12
    # we have Ansible automatically set the myid file on each host to last octet of the node's IP address
    # and we set the 'X' of the zookeeper['server.X'] keys to those same numbers.
    zookeeper_myid: "{{ ansible_default_ipv4.address.split('.')[-1] }}"
    zookeeper:
      server.10: nifi_server1:2888:3888
      server.11: nifi_server2:2888:3888
      server.12: nifi_server3:2888:3888

Secure single node NiFi instance with LDAP:

- hosts: nifi_servers
  become: yes
  roles:
    - cavemandaveman.nifi
  vars:
    nifi_properties:
      # HTTP properties must be unset for HTTPS to work
      nifi.web.http.host: ""
      nifi.web.http.port: ""
      nifi.web.https.host: "{{ ansible_fqdn }}"
      nifi.web.https.port: 8443
      nifi.security.keystore: /path/to/keystore.jks
      nifi.security.keystoreType: JKS
      nifi.security.keystorePasswd: keystorePassword
      nifi.security.keyPasswd: keyPassword
      nifi.security.truststore: /path/to/truststore.jks
      nifi.security.truststoreType: JKS
      nifi.security.truststorePasswd: truststorePassword
    login_identity_providers:
      /loginIdentityProviders/provider/identifier: ldap-provider
      /loginIdentityProviders/provider/class: org.apache.nifi.ldap.LdapProvider
      /loginIdentityProviders/provider/property[@name="Authentication Strategy"]: SIMPLE
      /loginIdentityProviders/provider/property[@name="Manager DN"]: cn=nifi,ou=people,dc=example,dc=com
      /loginIdentityProviders/provider/property[@name="Manager Password"]: password
      /loginIdentityProviders/provider/property[@name="Referral Strategy"]: FOLLOW
      /loginIdentityProviders/provider/property[@name="Connect Timeout"]: 10 secs
      /loginIdentityProviders/provider/property[@name="Read Timeout"]: 10 secs
      /loginIdentityProviders/provider/property[@name="Url"]: ldap://hostname:port
      /loginIdentityProviders/provider/property[@name="User Search Base"]: OU=people,DC=example,DC=com
      /loginIdentityProviders/provider/property[@name="User Search Filter"]: sAMAccountName={0}
      /loginIdentityProviders/provider/property[@name="Identity Strategy"]: USE_DN
      /loginIdentityProviders/provider/property[@name="Authentication Expiration"]: 12 hours

    authorizers:
      userGroupProviders:
        /authorizers/userGroupProvider/identifier: file-user-group-provider
        /authorizers/userGroupProvider/class: org.apache.nifi.authorization.FileUserGroupProvider
        /authorizers/userGroupProvider/property[@name="Users File"]: "{{ nifi_config_dirs.external_config }}/users.xml"
        /authorizers/userGroupProvider/property[@name="Initial User Identity 1"]: cn=John Smith,ou=people,dc=example,dc=com

      accessPolicyProviders:
        /authorizers/accessPolicyProvider/identifier: file-access-policy-provider
        /authorizers/accessPolicyProvider/class: org.apache.nifi.authorization.FileAccessPolicyProvider
        /authorizers/accessPolicyProvider/property[@name="User Group Provider"]: file-user-group-provider
        /authorizers/accessPolicyProvider/property[@name="Authorizations File"]: "{{ nifi_config_dirs.  external_config }}/authorizations.xml"
        /authorizers/accessPolicyProvider/property[@name="Initial Admin Identity"]: cn=John Smith,ou=people,dc=example,dc=com
      
      authorizer:
        /authorizers/authorizer/identifier: managed-authorizer
        /authorizers/authorizer/class: org.apache.nifi.authorization.StandardManagedAuthorizer
        /authorizers/authorizer/property[@name="Access Policy Provider"]: file-access-policy-provider

Secure 3 node NiFi cluster with LDAP using embedded zookeeper:

- hosts: nifi_servers
  become: yes
  roles:
    - cavemandaveman.nifi
  vars:
    nifi_properties:
      # HTTP properties must be unset for HTTPS to work
      nifi.web.http.host: ""
      nifi.web.http.port: ""
      nifi.web.https.host: "{{ ansible_fqdn }}"
      nifi.web.https.port: 8443
      nifi.security.keystore: /path/to/keystore.jks
      nifi.security.keystoreType: JKS
      nifi.security.keystorePasswd: keystorePassword
      nifi.security.keyPasswd: keyPassword
      nifi.security.truststore: /path/to/truststore.jks
      nifi.security.truststoreType: JKS
      nifi.security.truststorePasswd: truststorePassword
      nifi.cluster.protocol.is.secure: true
      nifi.cluster.is.node: true
      nifi.cluster.node.address: "{{ ansible_fqdn }}"
      nifi.cluster.node.protocol.port: 11443
      nifi.cluster.flow.election.max.candidates: 3
      nifi.cluster.load.balance.host: "{{ ansible_fqdn }}"
      nifi.cluster.load.balance.port: 6342
      nifi.state.management.embedded.zookeeper.start: true
      nifi.zookeeper.connect.string: nifi_server1:2181,nifi_server2:2181,nifi_server3:2181
    login_identity_providers:
      /loginIdentityProviders/provider/identifier: ldap-provider
      /loginIdentityProviders/provider/class: org.apache.nifi.ldap.LdapProvider
      /loginIdentityProviders/provider/property[@name="Authentication Strategy"]: SIMPLE
      /loginIdentityProviders/provider/property[@name="Manager DN"]: cn=nifi,ou=people,dc=example,dc=com
      /loginIdentityProviders/provider/property[@name="Manager Password"]: password
      /loginIdentityProviders/provider/property[@name="Referral Strategy"]: FOLLOW
      /loginIdentityProviders/provider/property[@name="Connect Timeout"]: 10 secs
      /loginIdentityProviders/provider/property[@name="Read Timeout"]: 10 secs
      /loginIdentityProviders/provider/property[@name="Url"]: ldap://hostname:port
      /loginIdentityProviders/provider/property[@name="User Search Base"]: OU=people,DC=example,DC=com
      /loginIdentityProviders/provider/property[@name="User Search Filter"]: sAMAccountName={0}
      /loginIdentityProviders/provider/property[@name="Identity Strategy"]: USE_DN
      /loginIdentityProviders/provider/property[@name="Authentication Expiration"]: 12 hours

    authorizers:
      userGroupProviders:
        /authorizers/userGroupProvider/identifier: file-user-group-provider
        /authorizers/userGroupProvider/class: org.apache.nifi.authorization.FileUserGroupProvider
        /authorizers/userGroupProvider/property[@name="Users File"]: "{{ nifi_config_dirs.external_config }}/users.xml"
        /authorizers/userGroupProvider/property[@name="Initial User Identity 1"]: cn=John Smith,ou=people,dc=example,dc=com
        # Use the full DN of the node certificates here
        /authorizers/userGroupProvider/property[@name="Initial User Identity 2"]: CN=nifi_server1.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
        /authorizers/userGroupProvider/property[@name="Initial User Identity 3"]: CN=nifi_server2.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
        /authorizers/userGroupProvider/property[@name="Initial User Identity 4"]: CN=nifi_server3.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
      
      accessPolicyProviders:
        /authorizers/accessPolicyProvider/identifier: file-access-policy-provider
        /authorizers/accessPolicyProvider/class: org.apache.nifi.authorization.FileAccessPolicyProvider
        /authorizers/accessPolicyProvider/property[@name="User Group Provider"]: file-user-group-provider
        /authorizers/accessPolicyProvider/property[@name="Authorizations File"]: "{{ nifi_config_dirs.external_config }}/authorizations.xml"
        /authorizers/accessPolicyProvider/property[@name="Initial Admin Identity"]: cn=John Smith,ou=people, dc=example,dc=com
        /authorizers/accessPolicyProvider/property[@name="Node Identity 1"]: CN=nifi_server1.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
        /authorizers/accessPolicyProvider/property[@name="Node Identity 2"]: CN=nifi_server2.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US
        /authorizers/accessPolicyProvider/property[@name="Node Identity 3"]: CN=nifi_server3.example.com, O=ExampleLLC, L=Saint Louis, ST=Missouri, C=US

      authorizer:
        /authorizers/authorizer/identifier: managed-authorizer
        /authorizers/authorizer/class: org.apache.nifi.authorization.StandardManagedAuthorizer
        /authorizers/authorizer/property[@name="Access Policy Provider"]: file-access-policy-provider

    state_management:
      /stateManagement/cluster-provider/property[@name="Connect String"]: "{{ nifi_properties['nifi.zookeeper.connect.string'] }}"
    # Assuming nifi_server1 = 192.168.1.10, nifi_server2 = 192.168.1.11, nifi_server3 = 192.168.1.12
    # we have Ansible automatically set the myid file on each host to last octet of the node's IP address
    # and we set the 'X' of the zookeeper['server.X'] keys to those same numbers.
    zookeeper_myid: "{{ ansible_default_ipv4.address.split('.')[-1] }}"
    zookeeper:
      server.10: nifi_server1:2888:3888
      server.11: nifi_server2:2888:3888
      server.12: nifi_server3:2888:3888

License

GPLv3

Author Information

This role was created in 2018 by cavemandaveman.

About

An Ansible Role that installs NiFi on Linux. By default, it installs NiFi in a way that makes upgrading painless.

galaxy.ansible.com/cavemandaveman/nifi

Resources

Readme

License

GPL-3.0 license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

10 tags

Packages

No packages published

Languages

  • Jinja 100.0%