Releases: mybasementcloud/R8x-export-import-api-scripts
Release v00.60.12.100.750.1 2023-03-14
Release package of current v00.60.XX development operations
UPDATED: 2023-03-14
Final Interim Release, pending further testing and expansion
Future development on this script set is moving to GitHub Project R8X_mgmt_cli_API_bash_scripts
v00.60.12
v00.60.12 New Objects Supported
v00.60.12 New JSON exports
- Added support for Data Center objects: Data Center Servers, Data Center Objects, and Data Center Queries for json export
- Added support for custom application-site objects via generic object capture json export
- Added support for custom application-site objects via generic object capture and array evaluation to generate custom application-sites json export
v00.60.12 New JSON and CSV exports
- Added support for Global Properties special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for Policy Settings special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for API Settings special object/properties for json and csv export, when exporting domain "System Data" (also on SMS using domain "System Data")
- Addes support for Radius Server and Radius Group objects for API version 1.9 and later [R81.20 GA], for all operations
- Addes support for Repository Script objects for API version 1.9 and later [R81.20 GA], for all special object operations
- Addes support for SmartTasks objects for API version 1.9 and later [R81.20 GA], for all operations
v00.60.12 New CSV exports
- Added support for application-site objects url-list and additional-categories elements-CSV files (like group members), done in special objects export script or when enabling export of Critical Performance Impacting (CPI) objects
- Added support for custom application-site objects via generic object capture and array evaluation to generate custom application-sites csv export and import, two variants
- first export variant for CSV import with elements-CSV files (see next New Objects Supported) -- NOT A CPI object!
- second export variant for CSV import alternative, with up to 20 url-list entries and 10 additional-categories in the exported file
- Added support for custom application-site objects via generic object capture and array evaluation for actual additional-categories and url-list elements-CSV files (like group members), for csv export and import
- Added support for hosts that have NAT configured and that do not have NAT configured to explicit files for easier handling, for csv operations. The original hosts and hosts_NO_NAT operations are still available. Import should either utilize: hosts, hosts_NO_NAT, or hosts_with_NAT and hosts_without_NAT files. JSON export is possible for the object; however, it is disabled to avoid need for more complicated CSV export, as all the data is in the normal hosts file.
v00.60.12 Operational Changes
- Added Command Line Parameters to handle specific domains: "System Data" and "Global", --domain-System-Data|--dSD|--dsd and --domain-Global|--dG|--dg respectively, to handle issues with operational scripts and passing quoted parameters with spaces, as well as easier domain specific execution.
- Added support for export of special objects and properties to json
- Added support for basic plumbing for delete, export, import, set/update, rename, and augment CSV files for special objects and properties via CSV
- Added support for per object | special object/properties specific control of utilization of "details-level", "ignore-errors", "ignore-warnings"
- Modified CSV key value sets exported by default for application-site objects
- Added information more detailed error handling mgmt_cli and JQ calls, to help with identification of problems and performance related limitations
- Added object_operations script files for MDSM with max object limit configuration for 100 objects
- Added CLI parameters to determine handling of Critical Performance Impacting (CPI) objects, [--DO-CPI | --Override-Critical-Performance-Impact] or [--NO-CPI | --NO-Critical-Performance-Impact], like application-site objects with > 10,000 Check Point provided objects to handle. Default mode is to exclude CPI objects from export operations [--NO-CPI | --NO-Critical-Performance-Impact]
- Added script variants to handle special objects for object export, all domain objects export, and all domains objects CSV export.
- Added handler for json extraction of specific objects from a larger set based on export of a reference key value from a generic object query
- Harmonization of the json file slurp operation across different export functions and implementation of some common procedures based on redundant implementations
- Corrections of object association with different main script operations, like export, import, delete, etc.; to ensure that the correct things will work or get skipped
- Added files for operational export of minimum necessary exports for import under the root of the script:
_minimum_export.sh_minimum_exports_with_some_do_cpi.sh_minimum_system_data_exports_with_some_do_cpi.sh
- Modified CheckAPIKeepAlive to limit impact of mgmt_cli keep alive calls by checking the last time the procedure was called and only executing an actual mgmt_cli keep alive action if the current default interval of 60 seconds has passed since the last execution; otherwise, make a quick note in logs and continue. A future command line parameter may be added to control the actual interval between required mgmt_cli keep alive executions.
- Reorganized layout of Object Definition Data to make it easier to see variants when reviewing script code
- Cosmetic changes to enhance the flow of operation display, especially when utilization -v (Verbose) logging mode
- Homogenized and harmonized how the routines handling CSV export of complex objects based on generic object queries and associated arrays to the complex object operate, to reduce the number of places to adjust certain processing methods.
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.12.100.750.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.12.100.750.tgz /var/log/__customer/ )
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
Additional Documentation
- README
- What's New
- HOW TO Guidance
- Limitations and Caveats
- Supported Objects TSV
- Explanation of the Supported Objects TSV
- File and Folder Purpose and Utilization
- Check Point Software Technologies Management API Reference
- Check Point Software Technologies APIs
Related Projects on GitHub
- Consolidated mgmt_cli bash Script solutions (Under Development)
- Limited Export of mgmt_cli based Policy and Layers export and import scripts
- Windows Scripts for Check Point Operations
CAVEATS
This release package now works with R81.20.
- Tested successfully on R81.20 GA T627 JHF 8
- MDSM testing pending due to infrastructure technical issues with MDSM hosts
(**) Issues with performance throttling and maximum object limits for show operations for large data sets, e.g. application-site objects
With exception of the --MaaS (Smart-1 Cloud) authenticati...
Release v00.60.12.100.500 2023-03-08 INTERIM
Release package of current v00.60.XX development operations
UPDATED: 2023-03-08
Interim Release, pending further testing and expansion
v00.60.12
v00.60.12 New Objects Supported
v00.60.12 New JSON exports
- Added support for Data Center objects: Data Center Servers, Data Center Objects, and Data Center Queries for json export
- Added support for custom application-site objects via generic object capture json export
- Added support for custom application-site objects via generic object capture and array evaluation to generate custom application-sites json export
v00.60.12 New JSON and CSV exports
- Added support for Global Properties special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for Policy Settings special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for API Settings special object/properties for json and csv export, when exporting domain "System Data" (also on SMS using domain "System Data")
- Addes support for Radius Server and Radius Group objects for API version 1.9 and later [R81.20 GA], for all operations
- Addes support for Repository Script objects for API version 1.9 and later [R81.20 GA], for all special object operations
- Addes support for SmartTasks objects for API version 1.9 and later [R81.20 GA], for all operations
v00.60.12 New CSV exports
- Added support for application-site objects url-list and additional-categories elements-CSV files (like group members), done in special objects export script or when enabling export of Critical Performance Impacting (CPI) objects
- Added support for custom application-site objects via generic object capture and array evaluation to generate custom application-sites csv export and import, two variants
- first export variant for CSV import with elements-CSV files (see next New Objects Supported) -- NOT A CPI object!
- second export variant for CSV import alternative, with up to 20 url-list entries and 10 additional-categories in the exported file
- Added support for custom application-site objects via generic object capture and array evaluation for actual additional-categories and url-list elements-CSV files (like group members), for csv export and import
- Added support for hosts that have NAT configured and that do not have NAT configured to explicit files for easier handling, for csv operations. The original hosts and hosts_NO_NAT operations are still available. Import should either utilize: hosts, hosts_NO_NAT, or hosts_with_NAT and hosts_without_NAT files. JSON export is possible for the object; however, it is disabled to avoid need for more complicated CSV export, as all the data is in the normal hosts file.
v00.60.12 Operational Changes
- Added Command Line Parameters to handle specific domains: "System Data" and "Global", --domain-System-Data|--dSD|--dsd and --domain-Global|--dG|--dg respectively, to handle issues with operational scripts and passing quoted parameters with spaces, as well as easier domain specific execution.
- Added support for export of special objects and properties to json
- Added support for basic plumbing for delete, export, import, set/update, rename, and augment CSV files for special objects and properties via CSV
- Added support for per object | special object/properties specific control of utilization of "details-level", "ignore-errors", "ignore-warnings"
- Modified CSV key value sets exported by default for application-site objects
- Added information more detailed error handling mgmt_cli and JQ calls, to help with identification of problems and performance related limitations
- Added object_operations script files for MDSM with max object limit configuration for 100 objects
- Added CLI parameters to determine handling of Critical Performance Impacting (CPI) objects, [--DO-CPI | --Override-Critical-Performance-Impact] or [--NO-CPI | --NO-Critical-Performance-Impact], like application-site objects with > 10,000 Check Point provided objects to handle. Default mode is to exclude CPI objects from export operations [--NO-CPI | --NO-Critical-Performance-Impact]
- Added script variants to handle special objects for object export, all domain objects export, and all domains objects CSV export.
- Added handler for json extraction of specific objects from a larger set based on export of a reference key value from a generic object query
- Harmonization of the json file slurp operation across different export functions and implementation of some common procedures based on redundant implementations
- Corrections of object association with different main script operations, like export, import, delete, etc.; to ensure that the correct things will work or get skipped
- Added files for operational export of minimum necessary exports for import under the root of the script:
_minimum_export.sh,_minimum_exports_with_some_do_cpi.sh, and_minimum_system_data_exports_with_some_do_cpi.sh - Modified CheckAPIKeepAlive to limit impact of mgmt_cli keep alive calls by checking the last time the procedure was called and only executing an actual mgmt_cli keep alive action if the current default interval of 60 seconds has passed since the last execution; otherwise, make a quick note in logs and continue. A future command line parameter may be added to control the actual interval between required mgmt_cli keep alive executions.
- Reorganized layout of Object Definition Data to make it easier to see variants when reviewing script code
- Cosmetic changes to enhance the flow of operation display, especially when utilization -v (Verbose) logging mode
- Homogenized and harmonized how the routines handling CSV export of complex objects based on generic object queries and associated arrays to the complex object operate, to reduce the number of places to adjust certain processing methods.
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.12.100.500.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.12.100.500.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.1.
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package now works with R81.20.
- Tested successfully on R81.20 GA T627 JHF N/A
- MDSM testing pending due to infrastructure technical issues with MDSM hosts
(**) Issues with performance throttling and maximum object limits for show operations for large data sets, e.g. application-site objects
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customerchmod 775 /var/log/__customercd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.12.100.500.tgz -
Goto to the export import folder
cd ./devops.dev/objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
QUICK START SCRIPT FI...
Release v00.60.12.100.450 2023-02-26 INTERIM
Release package of current v00.60.XX development operations
UPDATED: 2023-02-26
Interim Release, pending further testing and expansion
v00.60.12
v00.60.12 New Objects Supported
- Added support for Global Properties special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for Policy Settings special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for API Settings special object/properties for json and csv export, when exporting domain "System Data" (also on SMS using domain "System Data")
- Added support for Radius Server and Radius Group objects for API version 1.9 and later [R81.20 GA], for all operations
- Added support for Repository Script objects for API version 1.9 and later [R81.20 GA], for all special object operations
- Added support for SmartTasks objects for API version 1.9 and later [R81.20 GA], for all operations
- Added support for application-site objects url-list and additional-categories sub-CSV files (like group members), done in special objects export script or when enabling export of Critical Performance Impacting (CPI) objects
- Added support for Data Center objects: Data Center Servers, Data Center Objects, and Data Center Queries for json export
- Added support for custom application-site objects via generic object capture and array evaluation for actual application-sites, for json and csv export
- Added support for hosts that have NAT configured and that do not have NAT configured to explicit files for easier handling, for both json and csv operations. The original hosts and hosts_NO_NAT operations are still available. Import should either utilize: hosts, hosts_NO_NAT, or hosts_with_NAT and hosts_without_NAT files.
v00.60.12 Operational Changes
- Added Command Line Parameters to handle specific domains: "System Data" and "Global", --domain-System-Data|--dSD|--dsd and --domain-Global|--dG|--dg respectively, to handle issues with operational scripts and passing quoted parameters with spaces, as well as easier domain specific execution.
- Added support for export of special objects and properties to json
- Added support for basic plumbing for delete, export, import, set/update, rename, and augment CSV files for special objects and properties via CSV
- Added support for per object | special object/properties specific control of utilization of "details-level", "ignore-errors", "ignore-warnings"
- Modified CSV key value sets exported by default for application-site objects
- Added information more detailed error handling mgmt_cli and JQ calls, to help with identification of problems and performance related limitations
- Added object_operations script files for MDSM with max object limit configuration for 100 objects
- Added CLI parameters to determine handling of Critical Performance Impacting (CPI) objects, [--DO-CPI | --Override-Critical-Performance-Impact] or [--NO-CPI | --NO-Critical-Performance-Impact], like application-site objects with > 10,000 Check Point provided objects to handle. Default mode is to exclude CPI objects from export operations [--NO-CPI | --NO-Critical-Performance-Impact]
- Added script variants to handle special objects for object export, all domain objects export, and all domains objects CSV export.
- Added handler for json extraction of specific objects from a larger set based on export of a reference key value from a generic object query
- Harmonization of the json file slurp operation across different export functions and implementation of some common procedures based on redundant implementations
- Corrections of object association with different main script operations, like export, import, delete, etc.; to ensure that the correct things will work or get skipped
- Added files for operational export of minimum necessary exports for import under the root of the script: _minimum_export.sh, _minimum_exports_with_some_do_cpi.sh, and _minimum_system_data_exports_with_some_do_cpi.sh
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.12.100.450.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.12.100.450.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.1.
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package now works with R81.20, but additional testing and work needed, as well as later takes for testing.
- Tested successfully on R81.20 GA T627 JHF N/A
- MDSM testing pending due to infrastructure technical issues with MDSM hosts
(**) Issues with performance throttling and maximum object limits for show operations for large data sets, e.g. application-site objects
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customerchmod 775 /var/log/__customercd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.12.100.tgz -
Goto to the export import folder
cd ./devops.dev/objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
QUICK START SCRIPT FILES
The following scripts are in the root of the folder ./devops.dev and will execute a minimu export for import on the local managemnet server (SMS). Other scripts for more detailed operations and testing are located under the ./devops.dev/objects.wip/object_operations folder.
Script Files:
_minimum_exports.sh: Execute the author's bare minimum export operations to enable import for SMS, Critical Performance Impact (CPI) objects, like all application-sites are not exported._minimum_exports_with_some_do_cpi.sh: Execute the author's bare minimum export operations to enable import for SMS, with some commands utilizing the CLI parameter to execute with Critical Performance Impact (CPI) objects._minimum_system_data_exports_with_some_do_cpi.sh: Execute the author's standard export operations using domain "System Data" for SMS, with some commands utilizing the CLI parameter to execute with Critical Performance Impact (CPI) objects.
QUICK START FOR UPDATING
To quickly start working with the scripts if there is an older version installed, do the following.
-
Download the release tgz file or the devops.dev.only.{version}.tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Delete or rename the existing devops.dev folder
Example:
rm /var/log/__customer/devops.dev
or Example:mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.oldrm /var/log/__customer/devops.dev
or
mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.old -
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.only.{version}.tgz
Or Example: ```tar -xvf devops.de...
Release v00.60.12.100.275 2023-01-12 INTERIM
Release package of current v00.60.XX development operations
UPDATED: 2023-01-12
Interim Release, pending further testing and expansion
v00.60.12
v00.60.12 New Objects Supported
- Added support for Global Properties special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for Policy Settings special object/properties for json and csv export, when exporting domain other than "System Data", or on SMS
- Added support for API Settings special object/properties for json and csv export, when exporting domain "System Data" (also on SMS using domain "System Data")
- Addes support for Radius Server and Radius Group objects for API version 1.9 and later [R81.20 GA], for all operations
- Addes support for Repository Script objects for API version 1.9 and later [R81.20 GA], for all special object operations
- Addes support for SmartTasks objects for API version 1.9 and later [R81.20 GA], for all operations
- Added support for application-site objects url-list and additional-categories sub-CSV files (like group members), done in special objects export script or when enabling export of Critical Performance Impacting (CPI) objects
v00.60.12 Operational Changes
- Added Command Line Parameters to handle specific domains: "System Data" and "Global", --domain-System-Data|--dSD|--dsd and --domain-Global|--dG|--dg respectively, to handle issues with operational scripts and passing quoted parameters with spaces, as well as easier domain specific execution.
- Added support for export of special objects and properties to json
- Added support for basic plumbing for delete, export, import, set/update, rename, and augment CSV files for special objects and properties via CSV
- Added support for per object | special object/properties specific control of utilization of "details-level", "ignore-errors", "ignore-warnings"
- Modified CSV key value sets exported by default for application-site objects
- Added information more detailed error handling mgmt_cli and JQ calls, to help with identification of problems and performance related limitations
- Added object_operations script files for MDSM with max object limit configuration for 100 objects
- Added CLI parameters to determine handling of Critical Performance Impacting (CPI) objects, like application-site objects with > 10,000 Check Point provided objects to handle. Default mode is to exclude CPI objects from export operations
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.12.100.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.12.100.275.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.1.
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package now works with R81.20, but additional testing and work needed, as well as later takes for testing.
- Tested successfully on R81.20 GA T627 JHF N/A
- MDSM testing pending due to infrastructure technical issues with MDSM hosts
(**) Issues with performance throttling and maximum object limits for show operations for large data sets, e.g. application-site objects
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customer chmod 775 /var/log/__customer cd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.12.100.275.tgz
-
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
QUICK START FOR UPDATING
To quickly start working with the scripts if there is an older version installed, do the following.
-
Download the release tgz file or the devops.dev.only.{version}.tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Delete or rename the existing devops.dev folder
Example:
rm /var/log/__customer/devops.dev
or Example:mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.oldrm /var/log/__customer/devops.dev
or
mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.old -
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.only.{version}.tgz
Or Example:tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.only.v00.60.12.100.275.tgz
or
tar -xvf devops.dev.v00.60.12.100.275.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
Key File Hashes
devops.dev.v00.60.12.100.275.tgz
- MD5 : 768E4EB3611062AA3C36B587F3FB51E6
- SHA-1 : D33540B46A7A45E702ADC584EA56609F6FF309F4
- SHA-256 : A0F335E03369B7899B899D0CE2BE8C84B65A1CA9495424B2E95FD465879674C9
devops.dev.only.v00.60.12.100.275.tgz
- MD5 : CF4B81E541BFF90C0DBD3834D820D924
- SHA-1 : A40855162C7168A42C36AAB7B1AD552BA2DE2773
- SHA-256 : FD6C825CC7192A25ED2E5A9BDCB440028AC3C1C53D81B9473C3DD2F104D97924
Generated by MD5 & SHA Checksum Utility @ http://raylin.wordpress.com/downloads/md5-sha-1-checksum-utility
TIPS AND TRICKS
Adding this section regarding approach, especailly with respect to performance related limitations that are encountered on Multi-Domain Security Management (MDSM).
HOW TO DETERMINE THE OPERATIONAL --MAXOBJECTS VALUE
Specifically for MDSM it may be necessary to tweak the execution CLI parameter for --MAXOBJECTS X, which for MDSM is set for 250 objects while for SMS is set for 500, the absolute maximum value for "limit" in a mgmt_cli show call. The easiest way to check what is possible on the target MDSM Multi-Domain Server (MDS) host, is executing a few direct mgmt_cli commands looking for the first success value. Starting at a limit value of 250 objects, work down in 100, 50, or 25 increments to find where there is a success output.
Example, start at 250, check 150, then 125, 100 would have been next:
[Expert@yourhostname:0]# mgmt_cli -r true -d "Global" show application-sites limit 250 offset 0 details-level "full" -f json --conn-timeout 600
{
"code" : "generic_error",
"message" : "Error 502. The Management API service is not available. Please check that the Management API server is up and running."
}
[Expert@yourhostname:0]# mgmt_cli -r true -d "Global" show application-sites limit 150 offset 0 details-level "full" -f json --conn-timeout 600
{
"code" : "generic_error",
"message" : "Error 502. The Management API service is not available. Please check that the Management API server is up and running."
}
[Expert@yourhostname:0]# mgmt_cli -r true -d "Global" sho...Release v00.60.12.000.050 2022-10-27 INTERIM
Release package of current v00.60.XX development operations
UPDATED: 2022-10-27
Interim update
What's New v00.60.12.000
- Added support for export of special objects and properties to json
- Added support for basic plumbing for delete, export, import, set/update, rename, and augment CSV files for special objects and properties via CSV
- Added support for per object | special object/properties specific control of utilization of "details-level", "ignore-errors", "ignore-warnings"
- Added support for Global Properties special object/properties for json export, when exporting domain other than "System Data", or on SMS
- Added support for Policy Settings special object/properties for json export, when exporting domain other than "System Data", or on SMS
- Added support for API Settings special object/properties for json export, when exporting domain "System Data"
- Modified CSV key value sets exported by default for application-site objects
- Added support for application-site objects url-list and additional-categories sub-CSV files (like group members), done in special objects export script
- Added information more detailed error handling mgmt_cli and JQ calls, to help with identification of problems and performance related limitations
- Added object_operations script files for MDSM with max object limit configuration for 100 objects
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.12.000.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.12.000.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.1.
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package now works with R81.20 EA, but additional testing and work needed, as well as later takes for testing.
- Tested successfully on R81.10 JHF OT 79 last
- Tested with partial success on MDSM R81.10 JHF OT 79 (**)
- Tested successfully on R81.20 EA T570 Public EA
(**) Issues with performance throttling and maximum object limits for show operations for large data sets, e.g. application-site objects
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customer chmod 775 /var/log/__customer cd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.12.000.tgz
-
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
QUICK START FOR UPDATING
To quickly start working with the scripts if there is an older version installed, do the following.
-
Download the release tgz file or the devops.dev.only.{version}.tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Delete or rename the existing devops.dev folder
Example:
rm /var/log/__customer/devops.dev
or Example:mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.oldrm /var/log/__customer/devops.dev
or
mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.old -
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.only.{version}.tgz
Or Example:tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.only.v00.60.12.000.tgz
or
tar -xvf devops.dev.v00.60.12.000.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
Key File Hashes
devops.dev.v00.60.12.000.tgz
- MD5 : 27C42E32093961B08FA49F5963FCCB1C
- SHA-1 : BB9897643FA56D62ECF5797B07E4F1C43E9E51E3
- SHA-256 : 7989F4E20F3D35BEEE530A19F5E40CFF1BE9C0A5810B84B2DB15FA791C6D2CFA
devops.dev.only.v00.60.12.000.tgz
- MD5 : 519D26CEE84B8591104688A2021543D8
- SHA-1 : B1C171ACAABC1CFC8B38C6BE27DE5F8631A7E516
- SHA-256 : 86745CA5F3CDBBBC0A75509D48B31B54F8740F7714AE90B4F1F0081B54D874B7
Generated by MD5 & SHA Checksum Utility @ http://raylin.wordpress.com/downloads/md5-sha-1-checksum-utility
TIPS AND TRICKS
Adding this section regarding approach, especailly with respect to performance related limitations that are encountered on Multi-Domain Security Management (MDSM).
HOW TO DETERMINE THE OPERATIONAL --MAXOBJECTS VALUE
Specifically for MDSM it may be necessary to tweak the execution CLI parameter for --MAXOBJECTS X, which for MDSM is set for 250 objects while for SMS is set for 500, the absolute maximum value for "limit" in a mgmt_cli show call. The easiest way to check what is possible on the target MDSM Multi-Domain Server (MDS) host, is executing a few direct mgmt_cli commands looking for the first success value. Starting at a limit value of 250 objects, work down in 100, 50, or 25 increments to find where there is a success output.
Example, start at 250, check 150, then 125, 100 would have been next:
[Expert@yourhostname:0]# mgmt_cli -r true -d "Global" show application-sites limit 250 offset 0 details-level "full" -f json --conn-timeout 600
{
"code" : "generic_error",
"message" : "Error 502. The Management API service is not available. Please check that the Management API server is up and running."
}
[Expert@yourhostname:0]# mgmt_cli -r true -d "Global" show application-sites limit 150 offset 0 details-level "full" -f json --conn-timeout 600
{
"code" : "generic_error",
"message" : "Error 502. The Management API service is not available. Please check that the Management API server is up and running."
}
[Expert@yourhostname:0]# mgmt_cli -r true -d "Global" show application-sites limit 125 offset 0 details-level "full" -f json --conn-timeout 600 | tail
"iso-8601" : "2022-02-25T15:32-0600"
},
"creator" : "System"
},
"read-only" : true
} ],
"from" : 1,
"to" : 125,
"total" : 10052
}
[Expert@yourhostname:0]#
Based on the above example, adding --OVERRIDEMAXOBJECTS --MAXOBJECTS 125 to the command line execution parameters should ensure proper execution and completion; however, the execution increment will produce ome fun numbers in the files generated. Using --OVERRIDEMAXOBJECTS --MAXOBJECTS 100 may be better, but does require more execution cycles.
Release v00.60.11.000.030 2022-06-25 INTERIM
Release package of current v00.60.XX development operations
UPDATED 2022-06-25
Relevant Changes:
-
Addresses issue where service objects that have aggressive aging settings need to be separated between those that are set to use the defautl time out and those that do not use the default timeout, which now generates explicit CSV files for each supported import operation, but also a dedicated reference export that combines all of that service time for review
-
Addresses issue where TACACS Server object import needs specific information for TACACS versus TACACS+ configuration. Specific CSV files for each TACACS server type are generated for input, which also addresses the limitation that the API exports a value for TACACS+ that can't import. Also generate a consolidated reference export.
-
Added support for objects to have an initial object specific selection criteria based on key and value data. Future expansion is possible.
-
Added support for objects type smtp-servers from API version 1.9 (R81.20) forward
-
Added support for objects type network-feeds from API version 1.9 (R81.20) forward
-
Added support for objects type interoperable-devices from API version 1.9 (R81.20) forward
-
Reference only exports now use standard name extension of "REFERENCE_DO_NOT_IMPORT" added to the regular name.
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.11.000.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.11.000.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package now works with R81.20 EA, but additional testing and work needed, as well as later takes for testing.
- Tested successfully on R81.10 JHF OT 61 last
- Tested successfully on R81.20 EA T461 Public EA
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customerchmod 775 /var/log/__customercd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.11.000.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
QUICK START FOR UPDATING
To quickly start working with the scripts if there is an older version installed, do the following.
-
Download the release tgz file or the devops.dev.only.{version}.tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Delete or rename the existing devops.dev folder
Example:
rm /var/log/__customer/devops.dev
or Example:mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.oldrm /var/log/__customer/devops.dev
or
mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.old -
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.only.{version}.tgz
Or Example:tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.only.v00.60.11.000.tgz
or
tar -xvf devops.dev.v00.60.11.000.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
Key File Hashes
devops.dev.v00.60.11.000.tgz
- MD5 : 9B3096CBBC8D4F8D6F1E4FD1AB5FFE55
- SHA-1 : 34F33B49EBD185348605079E655014AE5C184E9A
- SHA-256 : 39D4DAC31E993CDDD3F3193CFCC8ADAAE2C4930E05BC71B8BAED103C13D6378A
devops.dev.only.v00.60.11.000.tgz
- MD5 : 840908920A98EE1848F66261C9FE1BE4
- SHA-1 : 2D4CC96A59BC9F156A3B9253FA3E95F8026AE8A8
- SHA-256 : 3D161DC4BA9F7030248DE5813EC95B7E32EB0714EA752AB526F9921D69A5D541
Generated by MD5 & SHA Checksum Utility @ http://raylin.wordpress.com/downloads/md5-sha-1-checksum-utility
Release v00.60.10.000.060 2022-06-18 INTERIM
Release package of current v00.60.XX development operations
UPDATED 2022-06-18
Relevant Changes:
- Added capability to handle specific criteria for each object type, to handle situation where the exported data can't be imported because it contains values that are not compatible with each over, e.g. aggressive aging configuration value regarding utilization of default time out, or TACACS Servers not working the same for TACACS as TACACS+
- Added new export type "name-for-delete" to simplify export of CSV files needed to generate a set for the delete operation. Manual select of final files and content for delete is strongly recommended, but this removes the need to remember the other parameters needed for the operation
- Updated and corrected --HELP information
- Fixed issues with object query selection for export, which now works as required.
- Fixed cosmetic issue in display of object query parameters in logs
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.10.000.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.10.000.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package now works with R81.20 EA, but additional testing and work needed, as well as later takes for testing.
- Tested successfully on R81.10 JHF OT 61 last
- Tested successfully on R81.20 EA T461 Public EA
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customerchmod 775 /var/log/__customercd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.10.000.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
QUICK START FOR UPDATING
To quickly start working with the scripts if there is an older version installed, do the following.
-
Download the release tgz file or the devops.dev.only.{version}.tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Delete or rename the existing devops.dev folder
Example:
rm /var/log/__customer/devops.dev
or Example:mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.oldrm /var/log/__customer/devops.dev
or
mv /var/log/__customer/devops.dev /var/log/__customer/devops.dev.old -
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.only.{version}.tgz
Or Example:tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.only.v00.60.10.000.tgz
or
tar -xvf devops.dev.v00.60.10.000.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
Key File Hashes
devops.dev.v00.60.10.000.tgz
- MD5 : 71E78752343C1CA4BB4930FA01C0C8ED
- SHA-1 : ACA1037157C1F16FD156518ACAD6C428DA2C9D9E
- SHA-256 : AF8B7AF140A622DFC694EF75DBA4B528055C127D1E819328EB216E24CBF89985
devops.dev.only.v00.60.10.000.tgz
- MD5 : 8153BEB1E4DD5459231EA4FCB05335A9
- SHA-1 : 754D284B48012FD5F4C6C02105F6416150FBD757
- SHA-256 : 2BF40375D8E52602399AE2C28EE2FE002458B2D794753F6FB6933429F7BD9C12
Generated by MD5 & SHA Checksum Utility @ http://raylin.wordpress.com/downloads/md5-sha-1-checksum-utility
Release v00.60.09.020.085 2022-06-13 INTERIM
Release package of current v00.60.XX development operations
UPDATED 2022-06-13
Overhaul of method for creatign export query selection string, now with ability to add another criteria--internal plumbing.
Some adjustments for R81.20 EA testing work
Minor adjustments to the verbose logging output
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.09.020.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.09.020.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.
Tested on R81.20 SMS R8X Management API version 1.9.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package now works with R81.20 EA, but additional testing and work needed, as well as later takes for testing.
- Tested successfully on R81.10 JHF OT 61 last
- Tested successfully on R81.20 EA T437 Public EA
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customerchmod 775 /var/log/__customercd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.09.020.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
Key File Hashes
devops.dev.v00.60.09.020.tgz
- MD5 : 4F8688D51A6A323C2EE4250B8262D7BE
- SHA-1 : 37958CCB31E5535E31A9704498D18F4B4D6E8AE2
- SHA-256 : B10D1DA03B391731190601C1C793E29AE71D24A0D869DC5BAB6A7BE0EBA339A9
Generated by MD5 & SHA Checksum Utility @ http://raylin.wordpress.com/downloads/md5-sha-1-checksum-utility
Release v00.60.09.020.055 2022-06-13 INTERIM
Release package of current v00.60.XX development operations
UPDATED 2022-06-13
Reworked some issues in export handling after adding the -OSO option for Only System Objects.
Also, changed the approach to common exports and testing scripts for the export_import.wip scripts.
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.09.020.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.09.020.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package does not work with R81.20 EA due to some key changes in location of python assets on Gaia, so additional work needed.
- Tested successfully on R81.10 JHF OT 61 last
- Tested successfully on R81.20 EA T347 Public EA!
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customerchmod 775 /var/log/__customercd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.09.020.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
Key File Hashes
devops.dev.v00.60.09.020.tgz
- MD5 : 7BED71C1D916BC49F04C511B7F3A5B89
- SHA-1 : 8BD1FFE038ADC2E8A2C6D63DE2F8FB7B5DF32817
- SHA-256 : 8CDF42F88648604F4BA562E0C217B9B84CF46FAECDA34312B5727FF8DC324816
Generated by MD5 & SHA Checksum Utility @ http://raylin.wordpress.com/downloads/md5-sha-1-checksum-utility
Release v00.60.09.015 2022-06-11 INTERIM
Release package of current v00.60.XX development operations
UPDATED 2022-06-11
Reworked some issues in export handling after adding the -OSO option for Only System Objects.
Also, changed the approach to common exports and testing scripts for the export_import.wip scripts.
OVERVEIW
Operates in the devops.dev folder structure and provides templates with subscripts common to other scripts in the set. Includes example scripts for Check Point Management API based actions for Export, Import, Set [Update], Rename to New-Name, and Delete, and scripts for Zero Lock Session identification and cleanup/removal. Additional scripts examples for MDSM MDM operations (show domains on host) and CSV manipulation tools.
Rough documentation provided in the MD and TSV files (some have Microsoft Excel files analogues).
Identified limitations with certain object types or operations are in the LIMITATIONS_and_CAVEATS.md
To deploy and utilize, it is strongly recommended to copy the "devops.dev.v00.60.09.015.tgz" package to /var/log/__customer and untar-gzip (e.g. tar -xvf devops.dev.v00.60.09.015.tgz /var/log/__customer/ )
Tested on R81.10 MDSM MDS and SMS R8X Management API version 1.8.
Running full scripts with the "--help" option will provide command line options to run the script.
It is strongly recommended to think about credentials used for API operations, especially against other management hosts. Establishing a SmartConsole administrator account that utilizes API-KEY as authentication and then setting the api-key for the operation and using that in the script calls with --api-key option.
Execution of scripts and their capabilities will depend on the authority of the SmartConsole administrator user account authenticated, not the local Gaia OS administrator account.
Development, extension, and refinement continue, and this may not be the last v00.60.xx release.
THANKS
Thank you to those who have assisted with feedback and utilization reports and issues.
CAVEATS
This release package does not work with R81.20 EA due to some key changes in location of python assets on Gaia, so additional work needed.
- Tested successfully on R81.10 JHF OT 61 last
- FAILED on R81.20 EA T347 Public EA!
With exception of the --MaaS (Smart-1 Cloud) authentication functionality and support for objects lsm-gateways and lsm-clusters, this interim release should work as expected and provide working results for all other objects supported and authentication methods.
--MaaS (Smart-1 Cloud) operation authentication and action should work, and are tested, but still "questionable" due to technical issues with access to Smart-1 Cloud tenant on developer side, so any testing and feedback is greatly appreciated.
lsm-gateways and lsm-clusters require more CSV related refinement, but full export of JSON should work as required, the issue is what to export for CSV inclusion to make working import sets. lsm-gateways has a presumptive CSV export and also an additional CSV export with additional information that will not import (basically raw view of JSON exported), and is identified as DO_NOT_IMPORT moniker on CSV file.
QUICK START
To quickly start working with the scripts, do the following.
-
Create the working __customer folder under /var/log, if that does not exist and configure
mkdir /var/log/__customerchmod 775 /var/log/__customercd /var/log/__customer -
Download the release tgz file and deploy to a work folder on the target management host, like /var/log/__customer, the folder should be under the /var/log folder to ensure survival during upgrades
-
Expand the TGZ file, e.g.
Example:
tar -xvf devops.dev.{version}.tgztar -xvf devops.dev.v00.60.09.015.tgz -
Goto to the export import folder
cd ./objects.wip/export_import.wip -
Execute desired script with help parameter to show command options
Example:
./cli_api_export_objects_to_csv.sh --help
Key File Hashes
devops.dev.v00.60.09.015.tgz
- MD5 : DCACD37295D6CFCC94EEBE8831F74DB9
- SHA-1 : C3763C3EE004C882A16D9EA093F9B280D3A198D9
- SHA-256 : 462C8EA36D59667016056B7342B5F14AF2135ED85499FD19BD0A547EBDB10A35