UPDATED 2022-06-25-1310CDT v00.60.11.000.030

UPDATED 2022-06-25-1310CDT v00.60.11.000.030
Refactored object definition data in scripts, reworked some methods on object type selection, corrected some errors that crept in by copy-paste affecting some imports.  Finalized approach for object type specific selectors and standardized reference only export naming for file add-on.

.environment_info.json

@@ -2,8 +2,8 @@
 	"filetype" : "environmentment_info.api_scripts",
 	"fileversion" : "07.00.00",
 	"filerevision" : "000",
-	"filesubrevision" : "060",
-	"filedate" : "2022-06-18",
+	"filesubrevision" : "030",
+	"filedate" : "2022-06-24",
 	"fileinformationlevel" : "000",
 	"MinCMDTemplateVersion" : "07.00.00"
 	}, 
@@ -20,7 +20,7 @@
 		"Language" : "Windows_CMD",
 		"Extension" : "CMD"
 		},
-	"ScriptDate" : "2022-06-18",
+	"ScriptDate" : "2022-06-24",
 	"ScriptVersion" : "06.08.04",
 	"ScriptRevision" : "000",
 	"ScriptTweak" : "15",
@@ -31,13 +31,13 @@
 		"Language" : "bash",
 		"Extension" : "sh"
 		},
-	"ScriptDate" : "2022-06-18",
-	"ScriptVersion" : "00.60.10",
+	"ScriptDate" : "2022-06-24",
+	"ScriptVersion" : "00.60.11",
 	"ScriptRevision" : "000",
-	"ScriptTweak" : "060",
-	"TemplateVersion" : "00.60.10",
+	"ScriptTweak" : "030",
+	"TemplateVersion" : "00.60.11",
 	"APISubScriptsLevel" : "010",
-	"APISubScriptsVersion" : "00.60.10",
+	"APISubScriptsVersion" : "00.60.11",
 	"APISubScriptsRevision" : "000"
 	}
 

File_and_Folder_Purpose_and_Utilization.v00.60.10.000.md → File_and_Folder_Purpose_and_Utilization.v00.60.11.000.md

@@ -1,15 +1,15 @@
 # File and Folder Purpose and Utilization
 
-## UPDATED:  2022-06-18
+## UPDATED:  2022-06-25
 
 Updated to reflect changes to version, folder structure, and filenames.
 
-### Version v00.60.10.000:060 2022-06-18
+### Version v00.60.11.000:030 2022-06-25
 
 The value of ${version} for this document is one of the following:
 
-- Version:  v00.60.10       ${version}
-- Version:  v00.60.10.000   ${version.revision}
+- Version:  v00.60.11       ${version}
+- Version:  v00.60.11.000   ${version.revision}
 
 ## Overview
 

LIMITATIONS_and_CAVEATS.md

@@ -1,6 +1,6 @@
 # LIMITATIONS and CAVEATS
 
-## UPDATED:  2022-06-18
+## UPDATED:  2022-06-25
 
 This document outlines limitations and caveats to the implementation of R8X API export, import, set-update, and delete scripts utilizing bash mgmt_cli commands.
 
@@ -28,7 +28,7 @@ This is a best effort development operation and benefitting of financial incenti
 
 R81.20 EA Public T437 - This release has provided some issues with changes under the hood of Gaia and also some challenges in changes to the API version 1.9 handling of objects on import via mgmt_cli.
 
-- Service objects may fail to import if the values for aggressive aging set use of default timeout, but the column for timeout does not have a zero value, which might be exported because the database includes that issue.  FIXED:  release v00.60.10.000, now create multiple export files depending on the object type parameters.
+- Service objects may fail to import if the values for aggressive aging set use of default timeout, but the column for timeout does not have a zero value, which might be exported because the database includes that issue.  FIXED:  release v00.60.11.000, now create multiple export files depending on the object type parameters.
 - User and User Template objects import has some issues that still need investigation
 
 ## LIMITATIONS and CAVEATS Authentication
@@ -100,3 +100,17 @@ Currently RADIUS server object and RADIUS servers group object types do not exis
 - lsm-gateway and lsm-cluster objects are Work-In-Progress (WIP) and may not provide CSV results of value until further data can be collected and analyzed
 
 - v00.60.08.060 :  lsm-gateway[s] now provides limited CSV data that can work for import, as well as a CSV export NOT_FOR_IMPORT of raw information from the lsm-gateway objects as a reference for the implemented system.
+
+### SMTP Server Objects
+
+- The password field won't export, so it is populated with a placeholder value that needs to be either set before import, or reset on the object
+
+- Since the smtp-server object can exist with our without username and password, additional harvesting is required to harvest those smtp-servers with username and password (authenticaton = true), as well as those without username and password (authenticaton = false)
+
+### Network Feed Objects
+
+- This object has an option to configure any number of custom-header names and values and the current scripting (v00.60.11) harvests the first five (5) of these value pairs.  If more are required, either editing and expanding the number to cover the needed additional values is required.
+
+### Interoperable Device Objects
+
+- Since these are essentially third-party gateways, or gateways managed by other management servers, there are many more interfaces configured than just a single one.  The current implementation only harvests the first interface.  Additional interface harvesting may happen later, if code for that process is not significantly difficult.

README.md

@@ -4,13 +4,13 @@ Check Point R8x Export, Import, Set/Update, Rename to new-name, and Delete mgmt_
 
 Additional documentation and information will be provided in .md, .tsv, and potentially .xlsx files in the repository.
 
-## UPDATED:  2022-06-18
+## UPDATED:  2022-06-25
 
 Interim update, reorganize the README.MD and add a Quick Start
 
 ## Overview
 
-The export, import, set-update, rename-to-new-name, and delete using CSV files scripts in this post, currently version 00.60.10.000 dated 2022-06-18, are intended to allow operations on an existing R80, R80.10, R80.20[|.M1|.M2], R80.30, R80.40, R81, R81.10 and EA R81.20 [WIP (Work-In-Progress)]] Check Point management server (SMS or MDM) from bash expert mode on the Check Point management server host or another API enabled Check Point management server host instance (Check Point Gaia OS R8X) able to authenticate and reach the target management server host.  Utilization from other LINUX releases is not supported, tested, or assumed to work.
+The export, import, set-update, rename-to-new-name, and delete using CSV files scripts in this post, currently version 00.60.11.000 dated 2022-06-25, are intended to allow operations on an existing R80, R80.10, R80.20[|.M1|.M2], R80.30, R80.40, R81, R81.10 and EA R81.20 [WIP (Work-In-Progress)]] Check Point management server (SMS or MDM) from bash expert mode on the Check Point management server host or another API enabled Check Point management server host instance (Check Point Gaia OS R8X) able to authenticate and reach the target management server host.  Utilization from other LINUX releases is not supported, tested, or assumed to work.
 
 - Check Point Management API documentation is here:
 <https://sc1.checkpoint.com/documents/latest/APIs/index.html#introduction>
@@ -83,7 +83,7 @@ To quickly start working with the scripts, do the following.
 
       Example:  ```tar -xvf devops.dev.{version}.tgz```
 
-      ```tar -xvf devops.dev.v00.60.10.000.tgz```
+      ```tar -xvf devops.dev.v00.60.11.000.tgz```
 
 4. Goto to the export import folder
 

WHATS_NEW.v00.60.11.000.md

@@ -0,0 +1,31 @@
+# R8x-export-import-api-scripts - What's NEW in v00.60.11
+
+What's NEW in Check Point R8x Export, Import, Set/Update, Rename to new-name, and Delete mgmt_cli API scripts for bash on Check Point Gaia OS management hosts, using CSV files (v00.60.xx.yyy)
+
+## UPDATED:  2022-06-25
+
+Interim update
+
+## What's New
+
+As of v00.60.11 this file will document major changes and additions to the version.
+
+- Specific limitations and caveats are added to the LIMITATIONS_and_CAVEATS.md file for reference.
+
+- Details of object type specific capabilities are documented in the supported_objects.{version}.tsv
+
+### v00.60.11
+
+- Addresses issue where service objects that have aggressive aging settings need to be separated between those that are set to use the defautl time out and those that do not use the default timeout, which now generates explicit CSV files for each supported import operation, but also a dedicated reference export that combines all of that service time for review
+
+- Addresses issue where TACACS Server object import needs specific information for TACACS versus TACACS+ configuration.  Specific CSV files for each TACACS server type are generated for input, which also addresses the limitation that the API exports a value for TACACS+ that can't import.  Also generate a consolidated reference export.
+
+- Added support for objects to have an initial object specific selection criteria based on key and value data.  Future expansion is possible.
+
+- Added support for objects type smtp-servers from API version 1.9 (R81.20) forward
+
+- Added support for objects type network-feeds from API version 1.9 (R81.20) forward
+
+- Added support for objects type interoperable-devices from API version 1.9 (R81.20) forward
+
+- Reference only exports now use standard name extension of "REFERENCE_DO_NOT_IMPORT" added to the regular name.

devops.dev/.environment_info.json

@@ -2,8 +2,8 @@
 	"filetype" : "environmentment_info.api_scripts",
 	"fileversion" : "07.00.00",
 	"filerevision" : "000",
-	"filesubrevision" : "060",
-	"filedate" : "2022-06-18",
+	"filesubrevision" : "030",
+	"filedate" : "2022-06-24",
 	"fileinformationlevel" : "000",
 	"MinCMDTemplateVersion" : "07.00.00"
 	}, 
@@ -20,7 +20,7 @@
 		"Language" : "Windows_CMD",
 		"Extension" : "CMD"
 		},
-	"ScriptDate" : "2022-06-18",
+	"ScriptDate" : "2022-06-24",
 	"ScriptVersion" : "06.08.04",
 	"ScriptRevision" : "000",
 	"ScriptTweak" : "15",
@@ -31,13 +31,13 @@
 		"Language" : "bash",
 		"Extension" : "sh"
 		},
-	"ScriptDate" : "2022-06-18",
-	"ScriptVersion" : "00.60.10",
+	"ScriptDate" : "2022-06-24",
+	"ScriptVersion" : "00.60.11",
 	"ScriptRevision" : "000",
-	"ScriptTweak" : "060",
-	"TemplateVersion" : "00.60.10",
+	"ScriptTweak" : "030",
+	"TemplateVersion" : "00.60.11",
 	"APISubScriptsLevel" : "010",
-	"APISubScriptsVersion" : "00.60.10",
+	"APISubScriptsVersion" : "00.60.11",
 	"APISubScriptsRevision" : "000"
 	}
 

devops.dev/File_and_Folder_Purpose_and_Utilization.v00.60.10.000.md → devops.dev/File_and_Folder_Purpose_and_Utilization.v00.60.11.000.md

@@ -1,15 +1,15 @@
 # File and Folder Purpose and Utilization
 
-## UPDATED:  2022-06-18
+## UPDATED:  2022-06-25
 
 Updated to reflect changes to version, folder structure, and filenames.
 
-### Version v00.60.10.000:060 2022-06-18
+### Version v00.60.11.000:030 2022-06-25
 
 The value of ${version} for this document is one of the following:
 
-- Version:  v00.60.10       ${version}
-- Version:  v00.60.10.000   ${version.revision}
+- Version:  v00.60.11       ${version}
+- Version:  v00.60.11.000   ${version.revision}
 
 ## Overview
 

devops.dev/LIMITATIONS_and_CAVEATS.md

@@ -1,6 +1,6 @@
 # LIMITATIONS and CAVEATS
 
-## UPDATED:  2022-06-18
+## UPDATED:  2022-06-25
 
 This document outlines limitations and caveats to the implementation of R8X API export, import, set-update, and delete scripts utilizing bash mgmt_cli commands.
 
@@ -28,7 +28,7 @@ This is a best effort development operation and benefitting of financial incenti
 
 R81.20 EA Public T437 - This release has provided some issues with changes under the hood of Gaia and also some challenges in changes to the API version 1.9 handling of objects on import via mgmt_cli.
 
-- Service objects may fail to import if the values for aggressive aging set use of default timeout, but the column for timeout does not have a zero value, which might be exported because the database includes that issue.  FIXED:  release v00.60.10.000, now create multiple export files depending on the object type parameters.
+- Service objects may fail to import if the values for aggressive aging set use of default timeout, but the column for timeout does not have a zero value, which might be exported because the database includes that issue.  FIXED:  release v00.60.11.000, now create multiple export files depending on the object type parameters.
 - User and User Template objects import has some issues that still need investigation
 
 ## LIMITATIONS and CAVEATS Authentication
@@ -100,3 +100,17 @@ Currently RADIUS server object and RADIUS servers group object types do not exis
 - lsm-gateway and lsm-cluster objects are Work-In-Progress (WIP) and may not provide CSV results of value until further data can be collected and analyzed
 
 - v00.60.08.060 :  lsm-gateway[s] now provides limited CSV data that can work for import, as well as a CSV export NOT_FOR_IMPORT of raw information from the lsm-gateway objects as a reference for the implemented system.
+
+### SMTP Server Objects
+
+- The password field won't export, so it is populated with a placeholder value that needs to be either set before import, or reset on the object
+
+- Since the smtp-server object can exist with our without username and password, additional harvesting is required to harvest those smtp-servers with username and password (authenticaton = true), as well as those without username and password (authenticaton = false)
+
+### Network Feed Objects
+
+- This object has an option to configure any number of custom-header names and values and the current scripting (v00.60.11) harvests the first five (5) of these value pairs.  If more are required, either editing and expanding the number to cover the needed additional values is required.
+
+### Interoperable Device Objects
+
+- Since these are essentially third-party gateways, or gateways managed by other management servers, there are many more interfaces configured than just a single one.  The current implementation only harvests the first interface.  Additional interface harvesting may happen later, if code for that process is not significantly difficult.

devops.dev/README.md

@@ -4,13 +4,13 @@ Check Point R8x Export, Import, Set/Update, Rename to new-name, and Delete mgmt_
 
 Additional documentation and information will be provided in .md, .tsv, and potentially .xlsx files in the repository.
 
-## UPDATED:  2022-06-18
+## UPDATED:  2022-06-25
 
 Interim update, reorganize the README.MD and add a Quick Start
 
 ## Overview
 
-The export, import, set-update, rename-to-new-name, and delete using CSV files scripts in this post, currently version 00.60.10.000 dated 2022-06-18, are intended to allow operations on an existing R80, R80.10, R80.20[|.M1|.M2], R80.30, R80.40, R81, R81.10 and EA R81.20 [WIP (Work-In-Progress)]] Check Point management server (SMS or MDM) from bash expert mode on the Check Point management server host or another API enabled Check Point management server host instance (Check Point Gaia OS R8X) able to authenticate and reach the target management server host.  Utilization from other LINUX releases is not supported, tested, or assumed to work.
+The export, import, set-update, rename-to-new-name, and delete using CSV files scripts in this post, currently version 00.60.11.000 dated 2022-06-25, are intended to allow operations on an existing R80, R80.10, R80.20[|.M1|.M2], R80.30, R80.40, R81, R81.10 and EA R81.20 [WIP (Work-In-Progress)]] Check Point management server (SMS or MDM) from bash expert mode on the Check Point management server host or another API enabled Check Point management server host instance (Check Point Gaia OS R8X) able to authenticate and reach the target management server host.  Utilization from other LINUX releases is not supported, tested, or assumed to work.
 
 - Check Point Management API documentation is here:
 <https://sc1.checkpoint.com/documents/latest/APIs/index.html#introduction>
@@ -83,7 +83,7 @@ To quickly start working with the scripts, do the following.
 
       Example:  ```tar -xvf devops.dev.{version}.tgz```
 
-      ```tar -xvf devops.dev.v00.60.10.000.tgz```
+      ```tar -xvf devops.dev.v00.60.11.000.tgz```
 
 4. Goto to the export import folder
 

devops.dev/Session_Cleanup.wip/Session_Cleanup_version.v05.60.10.010.template.v00.60.10.000.version → devops.dev/Session_Cleanup.wip/Session_Cleanup_version.v05.60.11.000.template.v00.60.11.000.version

@@ -16,13 +16,13 @@
 # SCRIPTS Session_Cleanup operations
 #
 #
-ScriptVersion=05.60.10
+ScriptVersion=05.60.11
 ScriptRevision=000
-ScriptSubRevision=060
-ScriptDate=2022-06-18
-TemplateVersion=00.60.10
+ScriptSubRevision=030
+ScriptDate=2022-06-24
+TemplateVersion=00.60.11
 APISubscriptsLevel=010
-APISubscriptsVersion=00.60.10
+APISubscriptsVersion=00.60.11
 APISubscriptsRevision=000
 
 

devops.dev/Session_Cleanup.wip/remove_zerolocks_sessions.v05.60.10.sh → devops.dev/Session_Cleanup.wip/remove_zerolocks_sessions.v05.60.11.sh

@@ -16,13 +16,13 @@
 # SCRIPT for BASH to remove zero locks sessions
 #
 #
-ScriptVersion=05.60.10
+ScriptVersion=05.60.11
 ScriptRevision=000
-ScriptSubRevision=060
-ScriptDate=2022-06-18
-TemplateVersion=00.60.10
+ScriptSubRevision=030
+ScriptDate=2022-06-24
+TemplateVersion=00.60.11
 APISubscriptsLevel=010
-APISubscriptsVersion=00.60.10
+APISubscriptsVersion=00.60.11
 APISubscriptsRevision=000
 
 

devops.dev/Session_Cleanup.wip/remove_zerolocks_web_api_sessions.v05.60.10.sh → devops.dev/Session_Cleanup.wip/remove_zerolocks_web_api_sessions.v05.60.11.sh

@@ -16,13 +16,13 @@
 # SCRIPT Remove zerolocks sessions with web_api user
 #
 #
-ScriptVersion=05.60.10
+ScriptVersion=05.60.11
 ScriptRevision=000
-ScriptSubRevision=060
-ScriptDate=2022-06-18
-TemplateVersion=00.60.10
+ScriptSubRevision=030
+ScriptDate=2022-06-24
+TemplateVersion=00.60.11
 APISubscriptsLevel=010
-APISubscriptsVersion=00.60.10
+APISubscriptsVersion=00.60.11
 APISubscriptsRevision=000
 
 

devops.dev/Session_Cleanup.wip/show_zerolocks_sessions.v05.60.10.sh → devops.dev/Session_Cleanup.wip/show_zerolocks_sessions.v05.60.11.sh

@@ -16,13 +16,13 @@
 # SCRIPT Show zerolocks sessions all users
 #
 #
-ScriptVersion=05.60.10
+ScriptVersion=05.60.11
 ScriptRevision=000
-ScriptSubRevision=060
-ScriptDate=2022-06-18
-TemplateVersion=00.60.10
+ScriptSubRevision=030
+ScriptDate=2022-06-24
+TemplateVersion=00.60.11
 APISubscriptsLevel=010
-APISubscriptsVersion=00.60.10
+APISubscriptsVersion=00.60.11
 APISubscriptsRevision=000
 
 

devops.dev/Session_Cleanup.wip/show_zerolocks_web_api_sessions.v05.60.10.sh → devops.dev/Session_Cleanup.wip/show_zerolocks_web_api_sessions.v05.60.11.sh

@@ -16,13 +16,13 @@
 # SCRIPT Show zerolocks sessions with web_api user
 #
 #
-ScriptVersion=05.60.10
+ScriptVersion=05.60.11
 ScriptRevision=000
-ScriptSubRevision=060
-ScriptDate=2022-06-18
-TemplateVersion=00.60.10
+ScriptSubRevision=030
+ScriptDate=2022-06-24
+TemplateVersion=00.60.11
 APISubscriptsLevel=010
-APISubscriptsVersion=00.60.10
+APISubscriptsVersion=00.60.11
 APISubscriptsRevision=000
 
 

devops.dev/WHATS_NEW.v00.60.11.000.md

@@ -0,0 +1,31 @@
+# R8x-export-import-api-scripts - What's NEW in v00.60.11
+
+What's NEW in Check Point R8x Export, Import, Set/Update, Rename to new-name, and Delete mgmt_cli API scripts for bash on Check Point Gaia OS management hosts, using CSV files (v00.60.xx.yyy)
+
+## UPDATED:  2022-06-25
+
+Interim update
+
+## What's New
+
+As of v00.60.11 this file will document major changes and additions to the version.
+
+- Specific limitations and caveats are added to the LIMITATIONS_and_CAVEATS.md file for reference.
+
+- Details of object type specific capabilities are documented in the supported_objects.{version}.tsv
+
+### v00.60.11
+
+- Addresses issue where service objects that have aggressive aging settings need to be separated between those that are set to use the defautl time out and those that do not use the default timeout, which now generates explicit CSV files for each supported import operation, but also a dedicated reference export that combines all of that service time for review
+
+- Addresses issue where TACACS Server object import needs specific information for TACACS versus TACACS+ configuration.  Specific CSV files for each TACACS server type are generated for input, which also addresses the limitation that the API exports a value for TACACS+ that can't import.  Also generate a consolidated reference export.
+
+- Added support for objects to have an initial object specific selection criteria based on key and value data.  Future expansion is possible.
+
+- Added support for objects type smtp-servers from API version 1.9 (R81.20) forward
+
+- Added support for objects type network-feeds from API version 1.9 (R81.20) forward
+
+- Added support for objects type interoperable-devices from API version 1.9 (R81.20) forward
+
+- Reference only exports now use standard name extension of "REFERENCE_DO_NOT_IMPORT" added to the regular name.