From 136c0bbcae255b885427ddfd060cf6914a12e84e Mon Sep 17 00:00:00 2001 From: Shelly Tzohar <45915502+Shellyber@users.noreply.github.com> Date: Wed, 9 Oct 2024 11:00:18 +0300 Subject: [PATCH] Removed sdk docs and refer links (#1632) * Removed sdk docs and refer links * Another link changes * More updates * removed articles --- .../extra-docs/packs/content-management.md | 20 +- docs/concepts/demisto-sdk-docker.md | 112 ---- docs/concepts/demisto-sdk-validate.md | 381 ------------ docs/concepts/demisto-sdk.md | 562 ------------------ docs/concepts/dev-setup.md | 11 +- docs/concepts/faq.md | 2 +- docs/concepts/getting-started-guide.md | 2 +- docs/concepts/vscode-extension.md | 4 +- docs/contributing/checklist.md | 4 +- docs/contributing/contributing.md | 6 +- docs/documentation/readme_file.md | 6 +- docs/documentation/release-notes.md | 4 +- docs/integrations/code-conventions.md | 2 +- docs/integrations/openapi-codegen.md | 102 ---- docs/integrations/package-dir.md | 4 +- docs/integrations/postman-codegen.md | 86 --- docs/integrations/powershell-code.md | 2 +- docs/integrations/test-playbooks.md | 2 +- docs/packs/packs-format.md | 2 +- docs/partners/adopt.md | 2 +- docs/tutorials/tut-setup-dev.md | 10 +- sidebars.js | 3 - 22 files changed, 37 insertions(+), 1292 deletions(-) delete mode 100644 docs/concepts/demisto-sdk-docker.md delete mode 100644 docs/concepts/demisto-sdk-validate.md delete mode 100644 docs/concepts/demisto-sdk.md delete mode 100644 docs/integrations/openapi-codegen.md delete mode 100644 docs/integrations/postman-codegen.md diff --git a/content-repo/extra-docs/packs/content-management.md b/content-repo/extra-docs/packs/content-management.md index f12390074..487fbf08a 100644 --- a/content-repo/extra-docs/packs/content-management.md +++ b/content-repo/extra-docs/packs/content-management.md @@ -33,7 +33,7 @@ The CI/CD process helps you develop and maintain content for complex content dev The process is designed for more advanced users who have an understanding of CI/CD concepts and have a number of developers who work on local machines. In particular, it is recommended that you should have the following knowledge: - The ability to set up a build system - - The ability to use [demisto-sdk](https://xsoar.pan.dev/docs/concepts/demisto-sdk) + - The ability to use [demisto-sdk](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Introduction-to-Demisto-SDK) - A basic knowledge of Git - A basic knowledge of containers (Docker) @@ -67,11 +67,11 @@ For general information about the CI/CD process, see [CI/CD FAQs](#cicd-faqs). ## Development - In the development stage, [set up the CI/CD process](#setup-the-cicd-development-process) by creating or cloning a Git repository based on the [CI/CD template repository](https://github.com/demisto/content-ci-cd-template). You can then create and maintain your custom content with the built-in functionality of [demisto-sdk](https://xsoar.pan.dev/docs/concepts/demisto-sdk). The CI/CD process uses `demisto-sdk` to develop and deploy custom content packs. It downloads, uploads, validates, creates content, migrates content from a Cortex XSOAR server, etc. It also supports content pack structure generation, auto validation of custom content including running linters on the code, and generation of content pack documentation, etc. For a full list of features, commands, and arguments, see [demisto-sdk-commands](https://github.com/demisto/demisto-sdk#commands). + In the development stage, [set up the CI/CD process](#setup-the-cicd-development-process) by creating or cloning a Git repository based on the [CI/CD template repository](https://github.com/demisto/content-ci-cd-template). You can then create and maintain your custom content with the built-in functionality of [demisto-sdk](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Introduction-to-Demisto-SDK). The CI/CD process uses `demisto-sdk` to develop and deploy custom content packs. It downloads, uploads, validates, creates content, migrates content from a Cortex XSOAR server, etc. It also supports content pack structure generation, auto validation of custom content including running linters on the code, and generation of content pack documentation, etc. For a full list of features, commands, and arguments, see [demisto-sdk-commands](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Demisto-SDK-commands). #### Set Up the CI/CD Development Process - Before you begin, download and install the `demisto-sdk` on your machine. For more information, see [Demisto Installation and Setup](https://xsoar.pan.dev/docs/concepts/demisto-sdk#installation-and-setup). + Before you begin, download and install the `demisto-sdk` on your machine. For more information, see [Demisto Installation and Setup](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Install-Demisto-SDK). Ensure that the `DEMISTO_BASE_URL` and `DEMISTO_API_KEY` are set to the server environment that you want to use. For example, if you want to upload content directly to the server without using an artifact server you need to point the server to this environment. When working on a branch and you want to download/upload to a development environment, ensure that the `demisto-sdk` points to the development server. 1. **Create the CI/CD Repository.** @@ -86,7 +86,7 @@ For general information about the CI/CD process, see [CI/CD FAQs](#cicd-faqs). | `.vscode` | Used when using VSC as your IDE. | | `build_related_scripts` | Contains the CI/CD scripts. The `build_related_scripts/bucket_upload.py` script enables you to upload to Google Cloud Storage (artifact server) and the `build_related_scripts/bucket_upload_aws.py` script enables you to upload to AWS s3. Before deploying your content, you need to update the name of the bucket list when uploading the Google Cloud Storage. For more information, see Configure the the bucket_upload.py file in [Deployment](#deployment).
The `get_modified_packs.py` script enables you to get the latest version of the content pack before merging. | | `dev_envs/pytest`| A folder that contains the `conftest.py`, which validates python files. | - | `.demisto-sdk-conf`| The custom configuration file for the `demisto-sdk` commands. For more information, see [Setting a preset custom command configuration](https://xsoar.pan.dev/docs/concepts/demisto-sdk#setting-a-preset-custom-command-configuration). | + | `.demisto-sdk-conf`| The custom configuration file for the `demisto-sdk` commands. For more information, see [Setting a preset custom command configuration](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Create-a-preset-custom-command-configuration). | | `.gitignore` | Specifies intentionally untracked files that Git should ignore. | | `.private-repo-settings` | Set who can view your repository. For more information, see [Setting repository visibility](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/setting-repository-visibility). | | `CommonServerPowerShell.ps1` | Required when using Powershell. For more information, see [Powershell code](https://xsoar.pan.dev/docs/integrations/powershell-code). | @@ -202,7 +202,7 @@ For general information about the CI/CD process, see [CI/CD FAQs](#cicd-faqs). Run the `demisto-sdk format` command, which ensures that your new or modified files comply with Cortex XSOAR's standard. To check that your code is valid, run the `demisto-sdk validate` command and the `demisto-sdk lint` command on your code.
- **Troubleshooting:** If you receive many errors from the validate/lint commands, it is not a major issue, as you can control the [validations](https://xsoar.pan.dev/docs/concepts/demisto-sdk#validate) you want to ignore. + **Troubleshooting:** If you receive many errors from the validate/lint commands, it is not a major issue, as you can control the [validations](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/validate) you want to ignore. For example, if you receive the `BA111` - Problematic fields names error, or the `IN135` or `IN145` - Integration params issues, you can add it to the `.pack-ignore` file in your pack like this: ``` [file:some_playbook.yml] @@ -585,7 +585,7 @@ The structure of the repository is as follows: | README.md | A markdown file that provides a description of the pack. | | xsoar_config.json
[Example file](https://raw.githubusercontent.com/demisto/content/master/Packs/ContentManagement/doc_files/xsoar_config.json) | The configuration file that defines what packs lists, and jobs will be set up on the machine.
It consists of the following sections:
- *custom_packs* - Your own internal packs to be installed through the build process.
- *marketplace_packs* - Marketplace packs to be installed on the machine.
- *lists* - Lists to be created in the machine, when the list type set to dynamic then the list won’t be overwritten.
- *jobs* - Jobs to be created in the machine. | | .private-repo-settings | --- | -| .demisto-sdk-conf | Your custom configuration file for the demisto-sdk commands. For details, click [here](https://xsoar.pan.dev/docs/concepts/demisto-sdk#setting-a-preset-custom-command-configuration). | +| .demisto-sdk-conf | Your custom configuration file for the demisto-sdk commands. For details, click [here](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Create-a-preset-custom-command-configuration). | | requirements.txt | Contains a list of all the project’s dependencies. | | tox.ini | The command-line driven automated testing tool for Python. | | demistomock.py | You can copy this file from the Content repo (content/Tests/demistomock/). | @@ -602,13 +602,13 @@ Create a yml file for the CI/CD process. The [Example File](https://raw.githubus 1. Prepare the environment and the virtual environment on which to run the demisto-sdk. 2. Create an ID set for the private repository using the [demisto-sdk create-id-set](https://github.com/demisto/demisto-sdk/tree/master/demisto_sdk/commands/create_id_set) command. 3. Merge the ID set with the Content repository's ID set using the following command: ***demisto-sdk merge-id-sets -i1 -i2 -o ***. -4. Validate the packs' files using the [demisto-sdk validate](https://xsoar.pan.dev/docs/concepts/demisto-sdk#validate) command. -5. Run unit tests and linters on the packs using the [demisto-sdk lint](https://xsoar.pan.dev/docs/concepts/demisto-sdk#lint) command. -6. Create uploadable pack zips using the [demisto-sdk zip-packs](https://github.com/demisto/demisto-sdk/tree/master/demisto_sdk/commands/zip_packs) command. +4. Validate the packs' files using the [demisto-sdk validate](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/validate) command. +5. Run unit tests and linters on the packs using the [demisto-sdk pre-commit](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/pre-commit) command. +6. Create uploadable pack zips using the [demisto-sdk zip-packs](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/zip-packs) command. ​ Or ​ - Upload zipped packs directly to your machine using the [demisto-sdk upload](https://xsoar.pan.dev/docs/concepts/demisto-sdk#upload) command. + Upload zipped packs directly to your machine using the [demisto-sdk upload](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/upload) command. 7. Upload the artifacts to your artifact repository. ​ ​ diff --git a/docs/concepts/demisto-sdk-docker.md b/docs/concepts/demisto-sdk-docker.md deleted file mode 100644 index 8bdfbdb08..000000000 --- a/docs/concepts/demisto-sdk-docker.md +++ /dev/null @@ -1,112 +0,0 @@ -# Demisto-SDK docker - -Run Demisto-SDK validations from within a docker container. - -You can use this image to run Demisto-SDK commands locally or as a CI/CD process. - -## Get The Docker Image - -Pull the docker image with: -`docker pull demisto/demisto-sdk:` - -You can find the latest tags in the docker hub: -`http://hub.docker.com/r/demisto/demisto-sdk` - -## The Content Repository - -To use the Demisto-SDK, ensure you have a content-like repository with Cortex XSOAR content in a structure that matches the official [XSOAR Content repo](https://github.com/demisto/content). - -You can generate such a repository using the following [template](https://github.com/demisto/content-external-template) - -## Mounts - -Demisto-SDK uses volume mounts to run on the local content repository. -_Please note that mounting on macOS and Windows may cause slowness._ - -To ensure the best performance, please either: - -- Use a Linux machine -- Use [Windows WSL2](https://docs.microsoft.com/en-us/windows/wsl/install) - -## Environment Variable - -Some commands such as `demisto-sdk upload` and `demisto-sdk run` need the following environment variables to communicate with your XSOAR Server: - -- `DEMISTO_BASE_URL` - The URL of the XSOAR server to communicate with -- `DEMISTO_API_KEY` - API Key (Can be generated from XSOAR -> Settings -> API Key) -- `DEMISTO_VERIFY_SSL` (Default: true) - Whether to verify SSL certificates. - -To pass those variables, you should add the following option: - -```sh -docker run --env DEMISTO_BASE_URL="https://xsoar.com:443" -``` - -You can also use an env file: - -.env - -```sh -DEMISTO_BASE_URL="https://xsoar.com:443" -DEMISTO_API_KEY="xxxxxxxxxxxxx" -``` - -Command: - -```sh -docker run --env-file .env -``` - -## Docker In Docker (Docker Daemon Binding) - -To achieve Docker In Docker behavior. We want to bind the Docker Daemon with the following option: - -- `--mount source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind` - Mounts the docker daemon container to use docker commands from within a docker container. - -## Examples - -(All examples use Cortex XSOAR's official [content repository](https://github.com/demisto/content)). - -## Alias for easy usage - -You can create an alias to the command by adding the following line to your shell configuration files: - -```sh -alias demisto-sdk="docker run -it --rm \ ---mount type=bind,source="$(pwd)",target=/content \ ---mount source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind \ -demisto/demisto-sdk:" -``` - -### Validate command - -For more information about the validate command, please refer to its [documentation.](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/validate/README.md) on the [demisto-sdk repo](https://github.com/demisto/demisto-sdk). - -```sh -docker run -it --rm \ ---mount type=bind,source="$(pwd)",target=/content \ ---mount source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind \ -demisto/demisto-sdk: \ -demisto-sdk validate -i Packs/ipinfo/Integrations/ipinfo_v2 -``` - -#### Breaking down command arguments - -- `docker run` - Creates a container (if one does not exist) and runs the following command inside it -- `-it` - Keep the stdin open and connects tty -- `--rm` - Removes the docker container when done (ommit this part to re-use the container in the future) -- `--mount type=bind,source="$(pwd)",target=/content` - Connects the pwd (assuming you're in content) to the container's content directory -- `--mount source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind` -Bind the docker daemon to the container to enable execute docker-from-docker. -- `demisto/demisto-sdk:\` (Replace the tag with locked version, can be found at the [Docker Hub](https://hub.docker.com/r/demisto/demisto-sdk)) - The docker image name -- `demisto-sdk validate -i Packs/ipinfo/Integrations/ipinfo_v2` - The demisto-sdk command to be run inside the container diff --git a/docs/concepts/demisto-sdk-validate.md b/docs/concepts/demisto-sdk-validate.md deleted file mode 100644 index bbefa7383..000000000 --- a/docs/concepts/demisto-sdk-validate.md +++ /dev/null @@ -1,381 +0,0 @@ ---- -id: demisto-sdk-validate -title: Demisto-SDK Validate checks ---- - -This file contains information about our new validate flow. For more information about the old validate flow, refer to [old_validate_readme](demisto_sdk/commands/validate/old_validate_readme.md). - -## Validate - -Makes sure your content repository files are in order and have a valid file scheme. - -**Use Cases** -This command is used to make sure that the content repo files are valid and are able to be processed by the platform. -This is used in our validation process both locally and in gitlab. - -**Arguments**: -* **-g, --use-git** -Validate changes using git - this will check the current branch's changes against origin/master. -If the **--post-commit** flag is supplied: validation will run only on the current branch's changed files that have been committed. -If the **--post-commit** flag is not supplied: validation will run on all changed files in the current branch, both committed and not committed. -* **-a, --validate-all** -Whether to run all validation on all files or not. -* **-i, --input** -The path of the content pack/file to validate specifically. -* **-pc, --post-commit** -Whether the validation should run only on the current branch's committed changed files. This applies only when the **-g** flag is supplied. -* **-st, --staged** -Whether the validation should run only on the current branch's staged files. This applies only when the **-g** flag is supplied. -* **--prev-ver** -Previous branch or SHA1 commit to run checks against. -* **--no-multiprocessing** -Run validate all without multiprocessing, for debugging purposes. -* **-j, --json-file** -The JSON file path to which to output the command results. -* **--category-to-run** -Run specific validations by stating the category they're listed under in the config file. -* **-f, --fix** -Whether to autofix failing validations with an available auto fix or not. -* **--config-path** -Path for a config file to run. If not given - will run the default path at: [demisto_sdk/commands/validate/default_config.toml](default_config.toml) -* **--ignore-support-level** -Whether to skip validations based on their support level or not. -* **--run-old-validate** -Whether to run the old validate flow or not. Alternatively, you can configure the RUN_OLD_VALIDATE env variable -* **--skip-new-validate** -Whether to skip the new validate flow or not. Alternatively, you can configure the SKIP_NEW_VALIDATE env variable. -* **-sv, --run-specific-validations** -A comma separated list of validations to run stated the error codes. -* **--ignore** -An error code to not run. To ignore more than one error, repeat this option (e.g. `--ignore AA123 --ignore BC321`) -**Examples**: - -`demisto-sdk validate --prev-ver SHA1-HASH` -This will validate only changed files from the branch given (SHA1). - -`demisto-sdk validate --post-commit` -This indicates that the command runs post commit. - -`demisto-sdk validate -i Packs/HelloWorld/Integrations/HelloWorld/HelloWorld.yml` -This will validate the file Packs/HelloWorld/Integrations/HelloWorld/HelloWorld.yml only. - -`demisto-sdk validate -a` -This will validate all files under the `Packs` directory. - -`demisto-sdk validate -i Packs/HelloWorld` -This will validate all files under the content pack `HelloWorld`. - -`demisto-sdk validate --run-old-validate --skip-new-validate -a` -This will validate all files in the repo using the old validate method. - -`demisto-sdk validate --config-path {config_file_path} -a` -This will validate all files in the repo using the settings configured in the config file in the given path. - -### Error Codes and Ignoring Them -Each error found by validate has an error code attached to it. The code can be found in brackets preceding the error itself. -For example: `path/to/file: [IN103] - The type field of the proxy parameter should be 8` -In addition, each pack has a `.pack-ignore` file. In order to ignore a certain validation for a given file, the error-code needs to be listed in the **ignorable_errors** section in the config-file (see the [Config file section](#config-file)), and the user needs to mention the file name (only the name and extension, without the whole path), and the error code to ignore. -For example: This .pack-ignore will not fail ipinfo_v2.yml on the validations with the codes BA108 & BA109. -[file:ipinfo_v2.yml] -ignore=BA108,BA109 - -### Config file -You can define a config file to suit your business needs. If no file is defined, the [default config file](default_config.toml) will be used. -The default configuration covers basic validations, which prevents unsuccessful uploads of the validated content to Cortex XSOAR. -#### How to define a configuration file -You can define the following sections: -**ignorable_errors** - a list of the error codes that can be ignored for individual content items in the .pack-ignore file. -**path_based_validations** - the configurations to run when running with -a / -i flags. -**use_git** - the configurations to run when running with -g flag. -You can also define custom sections - which can be configured to run with the **category-to-run** flag. -Two example custom categories are given with the default config file: -**xsoar_best_practices_use_git** - our recommended set of validations to run when running with -g, may be modified from time to time. -**xsoar_best_practices_path_based_validations** - our recommended set of validations to run when running with -a / -i, may be modified from time to time. -Each section will have the following options: -**select** - The validations to run. -**warning** - Validations for which to only throw warnings (will not fail the flow). -The config file can also configure which validations to ignore based on the content item support level using the section header support_level.<support_type> where support_type is one of xsoar, partner, or community. -If the user wishes to ignore this feature in some of the calls, he can use the **--ignore-support-level** flag. - -**Examples**: -``` -[custom_category] -select = ["BA101"] -``` -Validate will run only BA101 validation. - -``` -[custom_category] -select = ["BA100", "BA101", "BA102"] -[support_level.community] -ignore = ["BA102"] -``` -Validate will run all the validations with error codes "BA100", "BA101", "BA102" except for BA102 in case of community supported files. - - -## `BA`: Basic -| Code | Description | Rationale | Autofixable | -|--------|-------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| BA100 | Marketplace content set to -1 makes it easier to tell from modified, versioned content. | The version for system content items should always be -1 as per the standard. | Yes | -| BA101 | Validate that the file id and name fields are identical. | The id attribute serves as the unique identifier of files across the platform | Yes | -| BA105 | Validate that the content item ID doesn't include any slashes (/). | ids with slashes are not allowed in the platform | Yes | -| BA106 | Validate that the item's fromversion field is sufficient. | This field makes sure content can use the latest and greatest features of the platform. The minimal value is the third-last platform release version. | Yes | -| BA108 | Check if there are separators in the folder name. | To ensure consistent, readable folder structures by avoiding separators like spaces, underscores, or hyphens. | | -| BA109 | Check if there are separators in the script or integration files names. | Filenames for scripts and integrations should not contain separators to maintain consistency and readability. | | -| BA110 | Check that the entity name or display name does not contain the entity type. | Improves clarity and simplicity in the content repository | | -| BA111 | Checks whether the name of a content item contains an excluded word. | Increases clarity by keeping content names simple | | -| BA113 | Checks for content item names with trailing spaces. | Ensures accurate referencing. | Yes | -| BA114 | Validate that we didn't move a content item from one pack to another. | Pack of a content item should not be changed. | | -| BA116 | validate that the CLI name and the id match for incident and indicators field | Consistency between the CLI name (used by the platform) and the id. | Yes | -| BA118 | Validate that the item's toversion is greater/equal then its fromversion. | Content with a from_version greater than to_version will not show in the platform. | Yes | -| BA119 | Validate that the python file doesn't have a copyright section with the words - BSD, MIT, Copyright, proprietary. | Content in the Cortex marketplace is licensed under the MIT license. | | -| BA124 | Validate that the script / integration has a unit test file. | Unit tests make sure that the behaviors in code are consistent between versions. | | -| BA125 | Validate that customer facing docs and fields don't contain any internal terms that aren't clear for customers. | Ensure customer-facing docs avoid internal terms for clarity. | | -| BA126 | Checks if script/integration is deprecated correctly | Deprecated scripts/integrations need clear descriptions for user guidance. For deprecation process, see: https://xsoar.pan.dev/docs/reference/articles/deprecation-process-and-hidden-packs#how-to-deprecate-and-hide-a-pack | | -## `BC`: Backward Compatability -| Code | Description | Rationale | Autofixable | -|--------|-------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| BC100 | Validate that the pack name subtype of the new file matches the old one. | Changing 'subtype' can break backward compatibility. For 'subtype' info, see: https://xsoar.pan.dev/docs/integrations/yaml-file#script | Yes | -| BC101 | Validate that no context output keys were removed from the script's output section. | To ensure we don't break bc. | | -| BC102 | Validate that the context path has been changed. | Changing the paths may break dependent content items, which rely on the existing paths. | | -| BC103 | Check if an argument name has been changed. | If an existing argument has been renamed, it will break backward compatibility. | | -| BC104 | Check if the command name or argument name has been changed. | If an existing command or argument has been renamed, it will break backward compatibility | | -| BC105 | Validate that the ID of the content item was not changed. | Changing the ID of an item will cause it to show as a new item in the platform, and not update properly for users who already have a previous version installed. | Yes | -| BC106 | Check that the fromversion property was not changed on existing Content files. | Changing the `fromversion` for a content item can break backward compatibility. For 'fromversion' info, see: https://xsoar.pan.dev/docs/integrations/yaml-file#version-and-tests | | -| BC107 | Check that the toversion property was not changed on existing Content files. | Changing the `toversion` field for a content item can break backward compatibility. | | -| BC108 | Ensuring that the 'marketplaces' property hasn't been removed or added in a manner that effectively removes all others. | Removing `marketplaces` or adding a new one that effectively removes all others can cause issues with the content item's visibility and availability. | | -| BC110 | Validate that no new *required* argument are added to an existing command. | Adding a new argument to an existing command and defining it as *required* or changing an non-required argument to be required will break backward compatibility. | | -| BC111 | Ensure that no new *required* arguments are added to an existing script. | Adding a new required argument or changing a non-required one to required without specifying a default value breaks backward compatibility. | | -| BC112 | Ensure that no parameters are removed from an existing integration. | Removed parameters can cause errors if the parameter is needed by the server of integration code. | | -| BC113 | Validate that no incident types were removed and no incident fields were changed. | We want to ensure no breaking changes are made to existing mappers so customers won't lose data between pack updates. | | -| BC114 | Ensure a pre-defined list of fields wasn't removed or modified in the integration yml. | We wish to keep our behavior between different versions of the content item so we wish to enforce backwards compatibility breaking. | | -## `CL`: Classifier -| Code | Description | Rationale | Autofixable | -|--------|---------------------------------------------------------------|-------------------------------------------------------------------------------------------|---------------| -| CL100 | Validate that a classifier has a type = classification field. | This standardization is for the platform to correctly identify and handle the classifier. | Yes | -## `CR`: Correlation Rule -| Code | Description | Rationale | Autofixable | -|--------|-------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|---------------| -| CR102 | Validates 'search_window' existence and non-emptiness for 'execution_mode' = 'SCHEDULED'. | 'SCHEDULED' execution must have a defined time frame 'search_window' to operate within | | -## `DA`: Dashboard -| Code | Description | Rationale | Autofixable | -|--------|------------------------------------------------------------------|--------------------------------------------------------------------------------------------|---------------| -| DA100 | Validate that the dashboard excludes all the unnecessary fields. | The Dashboard should contain only the required fields. | Yes | -| DA101 | Validate that the dashboard includes all the necessary fields. | The Dashboard should contains the required 'fromDate', 'toDate', 'fromDateLicense' fields. | | -## `DO`: Docker Image -| Code | Description | Rationale | Autofixable | -|--------|----------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| DO100 | Validate that the given content-item does not use the tag 'latest' in its docker image | Locking content to use a specific tag of a docker image ensures stability. The tag is usually updated in newer versions of the content item.For more details on Docker, visit https://xsoar.pan.dev/docs/integrations/docker. | Yes | -| DO101 | Validate that the given content-item uses demisto docker image | The Docker image used in a content item should originate from Demisto's Docker repository. This ensures that the Docker image is trusted, maintained, and compatible with the Demisto platform. | | -| DO102 | Validate that the given content item uses a docker image that is not the native image. | The 'native-image' Docker image is intended for internal development and should not be used for running integrations or scripts. | | -| DO103 | Validate that the given content item's docker-image actually exists in dockerhub | Platform supported content must have their images available in dockerhub so they can be widely pulled and used. | | -| DO104 | Validate that the given content item has a docker_image. | Python and Powershell content run in containers. | | -| DO105 | Validate that the given content item uses a docker image that is not deprecated | It is best practice to use images that are maintained by the platform. | | -| DO106 | Validate that the given content-item's docker image isn't outdated. | Updated Docker images ensure that the code doesn't use outdated dependencies, including bug fixes and fixed vulnerabilities. | Yes | -## `DS`: Description -| Code | Description | Rationale | Autofixable | -|--------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| DS100 | Check whether a description file exists for a beta integration. | Need a disclaimer for beta integrations. | | -| DS101 | Check if beta disclaimer exists in detailed description | Need a disclaimer for beta integrations. | | -| DS104 | Verifies that a Description file is present for an integration. | It is recommended to have a Description file for each integration, which for example will have additional details on how to configure the instance. | | -| DS105 | check if DESCRIPTION file contains contribution details. | the contribution/partner details will be generated automatically and we don't want should be duplicate. | | -| DS106 | Check if the description file exist and the name is valid. | We want to make sure all integrations have all required documentation and that the file name is according to our standards. | | -| DS107 | Validate that none of the description lines contains the the word 'demisto'. | Ensure that the current name of the product is used rather than the old one. | | -| DS108 | Ensure that all yml's description fields ends with a dot. | To ensure high documentation standards. | Yes | -## `GF`: Generic Field -| Code | Description | Rationale | Autofixable | -|--------|-----------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| GF100 | Checks if group field is set to 4. | Required by the platform. | Yes | -| GF101 | Checks if the id starts with `generic_`. | Required by the platform. | | -| GF102 | Checks if the unsearchable key is set to true | Marking many items searchable causes index and search loads on the platform. Official demisto/content GenericField files must be set to Unsearchable. In custom content, it's recommended to keep the number of searchable fields to a minimum. | | -## `GR`: Graph -| Code | Description | Rationale | Autofixable | -|--------|------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| GR100 | Detect content items that attempt to use other content items which are not supported in all of the marketplaces of the content item. | Content graph proper construction. | | -| GR103 | Validates that there is no usage of unknown content items | Content items should only use existing content items. | | -| GR104 | Validate that there are no duplicate display names of packs in the repo | Prevent confusion between packs. | | -| GR105 | Ensures that each content item has a unique ID to prevent conflicts. | Duplicate IDs can cause conflicts and confusion. | | -| GR106 | Checks that every test playbook is linked to at least one content item. (the content item has a 'tests:' key with the ID of the test playbook) | In the demisto/content repo, unlinked test playbooks are not run in CI (for PRs) unless the test playbook itself is modified. Proper linkage of test playbooks ensures content quality. See https://xsoar.pan.dev/docs/integrations/test-playbooks#adding-the-playbook-to-your-project | | -## `IF`: Incident Field -| Code | Description | Rationale | Autofixable | -|--------|---------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| IF100 | Checks if the name and cliName do not contain potential incident synonyms. | The name and cliName fields are limited by the platform. | | -| IF101 | Checks if the incident field is marked as content. | Required by the platform. | Yes | -| IF102 | Checks if system flag is false. | Required by the platform. | Yes | -| IF103 | Checks if given field type is valid. | The types of the IncidentField are limited by the platform. | | -| IF104 | Checks if group field is set to 0. | Required by the platform. | Yes | -| IF105 | Checks if cliName field is alphanumeric and lowercase. | Required by the platform. | | -| IF106 | Checks if `cliName` field is not a reserved word. | The cliName values of the incident field are limited by the platform. | | -| IF111 | Checks if the field type changed. | Changing type of IncidentField is not allowed by the platform. | Yes | -| IF113 | Checks if field name starts with its pack name or one of the item prefixes from pack metadata. | Required by the platform. | | -| IF115 | Checks if the `unsearchable` key set to true. | Marking many items searchable causes index and search loads on the platform. Official demisto/content IncidentField files must be set to Unsearchable. In custom content, it's recommended to keep the number of searchable fields to a minimum. | | -| IF116 | We do not allow for incidentFields with multySelect types to have in the selectValues emtpy options | Due to UI issues, we cannot allow empty values for selectValues field | Yes | -| IF119 | We do not allow for incidentFields with singleSelect types to have in the selectValues more than one or only emtpy option | Due to UI issues, we cannot allow more than one or only empty values for selectValues field | Yes | -## `IM`: Author Image -| Code | Description | Rationale | Autofixable | -|--------|----------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| IM100 | Checks if the integration has an image path. | Images make it easier to find integrations. | | -| IM101 | Checks that the image file size are matching the requirements. | Image needs to fit its place in the UI. For more information see: https://xsoar.pan.dev/docs/integrations/integration-logo | | -| IM106 | Checks if the integration has an image other than the default ones. | If an image is provided, it must not be the default ones. | | -| IM108 | Checks that the author image file is not empty | If an author image is provided, it must be a valid image. For more info, see: https://xsoar.pan.dev/docs/packs/packs-format#author_imagepng | | -| IM109 | Checks if the pack has an author image path. | Author images make it easier to identify the author. | | -| IM111 | Checks that the image file dimensions are matching the requirements. | Image needs to fit its place in the UI. For more information see: https://xsoar.pan.dev/docs/integrations/integration-logo | | -## `IN`: Integration -| Code | Description | Rationale | Autofixable | -|--------|----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| IN100 | Validate that the proxy & insecure params are configured correctly. | The 'proxy' and 'insecure' parameters in an integration are builtin platform parameters. | Yes | -| IN101 | Outputs of reputation commands must adhere to standards. | Uniform outputs allow creating generic content. For more information, see https://xsoar.pan.dev/docs/integrations/generic-commands-reputation | | -| IN102 | Validate that the checkbox param is configured correctly with required argument set to false. | A checkbox parameter that is required will fail (count as missing) when unchecked, thus forcing the users to always check it, practically turning it into a constant `True` value, rather than a dynamic checkbox. | Yes | -| IN104 | Validate that the Integrations category is valid. | See the list of allowed categories in the platform: https://xsoar.pan.dev/docs/documentation/pack-docs#pack-keywords-tags-use-cases--categories | | -| IN106 | Validate that the command is valid as a reputation command. | Reputation commands must follow standards for consistency and compatibility. For more details, see https://xsoar.pan.dev/docs/integrations/generic-commands-reputation | | -| IN107 | Validate that the reputation commands include the list of required contextPaths. | Reputation commands must include required contextPaths for consistency and reliable use in playbooks or scripts. For more details, see https://xsoar.pan.dev/docs/integrations/context-standards-mandatory | | -| IN108 | Validate wether the subtype is valid or not. | This field describes the major python version, `python2` or `python3`. | | -| IN109 | Validate that the ID field doesn't include the substring 'beta'. | The ID field in an integration should not contain the word 'beta'. This ensures unambiguous identification of production-ready integrations. | Yes | -| IN110 | Validate that the name field doesn't include the substring 'beta'. | The name field in an integration should not contain the word 'beta'. This ensures unambiguous identification of production-ready integrations. | Yes | -| IN112 | Validate that the display name contain the substring 'beta'. | Beta integrations should have 'beta' in the display name for clear identification and to manage user expectations. | | -| IN113 | Prevent duplicate arguments for integrations. | Duplicate arguments cause confusion and unpredictable behaviors. | | -| IN114 | Validate that there're no duplicated params for the integration. | Duplicate parameters cause confusion and unpredictable behaviors. | | -| IN115 | Validate that the contextPath field of each output is in the right format. | The 'contextPath' in an integration's output must follow the correct format to ensure data can be used in subsequent playbook steps. Incorrect 'contextPath' formats can lead to errors or unexpected behavior. For more details, see https://xsoar.pan.dev/docs/integrations/context-and-outputs | | -| IN117 | Validate that type 17 configuration params doesn't include the display field. | The display name is handle by the platform. | Yes | -| IN118 | Validate that the integration parameter has a display field if it's not of type 17. | Integration parameters should have a 'display' field for clear user understanding, except for type 17 parameters. | | -| IN121 | Validate that fetch integration has the required params in the right format. | Malformed or missing parameters can lead to errors or incomplete data. For more details, see https://xsoar.pan.dev/docs/integrations/fetching-incidents | | -| IN122 | Validate that all existing params are in the right format for feed params. | Malformed or missing parameters can lead to errors or incomplete data. For more details, see https://xsoar.pan.dev/docs/integrations/feeds | | -| IN123 | Checks if integration display name is versioned correctly, e.g.: ends with v<number>. | Integration display names should end with 'v<number>' for version clarity and consistency. | Yes | -| IN124 | Validate that a param is not hidden if it can not be hidden. | Hiding these parameters can lead to confusion and may prevent the integration from working as expected. Only the following parameters may be hidden: {'feedIncremental', 'feedReputation', 'longRunning'} | Yes | -| IN125 | Validate that the max_fetch param has a defaultvalue | To maintain optimal load in the platform, the maximal number of incidents per fetch is limited. For more details, see https://xsoar.pan.dev/docs/integrations/fetching-incidents#fetch-limit | Yes | -| IN126 | Validate that a fetch integration is not missing the first_fetch & max_fetch params. | 'first_fetch' and 'max_fetch' parameters in fetch integrations ensure correct incident retrieval. Their absence or incorrect format can lead to errors or inconsistencies. For more details, see https://xsoar.pan.dev/docs/integrations/fetching-incidents#first-run | | -| IN127 | Validate that a deprecated integration display name ends with (Deprecated). | Deprecated integrations should end with (Deprecated) in the display name to clearly indicate their status. This prevents inadvertent use of unsupported integrations. For more details, see https://xsoar.pan.dev/docs/reference/articles/deprecation-process-and-hidden-packs#how-to-deprecate-an-integration | Yes | -| IN130 | validate that the integration is runable | Integrations must have a functional purpose, such as executing commands, fetching incidents, fetching indicators from a feed, or running a long-running process. | | -| IN131 | Validate that the integration is valid as a mappable integration. | For easy debugging and troubleshooting, integrations supporting schema mapping (ismappable: true) should include the 'get-mapping-fields' command. This validator ensures the command is present in the integration YAML, enhancing maintainability and user experience.For more info, visit https://xsoar.pan.dev/docs/integrations/mirroring_integration#get-mapping-fields | | -| IN134 | Checks whether a content item has multiple default arguments. | Multiple default arguments are not supported by the platform, and could lead to unexpected behavior. For more info about command arguments, see https://xsoar.pan.dev/docs/integrations/yaml-file#command-arguments | | -| IN135 | Validate that the parameter display name starts with a capital letter and doesn't contain '_'. | Parameter display names should start with a capital letter and not contain underscores for consistency and readability. | Yes | -| IN139 | Validate that there's no 'incident' in any of the commands names or arguments names for core packs integrations. | This helps maintain the flexibility of the platform. | | -| IN141 | Validate that an endpoint command has at least one of the required fields. | Without them, the command may not function properly or may return incomplete or incorrect data. for more info see https://xsoar.pan.dev/docs/integrations/generic-endpoint-command | | -| IN142 | Validate that the integration contain the right additionalinfo fields for the list of params with predefined additionalinfo. | Ensuring the 'additionalinfo' fields are correctly set in an integration's parameters promotes consistency and enhances user understanding of each parameter's purpose and usage. | Yes | -| IN144 | Validate that a reputation command has isArray field set to True for its default argument. | Reputation commands often get multiple inputs to enrich. Without isArray=true, providing an array of inputs may impact performance.For more info about reputation commands, see https://xsoar.pan.dev/docs/integrations/generic-commands-reputation | | -| IN145 | Validate that xsoar supported integrations don't have a non-hidden type 4 params. | Parameters that contain API tokens or credentials should be of type 'Credentials' (type 9) instead of 'Encrypted' (type 4) to allow fetching credentials from an external vault. This ensures secure and efficient handling of sensitive data. Using the 'Encrypted' type could prevent the integration from fetching the credentials from an external vault, which could lead to operational issues. For more info see https://xsoar.pan.dev/docs/integrations/code-conventions#credentials | | -| IN146 | Validate that there's no fromlicense param field in non Xsoar supported integration | The `fromlicense` param is intended for XSOAR-supported integrations, as they rely on values coming stored in the platform. | Yes | -| IN149 | Validate that a contextPath has a description if it belongs to a predefined list of contextPaths that should have a description. | Common outputs in integrations need descriptions for clarity and effective usage. | Yes | -| IN151 | Validate that all commands has at least one argument | This prevents potential errors during execution due to missing arguments. | Yes | -| IN152 | Validate that the default value of a checkbox param is valid. | Checkbox parameters' default values should be lowercase boolean strings ('true', 'false') for correct interpretation and functionality. | Yes | -| IN153 | Validate that that the url default param starts with https rather than http. | URL parameters should default to 'https' for secure communication. 'http' could expose sensitive data. | Yes | -| IN154 | Validate that feed integration and integrations with reputation commands have a reliability param. | The reliability parameter is required to set indicator's reliability. For more info see, https://xsoar.pan.dev/docs/integrations/feeds#required-parameters | | -| IN156 | Validate that the hidden field value contain only valid values. | Incorrect values can cause unexpected behavior or compatibility issues. | | -| IN158 | Validate that the description for non-deprecated integrations is not in the deprecation format | This avoids confusion and ensures that users are correctly informed about the integration's status. For more about deprecation see: https://xsoar.pan.dev/docs/reference/articles/deprecation-process-and-hidden-packs#how-to-deprecate-an-integration | | -| IN159 | Validate that the capitalization of reputation command specific keys is correct. | This ensures consistency and effective data passage between playbook tasks. For more about the standard context output for reputation commands see: https://xsoar.pan.dev/docs/integrations/context-standards-mandatory | | -| IN160 | Validate that the display name for non-deprecated integration doesn't end with '(Deprecated)'. | This ensures accurate representation of the integration's status, avoiding confusion for users For more about deprecation see: https://xsoar.pan.dev/docs/reference/articles/deprecation-process-and-hidden-packs#how-to-deprecate-an-integration | | -| IN161 | Validate that the marketplacev2 tag appear in the marketplaces list of a siem integration. | SIEM integrations must have the 'marketplacev2' tag for visibility in marketplacev2, enhancing usability. Refer to https://xsoar.pan.dev/docs/integrations/event-collectors#required-keys. | Yes | -| IN162 | Validate that the support level header for a collector integration in a Partner pack is set to Xsoar. | Collector integrations in Partner packs should specify {XSOAR_SUPPORT} level support to accurately inform users about the support level provided by Cortex XSOAR. For more information about 'support level header' see https://xsoar.pan.dev/docs/documentation/integration-description#support-level-header-yml-metadata-key. | Yes | -| IN163 | Validate feedExpirationPolicy parameter is in the right formatfor both incremental and fully fetched feeds | Malformed expiration policy can lead to errors or incomplete data.For more details, see https://xsoar.pan.dev/docs/integrations/feeds | Yes | -## `IT`: Incident Type -| Code | Description | Rationale | Autofixable | -|--------|--------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| IT100 | Checks if the included fields have a positive integer value. | Fields that have to be included, cannot be of type integer. | | -| IT101 | Checks if playbook ID is valid. | Playbook ID has to be a non-UUID format. | | -| IT102 | Check if extractSettings field is valid. | extractSettings field is supposed to be in the correct format. | | -| IT103 | Check if auto extract mode valid. | auto extract mode is supposed to be: 'All' To extract all indicator types regardless of auto-extraction settings.'Specific' - To extract only the specific indicator types set in the auto-extraction settings. | | -## `LO`: Layout -| Code | Description | Rationale | Autofixable | -|--------|-----------------------------------------------------------------------------------|--------------------------|---------------| -| LO107 | Ensures that only supported types are used in the layout for XSIAM compatibility. | Limited by the platform. | | -## `MR`: Modeling Rule -| Code | Description | Rationale | Autofixable | -|--------|--------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| MR100 | Validate that each modeling rule has a corresponding schema file. | For each modeling rule, there has to be schema file. | | -| MR101 | Validate that the modeling rules keys - "rules" and "schema" exist and are empty | This validation is for compatibility resaons. Without those fields the modeling rules won't work. | | -| MR106 | Verifies that each specified data type in the schema files of modeling rules conforms to a predefined set of acceptable types. | Validating types in schema files is crucial for maintaining data integrity and compatibility across systems, preventing errors and ensuring reliable data processing. | | -| MR107 | Validate that the dataset name of a modeling rule shows in the xif and schema files match. | We want to make sure the datasets match between the schema and the XIF file to avoid discrepancy between the expected info and the info shown in the UI. | | -| MR108 | Checks that id and name in the modeling rule, end with the correct suffixes. | To prevent confusion caused by ambiguous naming of modeling and parsing rules in XSIAM UI, the validation ensures the rule ID and name end with 'ModelingRule' or 'Modeling Rule'. This will help avoid naming conflicts and improve clarity in release notes. | | -## `PA`: Pack -| Code | Description | Rationale | Autofixable | -|--------|----------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| PA100 | Validate that all the tags in tags field have a valid prefix. | This standardization allows for efficient filtering and grouping in the marketplace. For more info about the valid tags see https://xsoar.pan.dev/docs/documentation/pack-docs#pack-keywords-tags-use-cases--categories | Yes | -| PA101 | Validate that the version mentioned in the Pack metadata matches the latest RN version. | Clear documentation for each version change helps users know what's new.For more information, see https://xsoar.pan.dev/docs/packs/packs-format#content-packs-versioning | | -| PA102 | Validate that the pack is deprecated if it needs to. | This ensures clarity for users and prevents potential confusion of deprecated content. For more about deprecation see: https://xsoar.pan.dev/docs/reference/articles/deprecation-process-and-hidden-packs | Yes | -| PA103 | Validate that the pack categories are valid. | See the list of allowed categories in the platform: https://xsoar.pan.dev/docs/documentation/pack-docs#pack-keywords-tags-use-cases--categories | | -| PA104 | Validate that the modules field include only labels from the list of allowed labels. | See the list of allowed modules in the platform: compliance. | Yes | -| PA105 | Validate that the pack has the marketplacev2 label if it include modules. | This field is only used in XSIAM. | Yes | -| PA107 | Ensure that mandatory fields exist in the pack_metadata. | Missing these fields may lead to unexpected behavior when uploading the packs. | Yes | -| PA108 | Validate that the pack name field exist and is different from the default one. | A unique and meaningful pack name is crucial for identifying the pack and its contents. | | -| PA109 | Validate that the metadata description field isn't empty and is different from the default one. | A meaningful description helps users understand how the pack can help them. | | -| PA111 | Validate that certain metadata fields are not empty. | 'keywords', 'tags', 'categories', and 'useCases' fields in the pack metadata are mandatory for pack identification and searchability. For more info, see: https://xsoar.pan.dev/docs/documentation/pack-docs#pack-keywords-tags-use-cases--categories | | -| PA113 | Validate that a partner/developer pack has at least an email or a url address fields filled. | Packs with Partner or developer support levels require a contact method (email or URL) for user support and information. | | -| PA114 | Ensure that the pack metadata version is raised on relevant changes. | When updating a pack, its version needs to be raised to maintain traceability. | | -| PA115 | Validate that the pack_metadata created field is in ISO format. | The format is required by the platform. | Yes | -| PA117 | Validate that the pack's support type is a valid support type. | For valid support levels, see https://xsoar.pan.dev/docs/packs/packs-format#pack_metadatajson. | | -| PA118 | Validate that the metadata's certification field is valid. | See the list of allowed `certification` in the platform: https://xsoar.pan.dev/docs/packs/packs-format#pack_metadatajson. | | -| PA119 | Validate that the metadata's usecases field include valid usecases. | See the list of allowed `useCases` in the platform: https://xsoar.pan.dev/docs/documentation/pack-docs#pack-keywords-tags-use-cases--categories | Yes | -| PA120 | Validate that metadata's tag section include only approved tags. | Using approved tags makes it easier for users to find the packs that suit their needs. For more info, see https://xsoar.pan.dev/docs/documentation/pack-docs#pack-keywords-tags-use-cases--categories | Yes | -| PA121 | Validate that no changes were done to the pack's price. | Changing this field affects paying customer. In the demisto/content repo, this requires a force-merge. | Yes | -| PA123 | Validate that the pack has at least one of PB, incidents Types or Layouts if the tags section contains the 'Use Case' tag. | Correct categorization helps users find packs that suit their needs. | Yes | -| PA125 | Validate that the pack name is valid. | Pack names should follow conventions for consistency and readability in the marketplace. | | -| PA127 | Validate that the pack metadata contains a valid URL field. | URLs help users access support or report issues for the pack directly. For more info, see: https://xsoar.pan.dev/docs/packs/packs-format#pack_metadatajson | Yes | -| PA128 | Checks for required pack files | These files are standard in the demisto/content repo. | Yes | -| PA130 | Validate that the pack_metadata version field is in valid format. | Content versions use semantic versioning to make it easy to tell how significant changes are between two versions. | | -| PA131 | Validate that the pack_metadata contains a default datasource, if there are more than one datasource. | Wizards and other tools rely on the default datasource to be set. | Yes | -| PA132 | Validate that the pack_metadata contains a valid default datasource, when provided. | Wizards and other tools rely on the default datasource to be set. | Yes | -## `PB`: Playbook -| Code | Description | Rationale | Autofixable | -|--------|-------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| PB100 | Validate whether the playbook has a rolename. If the Playbook has a rolename it is not valid. | We shouldn't ship playbooks with a role set as this is customisable by the customer. | | -| PB101 | Checks whether an ask conditional has task with unreachable next task condition. | Checks whether an ask conditional has task with unreachable next task condition. | | -| PB103 | Validate whether there is an unconnected task. | Make sure there are no unconnected tasks to ensure the playbook will work as expected. | | -| PB104 | Validate whether a deprecated playbook has a valid description. | Description of deprecated content should be consistent. | | -| PB105 | Validate whether the playbook has a DeleteContext with all set to 'Yes'. If the Playbook has it, it is not valid. | Playbook can not have DeleteContext script with arg all set to yes. | | -| PB106 | Validate whether the playbook does not use an instance. If the Playbook use an instance it is not valid. | If the playbook uses a specific instance it can leads to errors because not all the users have the same instance. | Yes | -| PB108 | Validate that the task ID and the 'id' under the 'task' field are from UUID format. | Each task should have a unique id in UUID format to avoid unknown behavior and breaking the playbook. | | -| PB109 | Check that taskid field and id field under task field contains equal values. | System requirements | | -| PB114 | Validates that playbooks for indicator types will be on quiet mode. | Playbooks for indicators will likely be executing on thousands of indicators so they need to be on quiet mode. | Yes | -| PB115 | Checks if the 'quietmode' field of all tasks in playbook are not in default value. | Confirmation of turning off quitmode | Yes | -| PB116 | The validation checks that all playbook tasks stop when encountering an error. | For indicator playbooks, tasks will likely be executing on thousands of indicators. Without these validations in place, we may easily release playbooks for general availability that can crash Demisto instances if a playbook task does not stop on error, causing following tasks to rely on its output and fail. | | -| PB118 | Validate that all inputs described in the playbooks input section are used in tasks. | For more info, see: https://xsoar.pan.dev/docs/playbooks/playbooks-overview#inputs-and-outputs | | -| PB119 | Validates that all inputs used are defined. | Inputs that are used but not provided to a playbook is probably an oversight. | | -| PB122 | Validate whether branches of built-in conditional tasks are handled properly. | Ensures the playbook logic is complete. | | -| PB123 | Checks whether an ask conditional has unhandled reply options. | Checks whether an ask conditional has unhandled reply options. | | -| PB124 | Make sure that all conditional tasks contains at least 2 next tasks. | Ensure we don't miss unhandled cases in our playbook. | | -| PB125 | Validates that a condition task doesn't has only a default next-task. | Validates that a condition task doesn't has only a default next-task. | | -| PB126 | Ensure that conditional tasks have an execution path besides for the default. | We want to ensure that conditional tasks have more than path which is not the default one o/w it make no sense to have such. | | -| PB127 | Ensure that any yml keys that are marketplace only have a default non-marketplace counterpart. | To be able to validate the existence of required yml keys, we need to make sure that in addition to a specific marketplace key (say mykey:xsoar) there is also a basic key with no specific marketplace (for example: mykey). | Yes | -## `PR`: Parsing Rule -| Code | Description | Rationale | Autofixable | -|--------|-----------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| PR101 | Checks that id and name in the parsing rule, end with the correct suffixes. | To prevent confusion caused by ambiguous naming of modeling and parsing rules in XSIAM UI, the validation ensures the rule ID and name end with 'ParsingRule' or 'Parsing Rule'. This will help avoid naming conflicts and improve clarity in release notes. | | -## `RM`: Readme -| Code | Description | Rationale | Autofixable | -|--------|----------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| RM101 | Validate images absolute paths, and prints the suggested path if it's not valid. | In official marketplace content, ensures that the images can be used in the upload flow properly. | | -| RM102 | Validates that all context outputs defined in the README file are present in the YML file, and vice versa. | Ensuring consistency between the README and YML files helps maintain accurate documentation and prevents discrepancies. | | -| RM104 | Validate that the pack contains a full README.md file with pack information. | Meaningful, complete documentations make it easier for users to use the content. | | -| RM105 | Checks if the README.md file is not same as the pack description. | An informative README helps users know more about the product and its uses. | | -| RM106 | Validate that none of the readme lines contains the the word 'demisto'. | Ensure that the current name of the product is used rather than the old one. | | -| RM107 | Checks if there are the generic sentence '%%FILL HERE%%' in the README content. | Ensuring our documentation looks good and professional. | | -| RM108 | This validation verifies that images in the readme and description files are relative and stored in doc_files. | Using relative references to files in the repo folder enhances security by reducing reliance on external links, minimizing the risk of link manipulation or redirection attacks. | | -| RM109 | Validates if there is a readme file for the content item. | Ensure that the content item contains additional information about use-cases, inputs, and outputs. | | -| RM110 | Validates that all commands are mentioned in the README file | Ensuring all commands are documented in the README helps users understand the available functionality | | -| RM113 | Validate that non of the readme lines contains the disallowed copyright section keywords. | Content in the Cortex marketplace is licensed under the MIT license. | | -| RM114 | Validate that images placed under doc_files folder and used in README exist. | Missing images are not shown in rendered markdown | | -## `RN`: Release Note -| Code | Description | Rationale | Autofixable | -|--------|----------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|---------------| -| RN103 | Validate that the pack contains a full release note file. | Meaningful, complete documentations make it easier for users to use the content. | | -| RN105 | Validate there're no more than one Added rn for each pack. | Having more than one release note for a version may cause confusion and missing information. | | -| RN108 | Validate that a new pack doesn't have a RN, | New Packs doesn't require release notes since they don't have any updates. | | -| RN112 | Validate that if RN contains 'breaking change' then the breaking change release note exist and filled correctly. | Breaking changes should be well documented so they can pop up to users when updating versions. | | -| RN114 | Validate the existence of content types in the first-level headers (####) and the content items in second-level headers (#####). | Providing documentation with accurate information and avoiding confusion. | | -## `RP`: Reputation (Incident Type) -| Code | Description | Rationale | Autofixable | -|--------|------------------------------------------------------------------------|-------------------------------------------------------------|---------------| -| RP101 | Validate that the 'expiration' field has a non-negative integer value. | To align with the platform requirements. | | -| RP102 | Validate that the id and the details fields are equal. | To align with the platform requirements. | | -| RP103 | Validate that the 'id' field of indicator type has valid value. | we want to make sure the id of the indicator type is valid. | | -## `SC`: Script -| Code | Description | Rationale | Autofixable | -|--------|-------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| SC100 | Checks if script name is versioned correctly, e.g.: ends with V<number>. | This standardization ensures consistency across content items. | Yes | -| SC105 | Checks that script arguments do not container the word incident in core packs | Server has a feature where the word 'incident' in the system can be replaced by any other keyword of the user's choice. To ensure compatibility with this feature, we should make sure that command names, command arguments, and script arguments in core pack integrations and scripts do not use the word 'incident'. This helps maintain the flexibility of the system and prevents potential issues caused by keyword replacement. | | -| SC106 | Checks that the script runas is not equal to DBotRole | For security reasons, the `runas` field should not be set to DBotRole. | | -| SC109 | Validate that there are no scripts with the same type and the same name. | Duplicate names cause confusion and unpredictable behaviors. | | -## `ST`: Structure -| Code | Description | Rationale | Autofixable | -|--------|------------------------------------------------|---------------------------------------------|---------------| -| ST110 | Validate that the scheme's structure is valid. | Maintain valid structure for content items. | | \ No newline at end of file diff --git a/docs/concepts/demisto-sdk.md b/docs/concepts/demisto-sdk.md deleted file mode 100644 index 209b01cfe..000000000 --- a/docs/concepts/demisto-sdk.md +++ /dev/null @@ -1,562 +0,0 @@ ---- -id: demisto-sdk -title: Demisto SDK ---- -The [Demisto SDK](https://github.com/demisto/demisto-sdk) is a Python library designed to aid the development process, both to validate entities being developed and to assist in the interaction between your development setup and Cortex XSOAR or Cortex XSIAM. -This guide will help you get acquainted with the Demisto SDK, including installation and set up and will provide some -basic information about key commands to aid you in the development process. For additional information, please see our full [Demisto SDK documentation](https://github.com/demisto/demisto-sdk#demisto-sdk). - -Note that in some cases fetching remote files is needed. In the case of a private repository and an un-configured `DEMISTO_SDK_GITHUB_TOKEN` environment variable, files will be fetched from the remote branch of the local repo. - -## Installation and Setup - -The Demisto SDK library supports Python 3.8 and above. Before installing, please make sure you have a compatible Python version installed and active. -The Demisto SDK is designed to work on macOS and Linux machines. Windows users are encouraged to run Demisto-SDK commands through a dev-container, WSL or a docker image. See [here](../tutorials/tut-setup-dev-remote) for more information. - -If you have followed our development setup [`bootstrap` process](dev-setup.md#bootstrap) you don't need to install the SDK manually as it is installed for you on your `venv`. - -These installation steps are only required if you are not working with the `bootstrap` or if you are working on a repository which is not part of the Content repository. -To manually install the Demisto-SDK, enter your terminal and run the command: - -```buildoutcfg -pip3 install demisto-sdk -``` - -After running the command the library should install - as a first step, check what version of the SDK you are using, by running: - -```buildoutcfg -demisto-sdk -v -``` - -This command should print the version you are using. You can check the latest released version [here](https://pypi.org/project/demisto-sdk/#history). - -If you already have the SDK installed and you wish to upgrade the version, run the following command: - -```buildoutcfg -pip3 install --upgrade demisto-sdk -``` - -If you would like to contribute to the Demisto SDK project, check out the [Contributing to XSOAR SDK guide](../contributing/sdk.md). - -### Environment Variable Setup - -Some SDK commands require you to have an interaction with the Cortex XSOAR or Cortex XSIAM server. Examples of such interactions -include uploading and downloading entities to or from XSOAR or XSIAM and running commands in the CLI. - -To use these functions, set up the required environment variables: - -#### For Cortex XSOAR 6.x Only - 1. Get your API key by going to the Cortex XSOAR server -> `Settings` -> `Integrations` -> `API Keys` -> `Get Your Key` -> Give your key a name and press `Generate Key`. - 2. Copy the given key. - 3. Add the following parameters to `~/.zshrc` and `~/.bash_profile`: - - ```buildoutcfg - export DEMISTO_BASE_URL=://: - export DEMISTO_API_KEY= - ``` - - For example: - - ```buildoutcfg - export DEMISTO_BASE_URL=http://127.0.0.1:8080 - export DEMISTO_API_KEY=XXXXXXXXXXXXXXXXXXXXXX - ``` - - > Alternatively, you can use a `.env` file in the root repository with the above variables. - -#### For Cortex XSIAM or Cortex XSOAR 8.x and above -1. The base URL should be retrieved from the XSIAM instance by navigating to `Settings` -> `Configurations` -> `API Keys` -> clicking on the `Copy URL` button located at the top right corner of the page, and not the browser URL. -2. The API key should be set to a Standard security level and must have the Instance Administrator role. -3. The `XSIAM_AUTH_ID` environment variable needs to be set. Get it from the `ID` column of the created API Key. -4. Add the following parameters to `~/.zshrc` and `~/.bash_profile`: - - ```buildoutcfg - export DEMISTO_BASE_URL= - export DEMISTO_API_KEY= - export XSIAM_AUTH_ID= - ``` - - For example: - - ```buildoutcfg - export DEMISTO_BASE_URL=https://api-xsiam-server.us.paloaltonetworks.com - export DEMISTO_API_KEY=XXXXXXXXXXXXXXXXXXXXXX - export XSIAM_AUTH_ID=1 - ``` - - > Alternatively, you can use a `.env` file in the root repository with the above variables. - -Please note that once the `XSIAM_AUTH_ID` environment variable is set, the SDK commands will be configured to work with a Cortex XSIAM / XSOAR 8.x instance. -In order to set Demisto SDK to work with a Cortex XSOAR 6.x instance, you need to delete the `XSIAM_AUTH_ID` parameter from your environment. To do this, please run the following command: -```buildoutcfg -unset XSIAM_AUTH_ID -``` - -### Using the SDK in Private Repositories - -Finally, if you are using a private GitHub repository, some SDK functions require an interaction with Git. To use the SDK you should setup your GitHub token. -To generate your token on GitHub use [the following guide](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) and then run: - -```buildoutcfg -export DEMISTO_SDK_GITHUB_TOKEN= -``` - -## General CLI Usage - -Run SDK commands from the CLI by following this basic structure: - -```buildoutcfg -demisto-sdk -``` - -If you are not sure of the available commands, run: - -```buildoutcfg -demisto-sdk -h -``` - -This command gives you a full list of available commands as well as a short description of each command's purpose. -If you need the list of arguments available for a command, run: - -```buildoutcfg -demisto-sdk -h -``` - -This gives you a full list of the command's arguments and their descriptions. We add new arguments and features regularly, so please feel free to approach us with new ideas and suggestions! - -## Run using Docker image - -You can run the Demisto-SDK using a docker image. For more details go to [Demisto-SDK Docker](./demisto-sdk-docker). - -## Commands - -This guide will give you a short tutorial about the basic SDK commands. While there are additional commands and use cases found in the SDK, these are the main commands you should know to begin: - -### init - -Use this command to create a new pack, integration or a script. - -#### Examples and Use Cases - -- Create a new pack: - - ```buildoutcfg - demisto-sdk init -n myNewPackName - ``` - - This creates a new pack under the `Packs` directory with the name `myNewPackName`. - After the pack initialization ends the command gives you the option to create a new integration in the pack. - -- Create a new integration in a specified directory: - - ```buildoutcfg - demisto-sdk init --integration -n myIntegration -o Packs/myNewPack/Integrations - ``` - - This creates a new integration with the name `myIntegration` under the directory `Packs/myNewPack/Integrations`. - -- Create a new *feed* integration: - - ```buildoutcfg - demisto-sdk init --integration -n myFeed -t FeedHelloWorld - ``` - - This creates a new integration named `myFeed` which will be based on the `FeedHelloWorld` integration. The `init` command has several preset templates that you can choose as the basis for your integration or script. - By default the template is the `BaseIntegration` and `BaseScript` but you can also use `HelloWorld`, `HelloIAMWorld`, or `FeedHelloWorld` for integrations and `HelloWorldScript` for scripts. - These templates have more robust integration information and examples. Additionally, some templates are used to create the basic structure for feed and IAM integrations. - -- Create a new script: - - ```buildoutcfg - demisto-sdk init --script -n myScript - ``` - - This command creates a new script named `myScript`. - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/init/README.md#init). - -### validate - -Use this command to make sure your `.yml`, `.json` and `.md` files are up to Cortex XSOAR or Cortex XSIAM standards. - -#### Examples and Use Cases - -- Validate all committed files: - - ```buildoutcfg - demisto-sdk validate - ``` - - This command identifies all the files that were **committed** via `git` in your current branch and runs validations on them. - Note that this is the variation of the command that runs in our build process, in case of a build failure run this to recreate it locally. - -- Validate all committed and staged files: - - ```buildoutcfg - demisto-sdk validate -g - ``` - - This command identifies all the files that were **committed and staged** via `git` in your current branch and runs validations on them. - -- Validate a specific file: - - ```buildoutcfg - demisto-sdk validate -i Packs/myPack/Integrations/myIntegration/myIntegration.yml - ``` - - This runs validation only on the file `Packs/myPack/Integrations/myIntegration/myIntegration.yml`. This variation does not detect the file status via `git` and thus will not check for backwards compatibility changes. - -- Validate all committed and staged files, print ignored files and errors and debug git: - - ```buildoutcfg - demisto-sdk validate -g --print-ignored-errors --print-ignored-files --debug-git - ``` - - This command validates all committed and staged files and prints out additional information which is hidden by default: - - Which files were ignored. - - Error codes that were ignored. - - A report on which files were identified and in what status via `git`. - This additional information might be helpful in debugging SDK commands or figuring out why a file did not validate. - -#### Validate Error Structure and Ignoring Errors - -Each one of our validation errors follows a basic structure: - -```buildoutcfg -File_Path: [Error_Code] - Error Message -``` - -For example: - -```buildoutcfg -Packs/CortexXDR/Integrations/CortexXDRIR/CortexXDRIR.yml: [BA100] - The version for our files should always be -1, please update the file. -``` - -At the end of the validation a summary appears with the file paths and error codes. - -The error codes serve two main functions: - -1. The first two letters are used to identify the type of error encountered. For example, `DO` is a Docker related error, `BC` is backwards compatibility related and `ST` is a yml/json structure error. - A full list of abbreviations can be found in the [full validation documentation](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/validate/README.md#validate). -2. The error code can be used to ignore specific error types. To ignore an error go to the file `.pack-ignore` file and input the following structure: - - ```buildoutcfg - [file:file_name] - ignore= - ``` - - For example: - - ```buildoutcfg - [file:CortexXDRIR.yml] - ignore=IN126,IN135 - ``` - - - Note: **Not all error codes can be ignored! It is preferable to fix errors rather than ignore them.** - Please consult with a Cortex XSOAR or XSIAM team member before ignoring an error. - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/validate/README.md#validate). - -### lint - -Use this command to make sure your `.py` and `.ps1` files are up to Cortex XSOAR or XSIAM standards. -This command runs several libraries to validate your code, including: - -- [Flake8](https://pypi.org/project/flake8/) - Makes sure your code is up to PEP8 standards. -- [MyPy](https://pypi.org/project/mypy/) - Validates type annotations, assignments and additional Python checks. -- [Vulture](https://pypi.org/project/vulture/) - Finds unused code. -- [Bandit](https://pypi.org/project/bandit/) - Finds security issues. -- XSOAR Linter - An internal linter used to identify specific restrictions and provide best practice advice for your code. -- Pylint and Pytest - Used to run your attached unit test (in the `_test.py`) file on the integration/script's Docker and will report your code coverage. -- PowerShell test and analyze - Code linters for PowerShell. - -At the end of the command a short report appears with all of the error and warnings found, as well as failed and passed unit tests. - -For additional information please see our documentation about [linting](../integrations/linting.md) and [unit testing](../integrations/unit-testing.md). - -#### Examples and Use Cases - -- Run lint on all committed and changed files: - - ```buildoutcfg - demisto-sdk lint -g - ``` - - This command identifies all the files that were **committed and staged** via `git` in your current branch and runs `lint` on them. - Note that this is the variation of the command that runs in our build process, in case of a build failure run this to recreate it locally. - -- Run lint without flake8 on a specific file: - - ```buildoutcfg - demisto-sdk lint -i Packs/myPack/Integrations/myIntegration/ --no-flake8 - ``` - - This runs the lint command on the integration found in `Packs/myPack/Integrations/myIntegration/` - please note that the path requested is **to the directory not to the file itself**. - Also, it should be noted that there are additional flags used to turn off any specific linter such as `--no-mypy` and `--no-xsoar-linter`, etc. - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/lint/README.md#lint). - -### secrets - -Use this command to find secrets such as emails and IP addresses in your files. -Cortex XSOAR is an open source product, its code can be found on a public repository on GitHub and thus is very visible. -You have a responsibility to identify and eliminate any secrets before they get to our repository or even to our pull requests. - -> Please note: **this command is not fool proof and a manual review of the files is still highly recommended**. - -#### Examples and Use Cases - -- Detect secrets in your files: - - ```buildoutcfg - demisto-sdk secrets - ``` - - This detects secrets in all your changed files. Please note that this command can have some false positives. You can make the command less sensitive by adding the `-ie` flag. - -#### Ignoring secrets - -At times there is information that might be flagged incorrectly by the command as a secret. Or it may be "secret" but we wish to share it publicly (for example the support email address). -To ignore a specific secret, enter it to the packs's `.secrets-ignore` file. -For example: - -```text -https://myurl.com -https://my2ndUrl.com -``` - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/secrets/README.md#secrets). - -### format - -Use this command to format an XSOAR or XSIAM entity in accordance with our standards. -In some cases, when a file is downloaded from the XSOAR server, it might contain additional fields that are -not required when entering the entity to the `content` repository. The `format` command will remove the unnecessary fields and make any fixes needed to the existing fields. - -#### Examples and Use Cases - -- Format a specific file: - - ```buildoutcfg - demisto-sdk format -i Packs/myPack/Integrations/myIntegration/myIntegration.yml - ``` - - This formats the file `Packs/myPack/Integrations/myIntegration/myIntegration.yml` in accordance with our standards. - -- Format a pack and update script and integration Docker images: - - ```buildoutcfg - demisto-sdk format -i Packs/myPack -ud - ``` - - This formats all files in `myPack`. It also updates any Docker images in integrations and scripts in the pack. - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/format/README.md#format). - -### upload - -Use this command to upload an entity to a Cortex XSOAR or Cortex XSIAM. Be sure to set up the required [environment variables](#environment-variable-setup) prior to running this command in order to establish a connection between `demisto-sdk` and the XSOAR or the XSIAM server. - - -#### Examples and Use Cases - -- Upload an integration to the server: - - ```buildoutcfg - demisto-sdk upload -i Packs/myPack/Integrations/myIntegration - ``` - - This uploads the integration found in `Packs/myPack/Integrations/myIntegration` to the preset server. - -- Upload a pack to the server without certificate validation: - - ```buildoutcfg - demisto-sdk upload -i Pack/myPack --insecure - ``` - - This iterates over all the content entities in the pack `myPack` and uploads them to the preset server - without checking the certification. Note that this command does not upload the entities as a whole pack but instead uploads them individually. - -- Zip and upload a whole pack to the Marketplace: - - ```buildoutcfg - demisto-sdk upload -i Pack/myPack -z - ``` - - This will zip, upload and install the pack `myPack` in the preset server's Marketplace. - - **When uploading to the Marketplace, the command will overwrite any pack which already exists in the server. - In order to avoid mistakes, a prompt will appear asking you to approve the upload.** - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/upload/README.md#upload). - -### zip-packs - -Use this command to generate zipped packs and upload them to the Marketplace in Cortex XSOAR or Cortex XSIAM. Be sure to set up the required [environment variables](#environment-variable-setup) prior to running this command in order to establish a connection between `demisto-sdk` and the XSOAR or the XSIAM server. - -#### Examples and Use Cases - -- Zip a pack: - - ```buildoutcfg - demisto-sdk zip-packs -i Campaign -o Packs/Campaign - ``` - - This will zip the `Campaign` pack into Campaign.zip file in the `Packs/Campaign` directory. - -- Zip and upload a whole pack to the Marketplace: - - ```buildoutcfg - demisto-sdk zip-packs -i Campaign -o Packs/Campaign -u - ``` - - This will zip, upload and install the pack `Campaign` in the preset server's Marketplace. - - **When uploading to the Marketplace, the command will overwrite any pack which already exists in the server. - In order to avoid mistakes, a prompt will appear asking you to approve the upload.** - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/zip_packs/README.md#zip-packs). - -### download - -Use this command to download entities from a Cortex XSOAR or XSIAM server to your local repository. Be sure to set up the required [environment variables](#environment-variable-setup) prior to running this command in order to establish a connection between `demisto-sdk` and the server. - -This command can be useful when developing within the Cortex XSOAR or Cortex XSIAM server itself and downloading the new entities to your -local environment in order to continue with the contribution process. - -#### Notes and Limitations - -- `JavaScript` integrations and scripts are not downloadable using this command. -- If there are files that exist both in the output directory and are specified in the input, they will be ignored. To override this behavior so that existing files will be merged with their newer version, use the `--force`/`-f` flag. -- For consistency, it is assumed that for each integration or script the folder containing it will have the same name as the integration/script name with no separators. For example the integration `Test Integration_Full-Name`, will be under `~/.../Packs/TestPack/Integrations/TestIntegrationFullName/`. - -#### Examples and Use Cases - -- See which files are downloadable: - - ```buildoutcfg - demisto-sdk download -lf - ``` - - This lists all the files which are downloadable using this command from the preset server. - Note: Do not run the `-lf` flag with `-i` or `-o`. - -- Download a file to a given pack: - - ```buildoutcfg - demisto-sdk download -i "My Integration" -o Packs/myPack - ``` - - This downloads the entity named `My Integration` and places it in the appropriate directory within `myPack`. - Note that if `My Integration` exists in the pack, it will not be downloaded. - -- Download several files to a given pack and overwrite any file which already exists: - - ```buildoutcfg - demisto-sdk download -i "My Integration" -i myScript -o Packs/myPack -f - ``` - - This downloads both the `My Integration` and `myScript` entities to `myPack`. If any of the files already exist in the pack they are overwritten. - -- Download all available custom files to a given pack: - - ```buildoutcfg - demisto-sdk download -a -o Packs/myPack - ``` - - This downloads all accessible files (listed with the `-lf` flag) to the appropriate directories in `myPack`. - -For additional information see [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/download/README.md#download). - -### update-release-notes - -Use this command to automatically generate release notes for a given pack and update the `pack_metadata.json` version. - -Running this command creates a new release notes file under the `ReleaseNotes` directory in the given pack in the form of `X_Y_Z.md` where X_Y_Z is the new pack version. -After running this command, add the newly created release notes file to `GitHub` and add your notes under their respective headlines. For example: - -```buildoutcfg - -#### Integrations -##### Cortex XDR - IOC -- Fixed an issue where searching more than 10K indicators failed when using ElasticSearch. -- Updated the Docker image to: *demisto/python3:3.9.4.18682*. - -#### Incident Fields -- **XDR Similar Incidents** - New incident field. - -#### Playbooks -##### Cortex XDR incident handling v3 -- Added a new machine learning script to search for similar incidents by shared incident fields and indicators. - -#### Layouts -##### Cortex XDR Incident -- Updated layout with a new section for similar incidents. - -``` - -Further information about how to run this command can be found [here](../documentation/release-notes.md). - -### generate-docs - -Use this command to generate a readme file for your integration, script or playbook. - -Running this command creates a new `README.md` file in the same directory as the entity on which it ran, unless otherwise specified using the `-o` flag. -To generate command examples, set up the required [environment variables](#environment-variable-setup) prior to running this command in order to establish a connection between `demisto-sdk` and the server, as well as create a file containing some command examples to be run for the documentation. - -Further information about how to run this command can be found [here](../documentation/readme_file.md#creating-documentation). - -### generate-yml-from-python -Use this command to generate a YML file from Python code that includes its special syntax. - -The output file name will be the same as the Python code with the `.yml` extension instead of `.py`. - -The generation currently supports integrations only. -The feature is supported from content Base pack version 1.20.0 and on. - -Example run: -```buildoutcfg -demisto-sdk -i my_integration.py -v -``` - -Further information about the syntax and how to run this command can be found [here](../integrations/yml-from-python-code-gen.md). - - -## Setting a Preset Custom Command Configuration - -You can create your own configuration for the `demisto-sdk` commands by creating a file named `.demisto-sdk-conf` -within the directory from which you run the commands. This file will enable you to set a default value to the existing command flags that will take effect whenever the command is run. This can be done by entering the following structure in the file: - -```buildoutcfg -[command_name] -flag_name=flag_default_value -``` - -Note: Be sure to use the flag's full name and use `_` instead of a `-` if it exists in the flag name -(e.g. instead of `no-docker-checks` use `no_docker_checks`). - -Here are a few examples: - -- As a user, I would like to not use the `MyPy` linter in my environment when using the `lint` command. - In the `.demisto-sdk-conf` file I'll enter: - - ```buildoutcfg - [lint] - no_mypy=true - ``` - -- As a user, I would like to include untracked git files in my validation when running the `validate` command. - In the `.demisto-sdk-conf` file I'll enter: - - ```buildoutcfg - [validate] - include_untracked=true - ``` - -- As a user, I would like to automatically use minor version changes when running the `update-release-notes` command. - In the `.demisto-sdk-conf` file I'll enter: - - ```buildoutcfg - [update-release-notes] - update_type=minor - ``` diff --git a/docs/concepts/dev-setup.md b/docs/concepts/dev-setup.md index 4837e1799..8e0fce333 100644 --- a/docs/concepts/dev-setup.md +++ b/docs/concepts/dev-setup.md @@ -72,16 +72,7 @@ Installation: This is our help tool that will ease the contribution process. It will help you to generate a [Pack](../packs/packs-format), maintain your files, and validate them before committing to the branch. -To install **demisto-sdk** using `pipx` -```bash -pipx install demisto-sdk --force -``` - -To check the you have the latest version of the SDK, run: - -```bash -demisto-sdk --version -``` +For more information about how to install *demisto-sdk* please visit: [Demisto-SDK In installation](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Install-Demisto-SDK) #### Bootstrap diff --git a/docs/concepts/faq.md b/docs/concepts/faq.md index 18edef9e7..b5b9ea2a9 100644 --- a/docs/concepts/faq.md +++ b/docs/concepts/faq.md @@ -13,7 +13,7 @@ We developed a free [VSCode extension](vscode-extension) that simplifies/automat - Creating a blank integration or automation script - Uploading/Downloading your integration code to/from Cortex XSOAR -However, if you want to a different IDE (Sublime, vi, emacs, etc.) it's totally fine! It just means that some of those tasks must be performed manually. To automate them, you can use the [demisto-sdk](https://github.com/demisto/demisto-sdk). +However, if you want to a different IDE (Sublime, vi, emacs, etc.) it's totally fine! It just means that some of those tasks must be performed manually. To automate them, you can use the [demisto-sdk](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Introduction-to-Demisto-SDK). You can also write code directly in the [Cortex XSOAR UI](../concepts/xsoar-ide) but is not recommended if you want to contribute supported content. Check [here](getting-started-guide#creating-new-content) for details. diff --git a/docs/concepts/getting-started-guide.md b/docs/concepts/getting-started-guide.md index 4b16984c3..a59308a6e 100644 --- a/docs/concepts/getting-started-guide.md +++ b/docs/concepts/getting-started-guide.md @@ -33,7 +33,7 @@ To become a Technology Partner, read the "[Become a Technology Partner](../partn Whether you are using the built-in Cortex XSOAR IDE, or a full development environment, we have an official SDK that will help you with your development process. The SDK is a command-line tool that can be used to upload, download, lint, validate and run code on Cortex XSOAR (or XSIAM) directly from your command line. -For more information, refer to the [demisto-sdk](../concepts/demisto-sdk) article. +For more information, refer to the [demisto-sdk](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Introduction-to-Demisto-SDK). ### Basic XSOAR Environment diff --git a/docs/concepts/vscode-extension.md b/docs/concepts/vscode-extension.md index de3c777ef..092de53dc 100644 --- a/docs/concepts/vscode-extension.md +++ b/docs/concepts/vscode-extension.md @@ -11,7 +11,7 @@ The extension provides an easy virtual environment setup for Cortex XSOAR integr ## Extension Workflow The extension defines a slightly different workflow than Cortex XSOAR. With the plugin, you can work on your code (whether it is Python, PowerShell or JavaScript). -It utilizes the excellent [demisto-sdk](./demisto-sdk) python package. +It utilizes the excellent [demisto-sdk](https://github.com/demisto/demisto-sdk) python package. ## Prerequisites @@ -38,7 +38,7 @@ The default is _false_, and for now, it is recommended not to enable this config All of the commands in the extension start with the easy-to-find pattern _XSOAR_. Notable commands: -- **_XSOAR: Demisto-SDK Lint/Validate/Update Release Notes..._**: Will run the [demisto-sdk](https://github.com/demisto/demisto-sdk/) commands. +- **_XSOAR: Demisto-SDK Pre-Commit/Validate/Update Release Notes..._**: Will run the [demisto-sdk](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Demisto-SDK-commands) commands. - **_XSOAR: Configure XSOAR unit tests_**: Will configure the integration unit tests. diff --git a/docs/contributing/checklist.md b/docs/contributing/checklist.md index 367e981fe..0e9858c82 100644 --- a/docs/contributing/checklist.md +++ b/docs/contributing/checklist.md @@ -27,13 +27,13 @@ Use PascalCase (e.g., "PackName") for the names of the pack, its directories, an As a reference, you can check the tree of the [Hello World](https://github.com/demisto/content/tree/master/Packs/HelloWorld) pack on GitHub. -To create a new content pack directory tree and structure, you should use the [`demisto-sdk init`](https://github.com/demisto/demisto-sdk/tree/master/demisto_sdk/commands/init#init) , as described in the [tutorial](../tutorials/tut-setup-dev#step-6-create-a-branch-and-integration-directory). +To create a new content pack directory tree and structure, you should use the [`demisto-sdk init`](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/init) , as described in the [tutorial](../tutorials/tut-setup-dev#step-6-create-a-branch-and-integration-directory). ### Where do I get all these files? Integrations and Scripts should be written with your favorite IDE (see [here](../concepts/dev-setup) and [here](../tutorials/tut-setup-dev) for details on how to set up your dev environment) -All other entity types (Playbooks, Test Playbooks, Incidents/Indicators Fields and Types, Layouts, Classifiers and Mappers, Widgets, Dashboards) should be created in the Cortex XSOAR UI and exported using [`demisto-sdk download`](https://github.com/demisto/demisto-sdk/tree/master/demisto_sdk/commands/download#download) (using the `-fmt` argument). You can also export the files manually via the Cortex XSOAR UI (either individually using the download icons, or using the `Export Custom Content` feature under `Settings` - `About` - `Troubleshooting`) but, in that case, you'll need to format them using [`demisto-sdk format`](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/format/README.md). +All other entity types (Playbooks, Test Playbooks, Incidents/Indicators Fields and Types, Layouts, Classifiers and Mappers, Widgets, Dashboards) should be created in the Cortex XSOAR UI and exported using [`demisto-sdk download`](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/download) (using the `-fmt` argument). You can also export the files manually via the Cortex XSOAR UI (either individually using the download icons, or using the `Export Custom Content` feature under `Settings` - `About` - `Troubleshooting`) but, in that case, you'll need to format them using [`demisto-sdk format`](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/format). ### Integrations diff --git a/docs/contributing/contributing.md b/docs/contributing/contributing.md index 1d07f61dc..49498c7a9 100644 --- a/docs/contributing/contributing.md +++ b/docs/contributing/contributing.md @@ -90,10 +90,10 @@ Carefully review the following guidelines, as they will greatly enhance the like 3. Set up a development environment by following the [short development setup](../concepts/dev-setup) guide and the more detailed [tutorial](../tutorials/tut-setup-dev), or [use a pre-configured GitHub Codespace environment](../tutorials/tut-setup-dev-codespace.md). 4. Follow the [content pack structure](../packs/packs-format) to build your contribution. - The "[demisto-sdk init](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/init/README.md)" command will help you create the basic structure. + The "[demisto-sdk init](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/init)" command will help you create the basic structure. 5. If you are updating an **existing** content pack, **make you have the latest available version installed before proceeding**. 6. If you are creating integrations and / or scripts (aka *automations*), make sure that you: - * Use the proper [directory structure](../integrations/package-dir). [demisto-sdk init](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/init/README.md) will help you create it. + * Use the proper [directory structure](../integrations/package-dir). [demisto-sdk init](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/init) will help you create it. If working on existing code that doesn't follow this structure, we require converting to this structure (as long as the changes aren't minor), as it allows running linting and unit-tests, and provides a smoother review process. * Understand the structure of the [YAML file](../integrations/yaml-file), and the [parameter types](../integrations/parameter-types) within it. * Make sure your integration follows our [logo guidelines](../integrations/integration-logo). @@ -122,7 +122,7 @@ If you are an XSOAR Technology Partner, make sure the [Cortex XSOAR Alliances Te ### Prerequisites Before submitting a Pull Request to the [Cortex XSOAR GitHub Repository](https://github.com/demisto/content), the following requirements must be met: 1. Create a GitHub **fork** of the XSOAR Content repository, and submit your changes **on a new branch** (do **not** push your changes to the *master* branch). -2. Validate your content using [demisto-sdk validate](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/validate/README.md): +2. Validate your content using [demisto-sdk validate](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/validate): `demisto-sdk validate -i Packs/YourPackName` 3. If your contribution has integrations or scripts, run linting (which also runs [unit-tests](../tutorials/tut-setup-dev#step-5-run-the-linter-and-unit-tests)) by running: `demisto-sdk lint -i Packs/YourPakName` diff --git a/docs/documentation/readme_file.md b/docs/documentation/readme_file.md index 472f2417e..0ebf79145 100644 --- a/docs/documentation/readme_file.md +++ b/docs/documentation/readme_file.md @@ -22,7 +22,7 @@ The entity README.md file should not be confused with the Content Pack README.md ## Creating Documentation -Use the `demisto-sdk generate-docs` command to generate documentation for your content entity. Documentation for the command is available [here](https://github.com/demisto/demisto-sdk#generate-docs). +Use the `demisto-sdk generate-docs` command to generate documentation for your content entity. Documentation for the command is available [here](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/generate-docs). ### Command Examples - Integration Documentation @@ -51,10 +51,10 @@ Output file was saved to : ``` :::note -If you are connecting to a Server with a self signed certificate, make sure to pass the `--insecure` option to the `generate-docs` command. +If you are connecting to a Server with a self-signed certificate, make sure to pass the `--insecure` option to the `generate-docs` command. ::: -For additional information you can view the full documentation for the command [here](https://github.com/demisto/demisto-sdk/tree/master/demisto_sdk/commands/generate_docs#generate-docs). +For additional information you can view the full documentation for the command [here](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/generate-docs). #### New Versions of Existing Integrations diff --git a/docs/documentation/release-notes.md b/docs/documentation/release-notes.md index d8a95354f..35f0231be 100644 --- a/docs/documentation/release-notes.md +++ b/docs/documentation/release-notes.md @@ -30,7 +30,7 @@ demisto-sdk update-release-notes -i [Changed pack path] ``` For more information regarding the ***update-release-notes*** command in the **demisto-sdk**, please refer to the -[command documentation](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/update_release_notes/README.md). +[command documentation](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/update-release-notes). ## Naming The release notes file is generated for you and is found in the *ReleaseNotes* folder within each pack. If this folder does not already exist, it is created for you. @@ -212,7 +212,7 @@ However, you may encounter a scenario where certain changes are not necessary to demisto-sdk doc-review -i Packs/Base/ReleaseNotes/1_11_10.md ``` -More info is available at the ***demisto-sdk doc-review*** command [README](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/doc_reviewer/README.md). +More info is available at the ***demisto-sdk doc-review*** command [README](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/doc-review). ## Breaking Changes Version In some cases, a new version is introduced which breaks backward compatibility. diff --git a/docs/integrations/code-conventions.md b/docs/integrations/code-conventions.md index 31fa9d781..7f2583dec 100644 --- a/docs/integrations/code-conventions.md +++ b/docs/integrations/code-conventions.md @@ -283,7 +283,7 @@ def main(): ## IOC Reputation Commands -There are two implementation requirements for reputation commands (aka `!file`, `!email`, `!domain`, `!url`, and `!ip`) that are enforced by checks in the [demisto-sdk](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/validate/README.md). +There are two implementation requirements for reputation commands (aka `!file`, `!email`, `!domain`, `!url`, and `!ip`) that are enforced by checks in the [demisto-sdk](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/validate). - The reputation command's argument of the same name must have `default` set to `True`. - The reputation command's argument of the same name must have `isArray` set to `True`. diff --git a/docs/integrations/openapi-codegen.md b/docs/integrations/openapi-codegen.md deleted file mode 100644 index ab0ffe654..000000000 --- a/docs/integrations/openapi-codegen.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -id: openapi-codegen -title: OpenAPI (Swagger) Code-Gen ---- - -It is possible to generate a Cortex XSOAR integration package (YAML and Python files) with a dedicated tool in the Cortex XSOAR (demisto) SDK. -The integration will be usable right away after generation. - -### Requirements -* OpenAPI (Swagger) specification file (v2.0 is recommended) in JSON format. -* Cortex XSOAR (demisto) SDK - -### Steps -1. Install the Cortex XSOAR SDK: `pip3 install demisto-sdk`. -2. Run the `demisto-sdk openapi-codegen` command. Using the arguments and examples below. -3. Follow the instructions to generate the Cortex XSOAR integration files. -4. Use `demisto-sdk unify` to create a unified integration YAML file from the integration files. -5. Use `demisto-sdk upload` to upload the generated integration to Cortex XSOAR. -6. Set up an instance and run the integration commands. - -### Command instructions - -#### Arguments - -* **'-h', '--help'** - - Show command help. - -* **'-i', '--input_file'** - - The swagger file to load in JSON format. - -* **'-cf', '--config_file'** - - The integration configuration file. It is created in the first run of the command. - -* **'-n', '--base_name'** - - The base filename to use for the generated files. - -* **'-o', '--output_dir'** - - Directory to store the output in (default is current working directory). - -* **'-pr', '--command_prefix'** - - Add a prefix to each command in the code. - -* **'-c', '--context_path'** - - Context output path. - -* **'-u', '--unique_keys'** - - Comma separated unique keys to use in context paths (case sensitive). - -* **'-r', '--root_objects'** - - Comma separated JSON root objects to use in command outputs (case sensitive). - -* **'-v', '--verbose'** - - Be verbose with the log output. - -* **'-f', '--fix_code'** - - Fix the python code using autopep8. - -* **'-a', '--use_default'** - - Use the automatically generated integration configuration (Skip the second run). - -#### Examples -The Examples below are for the [Pet Store Swagger specification](https://petstore.swagger.io/). - -``` -demisto-sdk openapi-codegen -i pet_swagger.json -n PetStore -o PetStoreIntegration -u "id" -r "Pet" -``` - -This will create an integration configuration for the PetStore swagger file in the `PetStoreIntegration` directory. -It will use `id` to identify unique properties in outputs and `Pet` to identify root objects in the outputs. -That configuration can be modified and will be used in a second run of the command. - -``` -demisto-sdk openapi-codegen -i pet_swagger.json -n PetStore -o PetStoreIntegration -u "id" -r "Pet" -cf "PetStoreIntegration/PetStore.json" -``` - -This will create the Cortex XSOAR integration for the PetStore swagger file using the configuration file located in PetStoreIntegration/PetStore.json. - - ``` -demisto-sdk openapi-codegen -i pet_swagger.json -n PetStore -o PetStoreIntegration -u "id" -r "Pet" -a -``` - -This will create the Cortex XSOAR integration for the PetStore swagger file using the generated configuration file, thus skipping the second run of the command. - - -### Video Tutorial - diff --git a/docs/integrations/package-dir.md b/docs/integrations/package-dir.md index 928dd22da..726482162 100644 --- a/docs/integrations/package-dir.md +++ b/docs/integrations/package-dir.md @@ -50,11 +50,11 @@ For example, the integration [Cortex XDR](https://github.com/demisto/content/tre ## Split a YML file to Directory Structure If you need to split a Unified YAML file (exported via the Cortex XSOAR UI) into the Directory Structure you can use the following options: -- `demisto-sdk split`: This command will also format the code (using autopep8) and setup the proper Pipenv files. See full command documentation [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/split/README.md). +- `demisto-sdk split`: This command will also format the code (using autopep8) and setup the proper Pipenv files. See full command documentation [here](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/split). - [Cortex XSOAR IntelliJ Plugin](https://plugins.jetbrains.com/plugin/12093-demisto-add-on-for-pycharm) ## Generate a YML file from Directory Structure If you need to manually create the Unified YAML file (for example to manually import it in Cortex XSOAR via the UI), you can either use: -- `demisto-sdk unify`: See full command documentation [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/unify/README.md). +- `demisto-sdk prepare-content`: See full command documentation [here](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/prepare-content). - [Cortex XSOAR IntelliJ Plugin](https://plugins.jetbrains.com/plugin/12093-demisto-add-on-for-pycharm) diff --git a/docs/integrations/postman-codegen.md b/docs/integrations/postman-codegen.md deleted file mode 100644 index 444173ff4..000000000 --- a/docs/integrations/postman-codegen.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -id: postman-codegen -title: Postman Code Generator -description: Generate XSOAR integration from Postman Collection ---- -Use the `demisto sdk postman-codegen` command to generate an XSOAR integration (yml file) from a Postman Collection v2.1. Note the generated integration is in the yml format. Use the `demisto-sdk split` [command](package-dir#split-a-yml-file-to-directory-structure) to split the integration into the recommended [Directory Structure](package-dir) for further development. - -You can generate the integration either as a two step process or a single step. -- **Single Step:** Use this method to generate directly an integration yml file. -- **Two Steps:** Use this method for more configuration and customization of the generated integration and code. - 1. Generate an integration config file. - 1. Update the config file as needed. Then generate the integration from the config file using the `demisto-sdk generate-integration` [command](./code-generator). - -## Tutorial Video: - - -## Options -* **-h, --help** -* **-i, --input** - Postman collection 2.1 JSON file -* **-o, --output** - (Optional) The output directory. Default is the current directory. -* **-n, --name** - (Optional) Sets the integration name. -* **-op, --output-prefix** - (Optional) Sets the global integration output prefix. Default is the integration name without spaces and special characters. -* **-cp, --command-prefix** - (Optional) The prefix for every command in the integration. Default is the integration name in lower case. -* **--config-out** - (Optional) If passed, generates config json file for further integration customisation. - -## How the command converts Postman collection -- Collection name converts to integration name. -- Collection name converts to command prefix (if command prefix is not passed). - - Example: **Virus Total** -> **virus-total** -- Collection name converts to prefix of each command output. - - Example: **Virus&& Total** -> **VirusTotal.Scan.scan_id** -- Collection request name converts to integration command. - - Example: **Get Hosts** -> **get-hosts** -- Authentication - - Base authentication type converts to username/password parameter - - API Key authentication type converts to apikey encrypted parameter - - Bearer token authentication type converts to apikey encrypted parameter -- Request name converts to command name. Example: **Get Events** -> **get-events** -- Request url variables converts to command arguments and passed as part of the request url. Example: *https://virustotal.com/vtapi/v2/ip/{{ip}}* -> created **ip** argument -> *https://virustotal.com/vtapi/v2/ip/8.8.8.8* -- Request query parameters converts to command arguments. Example: *https://virustotal.com/vtapi/v2/ip?resource=8.8.8.8* -> created **resource** argument -> *https://virustotal.com/vtapi/v2/ip?resource=8.8.8.8* -- Path URL variable converts to command argument and passed as a part of the request URL. Example: *https://virustotal.com/vtapi/v2/ip/:ip* -> creates **ip** argument -> *https://virustotal.com/vtapi/v2/ip/8.8.8.8* if `:ip` path variable equals 8.8.8.8. -- Request body - each leaf value converts to command argument and **body_format** which will allow further body customisation. Example: `{"key1":"val1","key2":{"key3":"val3"}}` -> created **key1** and **key3** arguments and **body_format** with the following value `{"key1":"{{key1}}","key2":{"key3":"{{key3}}"}}` -- Response JSON output converts to command outputs. - -## Postman Collection Requirements -### Mandatory Requirements -- Collection v2.1 is supported -- Each request should be saved and contain at least one successful response (which also saved) -- If url contains variables like *https://virustotal.com/vtapi/v2/ip/8.8.8.8*, then make sure to set it as variable like *https://virustotal.com/vtapi/v2/ip/{{ip}}* -- Define the authentication method under Collection edit page -> Authorization section - - Under collection settings, Authorization section should be set (recommended way) - - Requests must contain Authorization header - - -### Optional Requirements -- Collection description -- Short request names **Get Endpoints** will convert to **get-endpoints** -- Set description to request - -## How to run -- `demisto-sdk postman-codegen -i VirusTotal.collection.json --name 'Virus Total' --command-prefix vt` - The above command do the following: - - Sets the name of the integration to `Virus Total`. - - Sets the commands prefix to `vt` (`vt-get-url`, `vt-scan-url`). - - Generates `integration-VirusTotal.yml` file in the current directory. - -- `demisto-sdk postman-codegen -i VirusTotal.collection.json --name 'Virus Total' -o /output/path --config-out` - The above command do the following: - - Generates `config-VirusTotal.json` file under `/output/path` directory. - - Sets the name of the integration `Virus Total`. - - -## Example files: -* [URLScan Postman Collection v2.1](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/postman_codegen/resources/urlscan.io.postman_collection.json) -* [URLScan generated config file](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/postman_codegen/resources/config-urlscanio.json) -* [URLScan generated integration yml](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/postman_codegen/resources/integration-urlscanio.yml) diff --git a/docs/integrations/powershell-code.md b/docs/integrations/powershell-code.md index 81dc512b1..29bd56494 100644 --- a/docs/integrations/powershell-code.md +++ b/docs/integrations/powershell-code.md @@ -10,7 +10,7 @@ Starting with Server 5.5, PowerShell is supported for developing Automations and Similar to Python, PowerShell Integrations and Automations run in a Docker container. All of the Demisto Docker images that support PowerShell are named with a prefix of either `demisto/powershell` or `demisto/pwsh`. If you need to create a new image follow the instructions at the `demisto/dockerfiles` project: https://github.com/demisto/dockerfiles. ## Directory Structure -Similar to Python, PowerShell Integrations/Automations should follow the [Directory Structure](package-dir). With a slight difference that unit test files should be named: `.Tests.ps1` (Pester Unit Testing naming convention). You can use `demisto-sdk split` to convert an exported PowerShell Integration/Automation to the Directory Structure (more info [here](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/split/README.md)). +Similar to Python, PowerShell Integrations/Automations should follow the [Directory Structure](package-dir). With a slight difference that unit test files should be named: `.Tests.ps1` (Pester Unit Testing naming convention). You can use `demisto-sdk split` to convert an exported PowerShell Integration/Automation to the Directory Structure (more info [here](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/split)). ## Linting We use [PSScriptAnalyzer](https://github.com/PowerShell/PSScriptAnalyzer) for linting and static code analysis of PowerShell Integrations/Automations. If you receive a false positive from the Analyzer, you can suppress the rule by decorating the function/script with `SuppressMessageAttribute`. Make sure to specify a `Justification` in the attribute why the suppression was necessary. An example usage of suppression can be seen [here](https://github.com/demisto/content/blob/master/Packs/Base/Scripts/CommonServerPowerShell/CommonServerPowerShell.ps1#L3). PSScriptAnalyzer suppression documentation is available [here](https://github.com/PowerShell/PSScriptAnalyzer#suppressing-rules). diff --git a/docs/integrations/test-playbooks.md b/docs/integrations/test-playbooks.md index 74633ff47..f47e06bde 100644 --- a/docs/integrations/test-playbooks.md +++ b/docs/integrations/test-playbooks.md @@ -18,7 +18,7 @@ We use a standard naming convention for our playbook tests which follows the for ## Auto-Generate a Test Playbook -To auto generate a Test playbook based upon an integration or script use the following `demisto-sdk generate-test-playbook` command which can be found [here](https://github.com/demisto/demisto-sdk#generate-test-playbook). You can then import the playbook and modify it to your needs. +To auto generate a Test playbook based upon an integration or script use the following `demisto-sdk generate-test-playbook` command which can be found [here](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/generate-test-playbook). You can then import the playbook and modify it to your needs. ## Manually Create a Test Playbook diff --git a/docs/packs/packs-format.md b/docs/packs/packs-format.md index 53c9f2f3a..a79ed899b 100644 --- a/docs/packs/packs-format.md +++ b/docs/packs/packs-format.md @@ -150,7 +150,7 @@ The file contains a general explanation for the pack and you are free to add any ### `.secrets-ignore` -This file will be used while running the `demisto-sdk secrets`([explanation](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/secrets/README.md)), we will determine the file and will +This file will be used while running the `demisto-sdk secrets`([explanation](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/secrets)), we will determine the file and will use it as an allow list of approved words for your PR. **Note**: We use `demisto-sdk secrets` as part of our pre-commit hook to check that possible secrets in the PR aren't exposed to a public repository. diff --git a/docs/partners/adopt.md b/docs/partners/adopt.md index 76aca29a9..c382b838c 100644 --- a/docs/partners/adopt.md +++ b/docs/partners/adopt.md @@ -155,7 +155,7 @@ Please visit ====> https://github.com/me/content/pull/new/partner-HelloWorld-ado
You can also perform the necessary steps to adopt using any text editor or an IDE of your choice and the command line. -**Requirements:** To follow along, you'll need to have [Demisto SDK installed on your machine](https://github.com/demisto/demisto-sdk#installation). +**Requirements:** To follow along, you'll need to have [Demisto SDK installed on your machine](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/Install-Demisto-SDK). Follow the steps below to adopt using the a text editor or IDE. For additional guidance, you can watch how to [perform the steps using Visual Studio Code](https://www.youtube.com/watch?v=9GPkhtRw4Oc). diff --git a/docs/tutorials/tut-setup-dev.md b/docs/tutorials/tut-setup-dev.md index 2bc391ece..1208cf292 100644 --- a/docs/tutorials/tut-setup-dev.md +++ b/docs/tutorials/tut-setup-dev.md @@ -348,7 +348,7 @@ Switched to a new branch 'my_integration_name' Create a directory under `Packs/` named after your product where you will put all your content files later, and add it to the staged changes in `git`. Make sure you use **PascalCase** in the directory name (i.e. `MyIntegration`). For a detailed description regarding what exactly a pack is please click [here](../packs/packs-format). -You can create a Pack and an Integration directory using the [`demisto-sdk init` command](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/init/README.md). +You can create a Pack and an Integration directory using the [`demisto-sdk init` command](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/init). An example of creating a pack called `MyNewPack`, with an integration called `MyIntegration`, and with the metadata file created automatically: ```bash ➜ content-docs2 git:(add-pack-and-sdk-docs) ✗ demisto-sdk init --pack @@ -411,11 +411,11 @@ The last step is to `commit` your changes and `push` them to the *origin* in ord But you can also run the hooks locally using the demisto-sdk, in order to do that you can run the commands: 1. `demisto-sdk format` - this will auto correct couple of things in order for our validation to pass. -You can see the [docs](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/format/README.md) +You can see the [docs](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/format) 2. `demisto-sdk validate -g` - this will validate the integrity of the yml files, and will make sure they follow -our pre-set of roles. You can see the [docs](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/validate/README.md) -3. `demisto-sdk lint -i ` - this will run lint and pytest on your -changed python files. You can see the [docs](https://github.com/demisto/demisto-sdk/blob/master/demisto_sdk/commands/lint/README.md) +our pre-set of roles. You can see the [docs](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/validate) +3. `demisto-sdk pre-commit -i ` - this will run variety of checks and linters on your +changed python files. You can see the [docs](https://docs-cortex.paloaltonetworks.com/r/1/Demisto-SDK-Guide/pre-commit) diff --git a/sidebars.js b/sidebars.js index 6c86c670a..e80e23bfc 100644 --- a/sidebars.js +++ b/sidebars.js @@ -39,7 +39,6 @@ const sidebars = { "concepts/pycharm-plugin", "concepts/vscode-extension", "concepts/dev-setup", - "concepts/demisto-sdk" ] }, { @@ -156,8 +155,6 @@ const sidebars = { "integrations/dt", "integrations/integration-cache", "integrations/mirroring_integration", - "integrations/openapi-codegen", - "integrations/postman-codegen", "integrations/code-generator", "integrations/scheduled-commands", "integrations/yml-from-python-code-gen",