Merge branch 'RSS-Bridge:master' into master

.dockerignore

@@ -1,4 +1,6 @@
 .git
+!.git/HEAD
+!.git/refs/heads/*
 .gitattributes
 .github/*
 .travis.yml

.gitattributes

@@ -1,5 +1,6 @@
 # Auto detect text files and perform LF normalization
 * text=auto
+*.sh text eol=lf
 
 # Custom for Visual Studio
 *.cs     diff=csharp

.github/CONTRIBUTING.md

@@ -1,49 +1,7 @@
 ### Pull request policy
 
-* [Fix one issue per pull request](https://github.com/RSS-Bridge/rss-bridge/wiki/Pull-request-policy#fix-one-issue-per-pull-request)
-* [Respect the coding style policy](https://github.com/RSS-Bridge/rss-bridge/wiki/Pull-request-policy#respect-the-coding-style-policy)
-* [Properly name your commits](https://github.com/RSS-Bridge/rss-bridge/wiki/Pull-request-policy#properly-name-your-commits)
-  * When fixing a bridge (located in the `bridges` directory), write `[BridgeName] Feature` <br>(i.e. `[YoutubeBridge] Fix typo in video titles`).
-  * When fixing other files, use `[FileName] Feature` <br>(i.e. `[index.php] Add multilingual support`).
-  * When fixing a general problem that applies to multiple files, write `category: feature` <br>(i.e. `bridges: Fix various typos`).
-
-Note that all pull-requests must pass all tests before they can be merged.
+See the [Pull request policy page on the documentation](https://rss-bridge.github.io/rss-bridge/For_Developers/Pull_Request_policy.html) for more information on the pull request policy.
 
 ### Coding style
 
-* [Whitespace](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitespace)
-  * [Add a new line at the end of a file](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitespace#add-a-new-line-at-the-end-of-a-file)
-  * [Do not add a whitespace before a semicolon](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitespace#add-a-new-line-at-the-end-of-a-file)
-  * [Do not add whitespace at start or end of a file or end of a line](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitespace#do-not-add-whitespace-at-start-or-end-of-a-file-or-end-of-a-line)
-* [Indentation](https://github.com/RSS-Bridge/rss-bridge/wiki/Indentation)
-  * [Use tabs for indentation](https://github.com/RSS-Bridge/rss-bridge/wiki/Indentation#use-tabs-for-indentation)
-* [Maximum line length](https://github.com/RSS-Bridge/rss-bridge/wiki/Maximum-line-length)
-  * [The maximum line length should not exceed 80 characters](https://github.com/RSS-Bridge/rss-bridge/wiki/Maximum-line-length#the-maximum-line-length-should-not-exceed-80-characters)
-* [Strings](https://github.com/RSS-Bridge/rss-bridge/wiki/Strings)
-  * [Whenever possible use single quoted strings](https://github.com/RSS-Bridge/rss-bridge/wiki/Strings#whenever-possible-use-single-quote-strings)
-  * [Add spaces around the concatenation operator](https://github.com/RSS-Bridge/rss-bridge/wiki/Strings#add-spaces-around-the-concatenation-operator)
-  * [Use a single string instead of concatenating](https://github.com/RSS-Bridge/rss-bridge/wiki/Strings#use-a-single-string-instead-of-concatenating)
-* [Constants](https://github.com/RSS-Bridge/rss-bridge/wiki/Constants)
-  * [Use UPPERCASE for constants](https://github.com/RSS-Bridge/rss-bridge/wiki/Constants#use-uppercase-for-constants)
-* [Keywords](https://github.com/RSS-Bridge/rss-bridge/wiki/Keywords)
-  * [Use lowercase for `true`, `false` and `null`](https://github.com/RSS-Bridge/rss-bridge/wiki/Keywords#use-lowercase-for-true-false-and-null)
-* [Operators](https://github.com/RSS-Bridge/rss-bridge/wiki/Operators)
-  * [Operators must have a space around them](https://github.com/RSS-Bridge/rss-bridge/wiki/Operators#operators-must-have-a-space-around-them)
-* [Functions](https://github.com/RSS-Bridge/rss-bridge/wiki/Functions)
-  * [Parameters with default values must appear last in functions](https://github.com/RSS-Bridge/rss-bridge/wiki/Functions#parameters-with-default-values-must-appear-last-in-functions)
-  * [Calling functions](https://github.com/RSS-Bridge/rss-bridge/wiki/Functions#calling-functions)
-  * [Do not add spaces after opening or before closing bracket](https://github.com/RSS-Bridge/rss-bridge/wiki/Functions#do-not-add-spaces-after-opening-or-before-closing-bracket)
-* [Structures](https://github.com/RSS-Bridge/rss-bridge/wiki/Structures)
-  * [Structures must always be formatted as multi-line blocks](https://github.com/RSS-Bridge/rss-bridge/wiki/Structures#structures-must-always-be-formatted-as-multi-line-blocks)
-* [If-Statement](https://github.com/RSS-Bridge/rss-bridge/wiki/if-Statement)
-  * [Use `elseif` instead of `else if`](https://github.com/RSS-Bridge/rss-bridge/wiki/if-Statement#use-elseif-instead-of-else-if)
-  * [Do not write empty statements](https://github.com/RSS-Bridge/rss-bridge/wiki/if-Statement#do-not-write-empty-statements)
-  * [Do not write unconditional if-statements](https://github.com/RSS-Bridge/rss-bridge/wiki/if-Statement#do-not-write-unconditional-if-statements)
-* [Classes](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes)
-  * [Use PascalCase for class names](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#use-pascalcase-for-class-names)
-  * [Do not use final statements inside final classes](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#do-not-use-final-statements-inside-final-classes)
-  * [Do not override methods to call their parent](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#do-not-override-methods-to-call-their-parent)
-  * [abstract and final declarations MUST precede the visibility declaration](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#abstract-and-final-declarations-must-precede-the-visibility-declaration)
-  * [static declaration MUST come after the visibility declaration](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#static-declaration-must-come-after-the-visibility-declaration)
-* [Casting](https://github.com/RSS-Bridge/rss-bridge/wiki/Casting)
-  * [Do not add spaces when casting](https://github.com/RSS-Bridge/rss-bridge/wiki/Casting#do-not-add-spaces-when-casting)
+See the [Coding style policy page on the documentation](https://rss-bridge.github.io/rss-bridge/For_Developers/Coding_style_policy.html) for more information on the coding style of the project.

.github/ISSUE_TEMPLATE/bridge-request.md

@@ -60,5 +60,5 @@ Please describe what you expect from the bridge. Whenever possible provide sampl
 
 Keep in mind that opening a request does not guarantee the bridge being implemented! That depends entirely on the interest and time of others to make the bridge for you.
 
-You can also implement your own bridge (with support of the community if needed). Find more information in the [RSS-Bridge Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki/For-developers) developer section.
+You can also implement your own bridge (with support of the community if needed). Find more information in the [RSS-Bridge Documentation](https://rss-bridge.github.io/rss-bridge/For_Developers/index.html) developer section.
 -->

.github/prtester-requirements.txt

@@ -0,0 +1,2 @@
+beautifulsoup4>=4.10.0
+requests>=2.26.0

.github/prtester.py

@@ -0,0 +1,101 @@
+import requests
+from bs4 import BeautifulSoup
+from datetime import datetime
+import os.path
+
+# This script is specifically written to be used in automation for https://github.com/RSS-Bridge/rss-bridge
+#
+# This will scrape the whitelisted bridges in the current state (port 3000) and the PR state (port 3001) of
+# RSS-Bridge, generate a feed for each of the bridges and save the output as html files.
+# It also replaces the default static CSS link with a hardcoded link to @em92's public instance, so viewing
+# the HTML file locally will actually work as designed.
+
+def testBridges(bridges,status):
+    for bridge in bridges:
+        if bridge.get('data-ref'): # Some div entries are empty, this ignores those
+            bridgeid = bridge.get('id')
+            bridgeid = bridgeid.split('-')[1] # this extracts a readable bridge name from the bridge metadata
+            bridgestring = '/?action=display&bridge=' + bridgeid + '&format=Html'
+            forms = bridge.find_all("form")
+            formid = 1
+            for form in forms:
+                # a bridge can have multiple contexts, named 'forms' in html
+                # this code will produce a fully working formstring that should create a working feed when called
+                # this will create an example feed for every single context, to test them all
+                formstring = ''
+                errormessages = []
+                parameters = form.find_all("input")
+                lists = form.find_all("select")
+                # this for/if mess cycles through all available input parameters, checks if it required, then pulls
+                # the default or examplevalue and then combines it all together into the formstring
+                # if an example or default value is missing for a required attribute, it will throw an error
+                # any non-required fields are not tested!!!
+                for parameter in parameters:
+                    if parameter.get('type') == 'hidden' and parameter.get('name') == 'context':
+                        cleanvalue = parameter.get('value').replace(" ","+")
+                        formstring = formstring + '&' + parameter.get('name') + '=' + cleanvalue
+                    if parameter.get('type') == 'number' or parameter.get('type') == 'text':
+                        if parameter.has_attr('required'):
+                            if parameter.get('placeholder') == '':
+                                if parameter.get('value') == '':
+                                    errormessages.append(parameter.get('name'))
+                                else:
+                                    formstring = formstring + '&' + parameter.get('name') + '=' + parameter.get('value')
+                            else:
+                                formstring = formstring + '&' + parameter.get('name') + '=' + parameter.get('placeholder')
+                    # same thing, just for checkboxes. If a checkbox is checked per default, it gets added to the formstring
+                    if parameter.get('type') == 'checkbox':
+                        if parameter.has_attr('checked'):
+                            formstring = formstring + '&' + parameter.get('name') + '=on'
+                for list in lists:
+                    selectionvalue = ''
+                    for selectionentry in list.contents:
+                        if 'selected' in selectionentry.attrs:
+                            selectionvalue = selectionentry.get('value')
+                            break
+                    if selectionvalue == '':
+                        selectionvalue = list.contents[0].get('value')
+                    formstring = formstring + '&' + list.get('name') + '=' + selectionvalue
+                if not errormessages:
+                    # if all example/default values are present, form the full request string, run the request, replace the static css
+                    # file with the url of em's public instance and then upload it to termpad.com, a pastebin-like-site.
+                    r = requests.get(URL + bridgestring + formstring)
+                    pagetext = r.text.replace('static/HtmlFormat.css','https://feed.eugenemolotov.ru/static/HtmlFormat.css')
+                    pagetext = pagetext.encode("utf_8")
+                    termpad = requests.post(url="https://termpad.com/", data=pagetext)
+                    termpadurl = termpad.text
+                    termpadurl = termpadurl.replace('termpad.com/','termpad.com/raw/')
+                    termpadurl = termpadurl.replace('\n','')
+                    with open(os.getcwd() + '/comment.txt', 'a+') as file:
+                        file.write("\n")
+                        file.write("| [`" + bridgeid + '-' + status + '-context' + str(formid) + "`](" + termpadurl + ") | " + date_time + " |")
+                else:
+                    # if there are errors (which means that a required value has no example or default value), log out which error appeared
+                    termpad = requests.post(url="https://termpad.com/", data=str(errormessages))
+                    termpadurl = termpad.text
+                    termpadurl = termpadurl.replace('termpad.com/','termpad.com/raw/')
+                    termpadurl = termpadurl.replace('\n','')
+                    with open(os.getcwd() + '/comment.txt', 'a+') as file:
+                        file.write("\n")
+                        file.write("| [`" + bridgeid + '-' + status + '-context' + str(formid) + "`](" + termpadurl + ") | " + date_time + " |")
+                formid += 1
+
+gitstatus = ["current", "pr"]
+now = datetime.now()
+date_time = now.strftime("%Y-%m-%d, %H:%M:%S")
+
+with open(os.getcwd() + '/comment.txt', 'w+') as file:
+    file.write(''' ## Pull request artifacts
+| file | last change |
+| ---- | ------ |''')
+
+for status in gitstatus: # run this twice, once for the current version, once for the PR version
+    if status == "current":
+        port = "3000" # both ports are defined in the corresponding workflow .yml file
+    elif status == "pr":
+        port = "3001"
+    URL = "http://localhost:" + port
+    page = requests.get(URL) # Use python requests to grab the rss-bridge main page
+    soup = BeautifulSoup(page.content, "html.parser") # use bs4 to turn the page into soup
+    bridges = soup.find_all("section") # get a soup-formatted list of all bridges on the rss-bridge page
+    testBridges(bridges,status) # run the main scraping code with the list of bridges and the info if this is for the current version or the pr version

.github/workflows/dockerbuild.yml

@@ -0,0 +1,61 @@
+name: Build Image on Commit and Release
+
+on:
+  push:
+    branches:
+      - 'master'
+    tags:
+      - '20*'
+
+env:
+  DOCKERHUB_SLUG: rssbridge/rss-bridge
+  GHCR_SLUG: ghcr.io/rss-bridge/rss-bridge
+
+jobs:
+  bake:
+    runs-on: ubuntu-latest
+    steps:
+      -
+        name: Checkout
+        uses: actions/checkout@v2.3.4
+      -  
+        name: Docker meta
+        id: docker_meta
+        uses: docker/metadata-action@v3.5.0
+        with:
+          images: |
+            ${{ env.DOCKERHUB_SLUG }}
+            ${{ env.GHCR_SLUG }}
+          tags: |
+            type=raw,value=latest
+            type=sha
+            type=ref,event=tag,enable=${{ startsWith(github.ref, 'refs/tags/20') }}
+            type=raw,value=stable,enable=${{ startsWith(github.ref, 'refs/tags/20') }}
+      -
+        name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+      -
+        name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1.6.0
+      -
+        name: Login to DockerHub
+        uses: docker/login-action@v1.10.0
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+      -
+        name: Login to GitHub Container Registry
+        uses: docker/login-action@v1.10.0
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      -
+        name: Build and push
+        uses: docker/bake-action@v1.6.0
+        with:
+          files: |
+            ./docker-bake.hcl
+            ${{ steps.docker_meta.outputs.bake-file }}
+          targets: image-all
+          push: true

.github/workflows/documentation.yml

@@ -0,0 +1,27 @@
+name: Documentation
+
+on:
+  push:
+    paths:
+    - 'docs/**'
+
+jobs:
+  documentation:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+      with:
+        persist-credentials: false
+    - name: Setup PHP
+      uses: shivammathur/setup-php@2.17.1
+      with:
+        php-version: 8.0
+    - name: Install dependencies
+      run: composer global require daux/daux.io
+    - name: Generate documentation
+      run: daux generate
+    - name: Deploy same repository 🚀
+      uses: JamesIves/github-pages-deploy-action@v4.2.5
+      with:
+        folder: "static"
+        branch: gh-pages

.github/workflows/lint.yml

@@ -8,10 +8,10 @@ on:
 
 jobs:
   phpcs:
-    runs-on: ubuntu-16.04
+    runs-on: ubuntu-20.04
     strategy:
       matrix:
-        php-versions: ['5.6', '7.0', '7.1', '7.2', '7.3', '7.4']
+        php-versions: ['7.1', '7.2', '7.3', '7.4']
     steps:
       - uses: actions/checkout@v2
       - uses: shivammathur/setup-php@v2
@@ -21,10 +21,10 @@ jobs:
       - run: phpcs . --standard=phpcs.xml --warning-severity=0 --extensions=php -p
 
   phpcompatibility:
-    runs-on: ubuntu-16.04
+    runs-on: ubuntu-20.04
     strategy:
       matrix:
-        php-versions: ['5.6', '7.4']
+        php-versions: ['7.1', '7.4']
     steps:
       - uses: actions/checkout@v2
       - uses: shivammathur/setup-php@v2
@@ -33,3 +33,16 @@ jobs:
       - run: composer global require dealerdirect/phpcodesniffer-composer-installer
       - run: composer global require phpcompatibility/php-compatibility
       - run: ~/.composer/vendor/bin/phpcs . --standard=phpcompatibility.xml --warning-severity=0 --extensions=php -p
+
+  executable_php_files_check:
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v2
+      - run: |
+          if find -name "*.php" -executable -type f -print -exec false {} +
+          then
+            echo 'Good, no executable php scripts found'
+          else
+            echo 'Please unmark php scripts above as non-executable'
+            exit 1
+          fi

.github/workflows/prhtmlgenerator.yml

@@ -0,0 +1,69 @@
+name: 'PR Testing'
+
+on:
+  pull_request_target:
+    branches: [ master ]
+
+jobs:
+  test-pr:
+    name: Generate HTML
+    runs-on: ubuntu-latest
+    # Needs additional permissions https://github.com/actions/first-interaction/issues/10#issuecomment-1041402989
+    steps:
+      - name: Check out self
+        uses: actions/checkout@v2.3.2
+        with:
+          ref: ${{github.event.pull_request.head.ref}}
+          repository: ${{github.event.pull_request.head.repo.full_name}}
+      - name: Check out rss-bridge
+        run: |
+          PR=${{github.event.number}};
+          wget -O requirements.txt https://raw.githubusercontent.com/RSS-Bridge/rss-bridge/master/.github/prtester-requirements.txt;
+          wget https://raw.githubusercontent.com/RSS-Bridge/rss-bridge/master/.github/prtester.py;
+          wget https://patch-diff.githubusercontent.com/raw/$GITHUB_REPOSITORY/pull/$PR.patch;
+          touch DEBUG;
+          cat $PR.patch | grep " bridges/.*\.php" | sed "s= bridges/\(.*\)Bridge.php.*=\1=g" | sort | uniq > whitelist.txt
+      - name: Start Docker - Current
+        run: |
+          docker run -d -v $GITHUB_WORKSPACE/whitelist.txt:/app/whitelist.txt -v $GITHUB_WORKSPACE/DEBUG:/app/DEBUG -p 3000:80 ghcr.io/rss-bridge/rss-bridge:latest
+      - name: Start Docker - PR
+        run: |
+          docker build -t prbuild .;
+          docker run -d -v $GITHUB_WORKSPACE/whitelist.txt:/app/whitelist.txt -v $GITHUB_WORKSPACE/DEBUG:/app/DEBUG -p 3001:80 prbuild
+      - name: Setup python
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.7'
+          cache: 'pip'
+      - name: Install requirements
+        run: |
+          cd $GITHUB_WORKSPACE
+          pip install -r requirements.txt
+      - name: Run bridge tests
+        id: testrun
+        run: |
+          mkdir results;
+          python prtester.py;
+          body="$(cat comment.txt)";
+          body="${body//'%'/'%25'}";
+          body="${body//$'\n'/'%0A'}";
+          body="${body//$'\r'/'%0D'}";
+          echo "::set-output name=bodylength::${#body}"
+          echo "::set-output name=body::$body"
+      - name: Find Comment
+        if: ${{ steps.testrun.outputs.bodylength > 130 }}
+        uses: peter-evans/find-comment@v2
+        id: fc
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          comment-author: 'github-actions[bot]'
+          body-includes: Pull request artifacts
+      - name: Create or update comment
+        if: ${{ steps.testrun.outputs.bodylength > 130 }}
+        uses: peter-evans/create-or-update-comment@v2
+        with:
+          comment-id: ${{ steps.fc.outputs.comment-id }}
+          issue-number: ${{ github.event.pull_request.number }}
+          body: |
+            ${{ steps.testrun.outputs.body }}
+          edit-mode: replace