Skip to content

Commit

Permalink
Merge branch 'patch-1' of github.com:GanZhiXiong/transfer.sh into pat…
Browse files Browse the repository at this point in the history
…ch-1

* 'patch-1' of github.com:GanZhiXiong/transfer.sh:
  issue dutchcoders#420 added MaxDate.IsZero() check (dutchcoders#427)
  fix go1.16, add go1.17
  bump bluemonday
  Revert "issue dutchcoders#420 return 400 response when Max-Days is too big (dutchcoders#422)" (dutchcoders#426)
  issue dutchcoders#420 return 400 response when Max-Days is too big (dutchcoders#422)
  Implement Nix Flake (dutchcoders#424)
  fix missed errors (dutchcoders#417)
  Fix path (dutchcoders#416)
  Edited code of condunct for more information and corrected a grammatical error (dutchcoders#421)
  Update README.md (dutchcoders#415)
  Update README.md
  • Loading branch information
GanZhiXiong committed Nov 10, 2021
2 parents 325a5c9 + c692b4d commit 71fd8d8
Show file tree
Hide file tree
Showing 11 changed files with 301 additions and 24 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: ^1.16
go-version: ^1.17

- name: Get project dependencies
run: go mod download
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ jobs:
- 1.14.x
- 1.15.x
- 1.16.x
- 1.17.x
name: Test with ${{ matrix.go_version }}
steps:
- uses: actions/checkout@v2
Expand Down
3 changes: 2 additions & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,13 @@ Examples of unacceptable behavior by participants include:
* Public or private harassment
* Publishing other's private information, such as physical or electronic addresses, without explicit permission
* Other unethical or unprofessional conduct
* Use of harsh language

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. By adopting this Code of Conduct, project maintainers commit themselves to fairly and consistently applying these principles to every aspect of managing this project. Project maintainers who do not follow or enforce the Code of Conduct may be permanently removed from the project team.

This code of conduct applies both within project spaces and in public spaces when an individual is representing the project or its community.

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.

This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 1.2.0, available at https://www.contributor-covenant.org/version/1/2/0/code-of-conduct.html
This Code of Conduct is adapted from the [Contributor Covenant] (https://www.contributor-covenant.org), version 1.2.0, available at https://www.contributor-covenant.org/version/1/2/0/code-of-conduct.html

4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Default to Go 1.16
ARG GO_VERSION=1.16
# Default to Go 1.17
ARG GO_VERSION=1.17
FROM golang:${GO_VERSION}-alpine as build

# Necessary to run 'go get' and to compile the linked binary
Expand Down
27 changes: 13 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,17 @@ The service at transfersh.com is of unknown origin and reported as cloud malware
$ curl --upload-file ./hello.txt https://transfer.sh/hello.txt
```

### Encrypt & upload:
### Encrypt & Upload:
```bash
$ cat /tmp/hello.txt|gpg -ac -o-|curl -X PUT --upload-file "-" https://transfer.sh/test.txt
````

### Download & decrypt:
### Download & Decrypt:
```bash
$ curl https://transfer.sh/1lDau/test.txt|gpg -o- > /tmp/hello.txt
```

### Upload to virustotal:
### Upload to Virustotal:
```bash
$ curl -X PUT --upload-file nhgbhhj https://transfer.sh/test.txt/virustotal
```
Expand All @@ -51,7 +51,7 @@ $ curl --upload-file ./hello.txt https://transfer.sh/hello.txt -H "Max-Days: 1"

### X-Url-Delete

The URL used to request the deletion of a file. Returned as a response header.
The URL used to request the deletion of a file and returned as a response header.
```bash
curl -sD - --upload-file ./hello https://transfer.sh/hello.txt | grep 'X-Url-Delete'
X-Url-Delete: https://transfer.sh/hello.txt/BAYh0/hello.txt/PDw0NHPcqU
Expand Down Expand Up @@ -161,23 +161,23 @@ To use a custom non-AWS S3 provider, you need to specify the endpoint as defined

## Storj Network Provider

To use the Storj Network as storage provider you need to specify the following flags:
To use the Storj Network as a storage provider you need to specify the following flags:
- provider `--provider storj`
- storj-access _(either via flag or environment variable STORJ_ACCESS)_
- storj-bucket _(either via flag or environment variable STORJ_BUCKET)_

### Creating Bucket and Scope

In preparation you need to create an access grant (or copy it from the uplink configuration) and a bucket.
You need to create an access grant (or copy it from the uplink configuration) and a bucket in preparation.

To get started, login to your account and go to the Access Grant Menu and start the Wizard on the upper right.
To get started, log in to your account and go to the Access Grant Menu and start the Wizard on the upper right.

Enter your access grant name of choice, hit *Next* and restrict it as necessary/preferred.
Aftwards continue either in CLI or within the Browser. You'll be asked for a Passphrase used as Encryption Key.
**Make sure to save it in a safe place, without it you will lose the ability to decrypt your files!**
Afterwards continue either in CLI or within the Browser. Next, you'll be asked for a Passphrase used as Encryption Key.
**Make sure to save it in a safe place. Without it, you will lose the ability to decrypt your files!**
Afterwards you can copy the access grant and then start the startup of the transfer.sh endpoint.
For enhanced security its recommended to provide both the access grant and the bucket name as ENV Variables.
Afterwards, you can copy the access grant and then start the startup of the transfer.sh endpoint.
It is recommended to provide both the access grant and the bucket name as ENV Variables for enhanced security.
Example:
```
Expand All @@ -196,8 +196,7 @@ For the usage with Google drive, you need to specify the following options:
### Creating Gdrive Client Json
You need to create a Oauth Client id from console.cloud.google.com
download the file and place into a safe directory
You need to create an OAuth Client id from console.cloud.google.com, download the file, and place it into a safe directory.
### Usage example
Expand All @@ -221,7 +220,7 @@ Contributions are welcome.
**Stefan Benten**
## Copyright and license
## Copyright and License
Code and documentation copyright 2011-2018 Remco Verhoef.
Code and documentation copyright 2018-2020 Andrea Spacca.
Expand Down
41 changes: 41 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

211 changes: 211 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
{
description = "Transfer.sh";

inputs.flake-utils.url = "github:numtide/flake-utils";

outputs = { self, nixpkgs, flake-utils }:
let
transfer-sh = pkgs: pkgs.buildGoModule {
src = self;
name = "transfer.sh";
vendorSha256 = "sha256-bgQUMiC33yVorcKOWhegT1/YU+fvxsz2pkeRvjf3R7g=";
};
in

flake-utils.lib.eachDefaultSystem (
system:
let
pkgs = nixpkgs.legacyPackages.${system};
in
rec {
packages = flake-utils.lib.flattenTree {
transfer-sh = transfer-sh pkgs;
};
defaultPackage = packages.transfer-sh;
apps.transfer-sh = flake-utils.lib.mkApp { drv = packages.transfer-sh; };
defaultApp = apps.transfer-sh;
}
) // rec {

nixosModules = {
transfer-sh = { config, lib, pkgs, ... }: with lib; let
RUNTIME_DIR = "/var/lib/transfer.sh";
cfg = config.services.transfer-sh;

general_options = {

enable = mkEnableOption "Transfer.sh service";
listener = mkOption { default = 80; type = types.int; description = "port to use for http (:80)"; };
profile-listener = mkOption { default = 6060; type = types.int; description = "port to use for profiler (:6060)"; };
force-https = mkOption { type = types.nullOr types.bool; description = "redirect to https"; };
tls-listener = mkOption { default = 443; type = types.int; description = "port to use for https (:443)"; };
tls-listener-only = mkOption { type = types.nullOr types.bool; description = "flag to enable tls listener only"; };
tls-cert-file = mkOption { type = types.nullOr types.str; description = "path to tls certificate"; };
tls-private-key = mkOption { type = types.nullOr types.str; description = "path to tls private key "; };
http-auth-user = mkOption { type = types.nullOr types.str; description = "user for basic http auth on upload"; };
http-auth-pass = mkOption { type = types.nullOr types.str; description = "pass for basic http auth on upload"; };
ip-whitelist = mkOption { type = types.nullOr types.str; description = "comma separated list of ips allowed to connect to the service"; };
ip-blacklist = mkOption { type = types.nullOr types.str; description = "comma separated list of ips not allowed to connect to the service"; };
temp-path = mkOption { type = types.nullOr types.str; description = "path to temp folder"; };
web-path = mkOption { type = types.nullOr types.str; description = "path to static web files (for development or custom front end)"; };
proxy-path = mkOption { type = types.nullOr types.str; description = "path prefix when service is run behind a proxy"; };
proxy-port = mkOption { type = types.nullOr types.str; description = "port of the proxy when the service is run behind a proxy"; };
ga-key = mkOption { type = types.nullOr types.str; description = "google analytics key for the front end"; };
uservoice-key = mkOption { type = types.nullOr types.str; description = "user voice key for the front end"; };
lets-encrypt-hosts = mkOption { type = types.nullOr (types.listOf types.str); description = "hosts to use for lets encrypt certificates"; };
log = mkOption { type = types.nullOr types.str; description = "path to log file"; };
cors-domains = mkOption { type = types.nullOr (types.listOf types.str); description = "comma separated list of domains for CORS, setting it enable CORS "; };
clamav-host = mkOption { type = types.nullOr types.str; description = "host for clamav feature"; };
rate-limit = mkOption { type = types.nullOr types.int; description = "request per minute"; };
max-upload-size = mkOption { type = types.nullOr types.int; description = "max upload size in kilobytes "; };
purge-days = mkOption { type = types.nullOr types.int; description = "number of days after the uploads are purged automatically "; };
random-token-length = mkOption { type = types.nullOr types.int; description = "length of the random token for the upload path (double the size for delete path)"; };

};

provider_options = {

aws = {
enable = mkEnableOption "Enable AWS backend";
aws-access-key = mkOption { type = types.str; description = "aws access key"; };
aws-secret-key = mkOption { type = types.str; description = "aws secret key"; };
bucket = mkOption { type = types.str; description = "aws bucket "; };
s3-endpoint = mkOption {
type = types.nullOr types.str;
description = ''
Custom S3 endpoint.
If you specify the s3-region, you don't need to set the endpoint URL since the correct endpoint will used automatically.
'';
};
s3-region = mkOption { type = types.str; description = "region of the s3 bucket eu-west-"; };
s3-no-multipart = mkOption { type = types.nullOr types.bool; description = "disables s3 multipart upload "; };
s3-path-style = mkOption { type = types.nullOr types.str; description = "Forces path style URLs, required for Minio. "; };
};

storj = {
enable = mkEnableOption "Enable storj backend";
storj-access = mkOption { type = types.str; description = "Access for the project"; };
storj-bucket = mkOption { type = types.str; description = "Bucket to use within the project"; };
};

gdrive = {
enable = mkEnableOption "Enable gdrive backend";
gdrive-client-json = mkOption { type = types.str; description = "oauth client json config for gdrive provider"; };
gdrive-chunk-size = mkOption { default = 8; type = types.nullOr types.int; description = "chunk size for gdrive upload in megabytes, must be lower than available memory (8 MB)"; };
basedir = mkOption { type = types.str; description = "path storage for gdrive provider"; default = "${cfg.stateDir}/store"; };
purge-interval = mkOption { type = types.nullOr types.int; description = "interval in hours to run the automatic purge for (not applicable to S3 and Storj)"; };

};

local = {
enable = mkEnableOption "Enable gdrive backend";
basedir = mkOption { type = types.str; description = "path storage for local provider"; default = "${cfg.stateDir}/store"; };
purge-interval = mkOption { type = types.nullOr types.int; description = "interval in hours to run the automatic purge for (not applicable to S3 and Storj)"; };
};

};
in
{
options.services.transfer-sh = fold recursiveUpdate {} [
general_options
{
provider = provider_options;
user = mkOption {
type = types.str;
description = "User to run the service under";
default = "transfer.sh";
};
group = mkOption {
type = types.str;
description = "Group to run the service under";
default = "transfer.sh";
};
stateDir = mkOption {
type = types.path;
description = "Variable state directory";
default = RUNTIME_DIR;
};
}
];

config = let

mkFlags = cfg: options:
let
mkBoolFlag = option: if cfg.${option} then [ "--${option}" ] else [];
mkFlag = option:
if isBool cfg.${option}
then mkBoolFlag option
else [ "--${option}" "${cfg.${option}}" ];

in
lists.flatten (map (mkFlag) (filter (option: cfg.${option} != null && option != "enable") options));

aws-config = (mkFlags cfg.provider.aws (attrNames provider_options)) ++ [ "--provider" "aws" ];
gdrive-config = mkFlags cfg.provider.gdrive (attrNames provider_options.gdrive) ++ [ "--provider" "gdrive" ];
storj-config = mkFlags cfg.provider.storj (attrNames provider_options.storj) ++ [ "--provider" "storj" ];
local-config = mkFlags cfg.provider.local (attrNames provider_options.local) ++ [ "--provider" "local" ];

general-config = concatStringsSep " " (mkFlags cfg (attrNames general_options));
provider-config = concatStringsSep " " (
if cfg.provider.aws.enable && !cfg.provider.storj.enable && !cfg.provider.gdrive.enable && !cfg.provider.local.enable then aws-config
else if !cfg.provider.aws.enable && cfg.provider.storj.enable && !cfg.provider.gdrive.enable && !cfg.provider.local.enable then storj-config
else if !cfg.provider.aws.enable && !cfg.provider.storj.enable && cfg.provider.gdrive.enable && !cfg.provider.local.enable then gdrive-config
else if !cfg.provider.aws.enable && !cfg.provider.storj.enable && !cfg.provider.gdrive.enable && cfg.provider.local.enable then local-config
else throw "transfer.sh requires exactly one provider (aws, storj, gdrive, local)"
);

in
lib.mkIf cfg.enable
{
systemd.tmpfiles.rules = [
"d ${cfg.stateDir} 0750 ${cfg.user} ${cfg.group} - -"
] ++ optional cfg.provider.gdrive.enable cfg.provider.gdrive.basedir
++ optional cfg.provider.local.enable cfg.provider.local.basedir;

systemd.services.transfer-sh = {
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
serviceConfig = {
User = cfg.user;
Group = cfg.group;
ExecStart = "${transfer-sh pkgs}/bin/transfer.sh ${general-config} ${provider-config} ";
};
};

networking.firewall.allowedTCPPorts = [ cfg.listener cfg.profile-listener cfg.tls-listener ];
};
};

default = { self, pkgs, ... }: {
imports = [ nixosModules.transfer-sh ];
# Network configuration.

# useDHCP is generally considered to better be turned off in favor
# of <adapter>.useDHCP
networking.useDHCP = false;
networking.firewall.allowedTCPPorts = [];

# Enable the inventaire server.
services.transfer-sh = {
enable = true;
provider.local = {
enable = true;
};
};

nixpkgs.config.allowUnfree = true;
};
};


nixosConfigurations."container" = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
nixosModules.default
({ ... }: { boot.isContainer = true; })
];
};

};
}
5 changes: 3 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ require (
github.com/VojtechVitek/ratelimit v0.0.0-20160722140851-dc172bc0f6d2
github.com/aws/aws-sdk-go v1.37.14
github.com/calebcase/tmpfile v1.0.2 // indirect
github.com/chris-ramon/douceur v0.2.0 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect
github.com/dutchcoders/go-clamd v0.0.0-20170520113014-b970184f4d9e
github.com/dutchcoders/go-virustotal v0.0.0-20140923143438-24cc8e6fa329
Expand All @@ -19,15 +20,15 @@ require (
github.com/gorilla/handlers v1.5.1
github.com/gorilla/mux v1.8.0
github.com/gorilla/securecookie v1.1.1 // indirect
github.com/microcosm-cc/bluemonday v1.0.5
github.com/microcosm-cc/bluemonday v1.0.16
github.com/nu7hatch/gouuid v0.0.0-20131221200532-179d4d0c4d8d // indirect
github.com/russross/blackfriday/v2 v2.1.0
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e
github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce
github.com/urfave/cli v1.22.5
go.opencensus.io v0.22.6 // indirect
golang.org/x/crypto v0.0.0-20210415154028-4f45737414dc
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
golang.org/x/net v0.0.0-20210614182718-04defd469f4e
golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99
google.golang.org/api v0.40.0
google.golang.org/genproto v0.0.0-20210218151259-fe80b386bf06 // indirect
Expand Down
Loading

0 comments on commit 71fd8d8

Please sign in to comment.