Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🆕 Devcontainer support, Docker support and IPv6 inside container fix #191

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
17 changes: 17 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "slrp dev",
"image": "mcr.microsoft.com/devcontainers/go:1-1.20-bookworm",
"postCreateCommand": ".devcontainer/postcreate.sh",
"remoteUser": "root",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need root?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

w/o the container can't init nvm (run source ~/.bashrc) properly.
Since this is only for the devcontainer, I think it's fine security-wise

"customizations": {
"vscode": {
"extensions": [
"foxundermoon.shell-format",
"GitHub.copilot",
"eamodio.gitlens",
"ms-vscode.makefile-tools",
"ms-azuretools.vscode-docker"
]
}
}
}
35 changes: 35 additions & 0 deletions .devcontainer/postcreate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/bin/bash
NVM_VERSION="0.38.0"
NODE_VERSION="20"

# # If you're developing in a container behind a VPNed/MDM-managed machine you might get "self signed certs in chain" error.
# # This snippet below bypasses that (ssl is required later for nvm installation)
# apt update && apt install -y git
# git config --global http.sslVerify false

echo "[ + ] Running post-create script. Installing the following:"
echo "[ + ] NVM version: $NVM_VERSION"
echo "[ + ] NodeJS version: $NODE_VERSION"
echo "[ + ] =============================="

# Check if we don't have nvm, if no - install it
echo "[ + ] Installing nvm at v$NVM_VERSION"

curl -k -o- https://raw.githubusercontent.com/nvm-sh/nvm/v$NVM_VERSION/install.sh | bash
# Activate nvm
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

# Refresh bash
source ~/.bashrc

# Install node at a set versions
echo "[ + ] Installing NodeJS at v$NODE_VERSION"
nvm install $NODE_VERSION && nvm use $NODE_VERSION

echo "[ + ] Installing UI dependencies and building..."
# Change to the "ui" directory
cd ui && npm install && cd ../

echo "[ + ] Installing Go dependencies and building..."
make build
18 changes: 18 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
// Run the main.go program
{
"name": "Launch Package",
"type": "go",
"request": "launch",
"mode": "debug",
"program": "${workspaceFolder}/main.go",
"env": {},
"args": []
},
]
}
53 changes: 30 additions & 23 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,27 +1,34 @@
FROM alpine
# Install node and deps to build the frontend
FROM node:20.11-bookworm AS NODE_INSTALL
WORKDIR /app
COPY . .
RUN npm --prefix ui install && \
npm --prefix ui run build

ENV PWD="/app"
# Install go and deps to build the backend
FROM golang:1.20.13-bookworm AS BUILD
WORKDIR /app
COPY --from=NODE_INSTALL /app .
RUN make build-go-for-docker

# Final image
FROM alpine:latest
# SLRP configuration environment variables
ENV SLRP_APP_STATE="$PWD/.slrp/data"
ENV SLRP_APP_SYNC="1m"
ENV SLRP_LOG_LEVEL="info"
ENV SLRP_LOG_FORMAT="pretty"
ENV SLRP_SERVER_ADDR="0.0.0.0:8089"
ENV SLRP_SERVER_READ_TIMEOUT="15s"
ENV SLRP_MITM_ADDR="0.0.0.0:8090"
ENV SLRP_MITM_READ_TIMEOUT="15s"
ENV SLRP_MITM_IDLE_TIMEOUT="15s"
ENV SLRP_MITM_WRITE_TIMEOUT="15s"
ENV SLRP_CHECKER_TIMEOUT="5s"
ENV SLRP_CHECKER_STRATEGY="simple"
ENV SLRP_HISTORY_LIMIT="1000"

WORKDIR $PWD
COPY slrp $PWD

RUN mkdir ./.slrp

ENV SLRP_APP_STATE="/opt/.slrp/data" \
SLRP_APP_SYNC="1m" \
SLRP_LOG_LEVEL="info" \
SLRP_LOG_FORMAT="pretty" \
SLRP_SERVER_ADDR="0.0.0.0:8089" \
SLRP_SERVER_READ_TIMEOUT="15s" \
SLRP_MITM_ADDR="0.0.0.0:8090" \
SLRP_MITM_READ_TIMEOUT="15s" \
SLRP_MITM_IDLE_TIMEOUT="15s" \
SLRP_MITM_WRITE_TIMEOUT="15s" \
SLRP_CHECKER_TIMEOUT="5s" \
SLRP_CHECKER_STRATEGY="simple" \
SLRP_HISTORY_LIMIT="1000"
WORKDIR /opt
COPY --from=BUILD /app/main /opt/slrp
RUN mkdir -p ./.slrp/data
EXPOSE 8089 8090

CMD ["./slrp"]
CMD ["/opt/slrp"]
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ build: build-ui
go mod vendor
go build -ldflags "-s -w" main.go

# When running inside Alpine images there are no classic OS packages/binaries enabled, hence - we compile statically (CGO)
build-go-for-docker:
go mod vendor
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags "-s -w" -o main main.go
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use cgo here at all, pls remove

Copy link
Author

@aviadhahami aviadhahami Feb 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, tho when running the project on Alpine it fails unless compiling w/ CGO.
The CGO is also used here: https://github.com/nfx/slrp/blob/main/.goreleaser.yaml#L13.
This topic specifically was previously discussed here: #90 (comment)

"...but slrp must be build with static linkage CGO_ENABLED=0 I believe..."

wdyt?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the situation where a code comment could have avoided a review comment 😉

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add the comment where needed ;)


docker:
docker build -t slrp:latest .

quick:
go build

Expand Down
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,17 @@ SLRP - rotating open proxy multiplexer
* Packaged as a single executable binary, that also includes Web UI

# Usage
For all methods, wait couple of minutes for the pool to pick up. Check the dashboard at [http://localhost:8089/](http://localhost:8089/) for progress.

Download service, start it up, wait couple of minutes for the pool to pick up. Now run `curl --proxy-insecure -D - -x http://127.0.0.1:8090 -k http://httpbin.org/get` couple of times and see different origins and user agent headers.
## Via binary
Download the binary from the releases, which can be found [here](https://github.com/nfx/slrp/releases)

## Via Docker
> Assuming you have docker and make present
Run `make docker`. Once done, invoke with `docker run -p 8089:8089 -p 8090:8090 -v $HOME/.slrp/data:/data nfx/slrp:latest`

Once running, you can access the UI at [http://localhost:8089/](http://localhost:8089/) and the proxy at [http://localhost:8090/](http://localhost:8090/)
Test using a simple curl command `curl --proxy-insecure -D - -x http:// http://127.0.0.1:8090 -k http://httpbin.org/get` couple of times and see different origins and user agent headers.
Copy link

@muzso muzso Feb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be an extra http:// in the curl example command.

So instead of this:
curl --proxy-insecure -D - -x http:// http://127.0.0.1:8090 -k http://httpbin.org/get

It should be this:
curl --proxy-insecure -D - -x http://127.0.0.1:8090 -k http://httpbin.org/get

The -k curl option is only necessary if the target URL uses https, so either the -k can be dropped, or the target URL changed into https://httpbin.org/get.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using slrp as a proxy seems to require a https connection (to the proxy).
The --proxy-insecure option in the example suggests this as well.
But the example uses plain http:// for the proxy URL.

So the correct example is:
curl --proxy-insecure -D - -x "https://127.0.0.1:8090" "http://httpbin.org/get"

Or this:
curl --proxy-insecure -D - -x "https://127.0.0.1:8090" -k "https://httpbin.org/get"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated 👍


# Concepts

Expand Down
10 changes: 10 additions & 0 deletions pmux/proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -247,8 +247,18 @@ func NewProxyFromURL(url string) Proxy {
}

func NewProxy(addr string, t string) Proxy {
// Check if the address is valid or contains "[::]"; This happens when running inside a docker container
// It means that the address is listening on all interfaces but via IPv6, which is not supported by the
// proxy package(or so). Hence we replace it with 0.0.0.0
if strings.Contains(addr, "[::]") {
// Set it to 0.0.0.0 but maintain the port
fmt.Println("Encountered [::]: in address, replacing with 0.0.0.0")
Copy link

@muzso muzso Feb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be a call to log.Info() instead of directly writing to stdout?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed 👍

addr = strings.Replace(addr, "[::]", "0.0.0.0", 1)
}

addrPort, err := netip.ParseAddrPort(addr)
if err != nil {
fmt.Println("Error parsing address:", err)
Copy link

@muzso muzso Feb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be a call to log.Info() instead of directly writing to stdout?

return 0
}
p, ok := protoMap[t]
Expand Down
2 changes: 1 addition & 1 deletion sources/try_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ func Test_mergeSrc_Len(t *testing.T) {
case pmux.HttpProxy("127.0.0.1:2048"):
canAssertB <- true
}
t.Logf("received: %s", v)
t.Logf("received: %v", v)
}
}()

Expand Down