Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace grpcio with tonic #2112

Merged
merged 24 commits into from
Jun 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
cc38079
Begin replacing grpio with tonic
Jake-Shadle May 25, 2021
5085d9b
"Finish" porting to tonic
Jake-Shadle May 26, 2021
1be03ed
Add metadata fields to manifest
Jake-Shadle May 26, 2021
2fedfac
Allow optional port assignment
Jake-Shadle May 27, 2021
4aa3db4
Fix field name
Jake-Shadle May 27, 2021
defc1ee
Remove unused generic
Jake-Shadle May 27, 2021
46a27e5
Print out error when unable to get game server
Jake-Shadle May 27, 2021
410f818
Pull out health check into its own separate task
Jake-Shadle May 27, 2021
89d5ffd
Update rust-simple example to work with updated SDK
Jake-Shadle May 28, 2021
ee05aef
Remove unused generic parameter
Jake-Shadle May 28, 2021
410940d
Fix rust tests
Jake-Shadle May 28, 2021
c30a32c
Remove unneeded cruft from rust build image
Jake-Shadle May 28, 2021
dafdd74
Remove lines causing excessive rebuilds
Jake-Shadle May 28, 2021
82269bd
Fix rust CI test, add prefix to log messages
Jake-Shadle May 31, 2021
dec667d
Address PR feedback
Jake-Shadle Jun 16, 2021
fd38dda
Update rust SDK
Jake-Shadle Jun 16, 2021
d650d4b
spawn_health_task => health_check
Jake-Shadle Jun 16, 2021
905fdf5
Use internal timeout for SDK connection
Jake-Shadle Jun 16, 2021
b7a9eba
Do continuous health checks
Jake-Shadle Jun 18, 2021
8b8f291
Copy protobuffers before build
Jake-Shadle Jun 18, 2021
5df8948
Bump example version
Jake-Shadle Jun 18, 2021
776c861
Add trailing newline
Jake-Shadle Jun 18, 2021
3982b07
Ensure protos are copied for the test
Jake-Shadle Jun 18, 2021
ed847f3
Vendor protos again
Jake-Shadle Jun 22, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 0 additions & 19 deletions build/build-sdk-images/rust/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,25 +31,6 @@ RUN wget -q https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/ru
cargo --version; \
rustc --version;

# install rust tooling for SDK generation
RUN cargo install protobuf-codegen --vers 2.16.2
RUN cargo install grpcio-compiler --vers 0.6.0

RUN wget -q https://cmake.org/files/v3.14/cmake-3.14.1-Linux-x86_64.sh
RUN mkdir /opt/cmake
RUN sh cmake-3.14.1-Linux-x86_64.sh --prefix=/opt/cmake --skip-license
RUN ln -s /opt/cmake/bin/cmake /usr/local/bin/cmake
RUN cmake --version

ENV GO_VERSION=1.10.2 \
GO_CHECKSUM=4b677d698c65370afa33757b6954ade60347aaca310ea92a63ed717d7cb0c2ff
RUN mkdir -p /usr/local/go \
&& curl -fSO https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz \
&& shasum -a 256 go${GO_VERSION}.linux-amd64.tar.gz | grep ${GO_CHECKSUM} \
&& tar xf go${GO_VERSION}.linux-amd64.tar.gz -C /usr/local/go --strip-components=1 \
&& rm -f go${GO_VERSION}.linux-amd64.tar.gz
ENV PATH $PATH:/usr/local/go/bin

# code generation scripts
COPY *.sh /root/
RUN chmod +x /root/*.sh
1 change: 1 addition & 0 deletions build/build-sdk-images/rust/build-sdk-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
# limitations under the License.

set -ex

mkdir -p /go/src/agones.dev/agones/test/sdk/rust/.cargo
mkdir -p /go/src/agones.dev/agones/test/sdk/rust/.cargo-targets
cd /go/src/agones.dev/agones/test/sdk/rust
Expand Down
25 changes: 6 additions & 19 deletions build/build-sdk-images/rust/gen.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,25 +16,12 @@

set -ex

googleapis=/go/src/agones.dev/agones/proto/googleapis
protos=/go/src/agones.dev/agones/proto
dest=/go/src/agones.dev/agones/sdks/rust

sdk=/go/src/agones.dev/agones/proto/sdk
cd /go/src/agones.dev/agones
rm -rf ${dest}/proto

protoc \
-I ${googleapis} -I ${sdk} sdk.proto \
--rust_out=sdks/rust/src/grpc --grpc_out=sdks/rust/src/grpc \
--plugin=protoc-gen-grpc=`which grpc_rust_plugin`
protoc \
-I ${googleapis} -I ${sdk}/alpha alpha.proto \
--rust_out=sdks/rust/src/grpc --grpc_out=sdks/rust/src/grpc \
--plugin=protoc-gen-grpc=`which grpc_rust_plugin`
echo "Copying protobuffers to rust sdk"
cp -r ${protos} ${dest}

cat ./build/boilerplate.go.txt ./sdks/rust/src/grpc/sdk.rs >> ./sdk.rs
cat ./build/boilerplate.go.txt ./sdks/rust/src/grpc/sdk_grpc.rs >> ./sdk_grpc.rs
cat ./build/boilerplate.go.txt ./sdks/rust/src/grpc/alpha.rs >> ./alpha.rs
cat ./build/boilerplate.go.txt ./sdks/rust/src/grpc/alpha_grpc.rs >> ./alpha_grpc.rs
mv ./sdk.rs ./sdks/rust/src/grpc/
mv ./sdk_grpc.rs ./sdks/rust/src/grpc/
mv ./alpha.rs ./sdks/rust/src/grpc/
mv ./alpha_grpc.rs ./sdks/rust/src/grpc/
echo "Rust code is generated at build time"
11 changes: 11 additions & 0 deletions examples/rust-simple/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,17 @@
[package]
name = "rust-simple"
version = "0.1.0"
edition = "2018"

[dependencies]
agones = { path = "../../sdks/rust" }

[dependencies.tokio]
version = "1.6"
default-features = false
features = [
"macros",
"rt-multi-thread",
"sync",
"time",
]
2 changes: 1 addition & 1 deletion examples/rust-simple/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ REPOSITORY ?= gcr.io/agones-images

mkfile_path := $(abspath $(lastword $(MAKEFILE_LIST)))
project_path := $(dir $(mkfile_path))
server_tag = $(REPOSITORY)/rust-simple-server:0.8
server_tag = $(REPOSITORY)/rust-simple-server:0.9

# _____ _
# |_ _|_ _ _ __ __ _ ___| |_ ___
Expand Down
121 changes: 81 additions & 40 deletions examples/rust-simple/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,25 +12,13 @@
// See the License for the specific language governing permissions and
// limitations under the License.

extern crate agones;

use std::result::Result;
use std::thread;
use std::time::Duration;

macro_rules! enclose {
( ($( $x:ident ),*) $y:expr ) => {
{
$(let mut $x = $x.clone();)*
$y
}
};
}

fn main() {
#[tokio::main(flavor = "multi_thread", worker_threads = 4)]
markmandel marked this conversation as resolved.
Show resolved Hide resolved
async fn main() {
println!("Rust Game Server has started!");

::std::process::exit(match run() {
::std::process::exit(match run().await {
Ok(_) => {
println!("Rust Game Server finished.");
0
Expand All @@ -42,57 +30,109 @@ fn main() {
});
}

fn run() -> Result<(), String> {
async fn run() -> Result<(), String> {
println!("Creating SDK instance");
let sdk = agones::Sdk::new().map_err(|_| "Could not connect to the sidecar. Exiting!")?;

let _health = thread::spawn(enclose! {(sdk) move || {
loop {
match sdk.health() {
(s, Ok(_)) => {
println!("Health ping sent");
sdk = s;
},
(s, Err(e)) => {
println!("Health ping failed : {:?}", e);
sdk = s;
let mut sdk = agones::Sdk::new(None /* default port */, None /* keep_alive */)
.await
.map_err(|e| format!("unable to create sdk client: {}", e))?;

// Spawn a task that will send health checks every 2 seconds. If this current
// thread/task panics or dropped, the health check will also be stopped
let _health = {
let health_tx = sdk.health_check();
let (tx, mut rx) = tokio::sync::oneshot::channel::<()>();

tokio::task::spawn(async move {
let mut interval = tokio::time::interval(Duration::from_secs(2));

loop {
tokio::select! {
_ = interval.tick() => {
if health_tx
.send(())
.await.is_err() {
eprintln!("Health check receiver was dropped");
break;
}
}
_ = &mut rx => {
println!("Health check task canceled");
break;
}
}
}
thread::sleep(Duration::from_secs(2));
}
}});
});

tx
};

let _watch = thread::spawn(enclose! {(sdk) move || {
println!("Starting to watch GameServer updates...");
let _ = sdk.watch_gameserver(|gameserver| {
println!("GameServer Update, name: {}", gameserver.object_meta.unwrap().name);
println!("GameServer Update, state: {}", gameserver.status.unwrap().state);
let _watch = {
let mut watch_client = sdk.clone();
let (tx, mut rx) = tokio::sync::oneshot::channel::<()>();

tokio::task::spawn(async move {
println!("Starting to watch GameServer updates...");
match watch_client.watch_gameserver().await {
Err(e) => println!("Failed to watch for GameServer updates: {}", e),
Ok(mut stream) => loop {
tokio::select! {
gs = stream.message() => {
match gs {
Ok(Some(gs)) => {
println!("GameServer Update, name: {}", gs.object_meta.unwrap().name);
println!("GameServer Update, state: {}", gs.status.unwrap().state);
}
Ok(None) => {
println!("Server closed the GameServer watch stream");
break;
}
Err(e) => {
eprintln!("GameServer Update stream encountered an error: {}", e);
}
}

}
_ = &mut rx => {
println!("Shutting down GameServer watch loop");
break;
}
}
},
}
});
}});

tx
};

println!("Setting a label");
sdk.set_label("test-label", "test-value")
.await
.map_err(|e| format!("Could not run SetLabel(): {}. Exiting!", e))?;

println!("Setting an annotation");
sdk.set_annotation("test-annotation", "test value")
.await
.map_err(|e| format!("Could not run SetAnnotation(): {}. Exiting!", e))?;

println!("Marking server as ready...");
sdk.ready()
.await
.map_err(|e| format!("Could not run Ready(): {}. Exiting!", e))?;

println!("...marked Ready");

println!("Setting as Reserved for 5 seconds");
sdk.reserve(Duration::new(5, 0)).map_err(|e| format!("Could not run Reserve(): {}. Exiting!", e))?;
sdk.reserve(Duration::from_secs(5))
.await
.map_err(|e| format!("Could not run Reserve(): {}. Exiting!", e))?;
println!("...Reserved");

thread::sleep(Duration::new(6, 0));
tokio::time::sleep(Duration::from_secs(6)).await;

println!("Getting GameServer details...");
let gameserver = sdk
.get_gameserver()
.await
.map_err(|e| format!("Could not run GameServer(): {}. Exiting!", e))?;

println!("GameServer name: {}", gameserver.object_meta.unwrap().name);
Expand All @@ -101,11 +141,12 @@ fn run() -> Result<(), String> {
let time = i * 10;
println!("Running for {} seconds", time);

thread::sleep(Duration::from_secs(10));
tokio::time::sleep(Duration::from_secs(10)).await;

if i == 5 {
println!("Shutting down after 60 seconds...");
sdk.shutdown()
.await
.map_err(|e| format!("Could not run Shutdown: {}. Exiting!", e))?;
println!("...marked for Shutdown");
}
Expand Down
1 change: 0 additions & 1 deletion sdks/rust/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ Cargo.lock
# These are backup files generated by rustfmt
**/*.rs.bk


# End of https://www.gitignore.io/api/rust
39 changes: 30 additions & 9 deletions sdks/rust/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,35 @@
name = "agones"
version = "0.1.0"
edition = "2018"

[features]
openssl = ["grpcio/openssl"]
openssl-vendored = ["grpcio/openssl-vendored"]
license = "Apache-2.0"
repository = "https://github.com/googleforgames/agones"
documentation = "https://docs.rs/agones"
homepage = "https://agones.dev/site/"
markmandel marked this conversation as resolved.
Show resolved Hide resolved

[dependencies]
grpcio = "0.6.0"
protobuf = "2.16.2"
futures = { version = "0.3", features = ["compat"] }
futures01 = "0.1"
error-chain = "0.12.3"
async-stream = "0.3"
http = "0.2"
prost = "0.7"
thiserror = "1.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This replaces error-chain, which has seen declining usage in the rust ecosystem due to its fairly confusing and heavy approach to errors. That being said, the current error usage is extremely minimal, and we could get rid of thiserror as well and just use a custom error, since it's only used to wrap grpc status messages anyways since the sdk protocol doesn't have its own error types at all.


[dependencies.tokio]
version = "1.0"
default-features = false
features = ["sync", "time"]
Comment on lines +30 to +33
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tonic is built on top of tokio which is (basically) the standard async runtime in the rust ecosystem.


[dependencies.tonic]
version = "0.4"
default-features = false
features = [
"codegen",
"transport",
"prost",
]
Comment on lines +35 to +42
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tonic is a rust native GRPC stack built on top of tokio.


[build-dependencies.tonic-build]
version = "0.4"
default-features = false
features = [
"prost",
"transport",
]
Comment on lines +44 to +50
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The common practice for tonic is to build the protocol buffers at build time, rather than checking the generated code into source, unlike the previous implementation. I can change this if needed.

10 changes: 10 additions & 0 deletions sdks/rust/build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
fn main() {
tonic_build::configure()
// The SDK is just a client, no need to build the server types
.build_server(false)
.compile(
&["proto/sdk/alpha/alpha.proto", "proto/sdk/sdk.proto"],
&["proto/googleapis", "proto/sdk/alpha", "proto/sdk"],
)
.expect("failed to compile protobuffers");
Comment on lines +2 to +9
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generates the Rust code for the protocol and places them in the OUT directory so that they can be included in the Rust code.

}
Loading