Skip to content

Commit

Permalink
Rename everything (#263)
Browse files Browse the repository at this point in the history
* session -> beacon: tavern tests pass

* session -> beacon: rust & UI, all tests pass

* job -> quest

* Missed a few, it's good now

* Removed accidentally copied port scanner
  • Loading branch information
KCarretto authored and hulto committed Oct 5, 2023
1 parent 942876f commit f122e76
Show file tree
Hide file tree
Showing 157 changed files with 13,334 additions and 13,233 deletions.
22 changes: 15 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,43 +4,49 @@
</div>

# Realm

![test-status](https://github.com/kcarretto/realm/actions/workflows/tests.yml/badge.svg?branch=main)
[![codecov](https://codecov.io/github/kcarretto/realm/branch/main/graph/badge.svg?token=KSRPHYDIE4)](https://app.codecov.io/github/kcarretto/realm)
[![Go Report Card](https://goreportcard.com/badge/github.com/kcarretto/realm)](https://goreportcard.com/report/github.com/kcarretto/realm)
[![Rust Report Card](https://rust-reportcard.xuri.me/badge/github.com/kcarretto/realm)](https://rust-reportcard.xuri.me/report/github.com/kcarretto/realm)
[![Docs](https://img.shields.io/badge/read%20our-docs-informational)](https://docs.realm.pub/)

Realm is a cross platform Red Team engagement platform with a focus on automation and reliability.

Realm is a cross platform Red Team engagement platform with a focus on automation and reliability.

![realm-logo](./docs/assets/img/realm_create_job.png)
![realm-logo](./docs/assets/img/realm_create_quest.png)

## Features

### Agent (imix)

- Written in rust with support for MacOS, Linux, and Windows.
- Supports long running jobs by reading output from jobs in real time.
- Supports long running tasks by reading output from tasks in real time.
- Interval callback times.
- Simple file based configuration.
- Embedded files.
- Built-in interpreter.

### Server (tavern)

- Web interface.
- Group actions.
- graphql backend for easy API access.
- OAuth login support.
- Cloud native deployment with pre-made terraform for production deployments.


### Built-in interpreter (eldritch)

- Reflective DLL Loader.
- Port scanning.
- Remote execution over SSH.
- And much much more: https://docs.realm.pub/user-guide/eldritch

## Quickstart guide

*To deploy a production ready instance see the [tavern setup guide](https://docs.realm.pub/user-guide/tavern).*

### Start the server

```bash
git clone https://github.com/KCarretto/realm.git
cd realm
Expand All @@ -49,7 +55,9 @@ go run ./tavern
# If you'd like to test without deploying an agent use the test data.
ENABLE_TEST_DATA=1 go run ./tavern
```

### Start the agent

```bash
git clone https://github.com/KCarretto/realm.git
cd realm/implants/imix
Expand Down Expand Up @@ -78,6 +86,6 @@ cargo run -- -c /tmp/imix-config.json

```


## Want to contribute start here
https://docs.realm.pub/dev-guide/introduction

<https://docs.realm.pub/dev-guide/introduction>
12 changes: 6 additions & 6 deletions docs/_docs/dev-guide/eldritch.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Eldritch
tags:
tags:
- Dev Guide
description: Want to implement new functionality in the agent? Start here!
permalink: dev-guide/eldritch
Expand Down Expand Up @@ -219,7 +219,7 @@ struct OsInfo {
}

fn handle_get_os() -> Result<OsInfo> {
return Ok(OsInfo {
return Ok(OsInfo {
arch: whoami::arch().to_string(),
});
}
Expand Down Expand Up @@ -262,12 +262,12 @@ PR #[238](https://github.com/KCarretto/realm/pull/238/files) This PR implements
# Notes about asynchronous Eldritch code
---
### Async example
In order to run concurrent tasks we need to build an asynchronous function. This is useful if you're building a function that needs to do two things at once or that can benefit from running discrete jobs in parallel.
In order to run concurrent tasks we need to build an asynchronous function. This is useful if you're building a function that needs to do two things at once or that can benefit from running discrete tasks in parallel.

The starlark bindings we're using to create Eldritch are not asynchronous therefore the Eldritch function itself cannot be asynchronous.
To get around this we use the [`tokio::runtime::Runtime.block_on()`](https://docs.rs/tokio/latest/tokio/runtime/struct.Runtime.html#method.block_on) function in conjunction with two asynchronous helpers.

We'll create the following three functions to manage concurrent tasks:
We'll create the following three functions to manage concurrent tasks:
* `pub fn function` - Eldritch function implementation which will implement the `tokio::runtime::Runtime.block_on()` function.
* `async fn handle_function` - Async helper function that will start, track, and join async tasks.
* `async fn run_function` - Async function runner that gets spawned by the `handle_function` function.
Expand All @@ -290,7 +290,7 @@ async fn handle_function(arg1: Vec<String>) -> Result<Vec<String>> {
all_result_futures.push(task::spawn(resulting_future));
}

// Await results of each job.
// Await results of each task.
// We are not acting on scan results independently so it's okay to loop through each and only return when all have finished.
for task in all_result_futures {
match task.await? {
Expand Down Expand Up @@ -323,7 +323,7 @@ pub fn function(arg1: Vec<String>) -> Result<Vec<String>> {
```

**Implementation tips:**
* If running a lot of concurrent tasks the system may run out of open file descriptors. Either handle this error with a wait and retry, or proactively rate limit your tasks well below the default limits.
* If running a lot of concurrent tasks the system may run out of open file descriptors. Either handle this error with a wait and retry, or proactively rate limit your tasks well below the default limits.


### Testing async code requires some additional work
Expand Down
41 changes: 24 additions & 17 deletions docs/_docs/dev-guide/tavern.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,10 +131,12 @@ If you'd like to explore the Graph API and try out some queries, head to the `/g
![/assets/img/tavern/graphiql.png](/assets/img/tavern/graphiql.png)

#### Some sample queries to get started
**List all sessions**

##### List all beacons

```graphql
query get_sessions {
sessions {
query get_beacons {
beacons {
id
identifier
name
Expand All @@ -145,35 +147,39 @@ query get_sessions {
}
```

**Create a tome**
##### Create a tome

```graphql
mutation CreateTome ($input: CreateTomeInput!) {
createTome(input: $input) {
id
name
}
}
}
```

```json
{
"input": {
"name": "Test tome",
"description": "Just a sample",
"eldritch": "print(input_params['print_string'])",
"fileIDs": [],
"paramDefs": "{\"print_string\":\"str\"}"
"paramDefs": "[{\"name\":\"print_string\",\"label\":\"Print String\",\"type\":\"string\",\"placeholder\":\"A message to print\"}]"
}
}
```

**Create a job**
##### Create a task

```graphql
mutation createJob($input: CreateJobInput!, $sess:[ID!]!){
createJob(input: $input, sessionIDs: $sess) {
mutation createQuest($input: CreateQuestInput!, $beaconIDs:[ID!]!){
createQuest(input: $input, beaconIDs: $beaconIDs) {
id
}
}
```

```json
{
"input": {
Expand All @@ -184,14 +190,16 @@ mutation createJob($input: CreateJobInput!, $sess:[ID!]!){
"sess": ["8589934593"]
}
```
**Get all task and job output**

##### Get all task and quest output

```graphql
query get_task_res {
jobs {
quests {
tasks {
id
output
job {
quest {
tome {
eldritch
}
Expand All @@ -203,7 +211,6 @@ query get_task_res {
}
```


### Creating a New Model

1. Initialize the schema `cd tavern && go run entgo.io/ent/cmd/ent init <NAME>`
Expand Down Expand Up @@ -241,7 +248,7 @@ query get_task_res {
|-----|-----------|--------|
|state| SQL queries that define the initial db state before the query is run.| no |
|requestor| Holds information about the authenticated context making the query. | no |
|requestor.session_token| Session token corresponding to the user for authentication. You may create a user with a predetermined session token using the `state` field. | no |
|requestor.beacon_token| Session token corresponding to the user for authentication. You may create a user with a predetermined session token using the `state` field. | no |
|query| GraphQL query or mutation to be executed | yes |
|variables| A map of variables that will be passed with your GraphQL Query to the server | no |
|expected| A map that defines the expected response that the server should return | no |
Expand Down Expand Up @@ -278,7 +285,7 @@ GraphQL mutations enable clients to _mutate_ or modify backend data. Tavern supp
"hostname": "test",
"hostIdentifier": "dodo",
"agentIdentifier": "bleep",
"sessionIdentifier": "123"
"beaconIdentifier": "123"
}
},
"operationName": "ClaimTasks"
Expand All @@ -289,13 +296,13 @@ In the above example, `$input` is used to pass variables from code to the GraphQ

### Claiming Tasks

The first GraphQL mutation an agent should utilize is `claimTasks`. This mutation is used to fetch new tasks from Tavern that should be executed by the agent. In order to fetch execution information, the agent should perform a graph traversal to obtain information about the associated job. For example:
The first GraphQL mutation an agent should utilize is `claimTasks`. This mutation is used to fetch new tasks from Tavern that should be executed by the agent. In order to fetch execution information, the agent should perform a graph traversal to obtain information about the associated quest. For example:

```graphql
mutation ClaimTasks($input: ClaimTasksInput!) {
claimTasks(input: $input) {
id
job {
quest {
tome {
id
eldritch
Expand Down
6 changes: 3 additions & 3 deletions docs/_docs/user-guide/imix.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Imix
tags:
tags:
- User Guide
description: Imix User Guide
permalink: user-guide/imix
Expand Down Expand Up @@ -49,10 +49,10 @@ The imix config is as follows:
- `priority`: The index that a domain should have.
- `uri`: The full URI of the callback endpoint.

## Functionality
## Functionality
Imix derives all it's functionality from the eldritch language.
See the [Eldritch User Guide](/user-guide/eldritch) for more information.

## Task management
Imix can execute up to 127 threads concurrently after that the main imix thread will block behind other threads.
Every callback interval imix will query each active thread for new output and rely that back to the c2. This means even long running jobs will report their status as new data comes in.
Every callback interval imix will query each active thread for new output and rely that back to the c2. This means even long running tasks will report their status as new data comes in.
File renamed without changes
10 changes: 5 additions & 5 deletions docs/pages/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ permalink: /
# Welcome to Realm!

## User Guide
Realm is a cross platform Red Team engagement platform with a focus on automation and reliability.
Realm is a cross platform Red Team engagement platform with a focus on automation and reliability.

Want to get hands on now? <a href="{{ '/user-guide/getting-started' | prepend: site.baseurl }}">Follow this guide to setup a dev instance of realm.</a>

![realm-logo](/assets/img/realm_create_job.png)
![realm-logo](/assets/img/realm_create_quest.png)

<h5><a href="{{ '/user-guide/tavern' | prepend: site.baseurl }}">Server (tavern)</a></h5>
<ul>
Expand All @@ -25,7 +25,7 @@ Want to get hands on now? <a href="{{ '/user-guide/getting-started' | prepend:
<h5><a href="{{ '/user-guide/imix' | prepend: site.baseurl }}">Agent (imix)</a></h5>
<ul>
<li>Written in rust with support for MacOS, Linux, and Windows.</li>
<li>Supports long running jobs by reading output from jobs in real time.</li>
<li>Supports long running tasks by reading output from tasks in real time.</li>
<li>Interval callback times.</li>
<li>Simple file based configuration.</li>
<li>Embedded files.</li>
Expand All @@ -50,12 +50,12 @@ Want to get hands on now? <a href="{{ '/user-guide/getting-started' | prepend:

<div class="section-index">
<hr class="panel-line">
{% for post in site.docs %}
{% for post in site.docs %}
<!-- Skip unrelated pages -->
{% if post.url contains 'dev-guide' %}
{% else %}
{% continue %}
{% endif %}
{% endif %}
<div class="entry">
<h5><a href="{{ post.url | prepend: site.baseurl }}">{{ post.title }}</a></h5>
<p>{{ post.description }}</p>
Expand Down
6 changes: 3 additions & 3 deletions docs/pages/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ permalink: /user-guide/
# User Guide

## User Guide
Realm is a cross platform Red Team engagement platform with a focus on automation and reliability.
Realm is a cross platform Red Team engagement platform with a focus on automation and reliability.

Want to get hands on now? <a href="{{ '/user-guide/getting-started' | prepend: site.baseurl }}">Follow this guide to setup a dev instance of realm.</a>

![realm-logo](/assets/img/realm_create_job.png)
![realm-logo](/assets/img/realm_create_quest.png)

<h5><a href="{{ '/user-guide/tavern' | prepend: site.baseurl }}">Server (tavern)</a></h5>
<ul>
Expand All @@ -25,7 +25,7 @@ Want to get hands on now? <a href="{{ '/user-guide/getting-started' | prepend:
<h5><a href="{{ '/user-guide/imix' | prepend: site.baseurl }}">Agent (imix)</a></h5>
<ul>
<li>Written in rust with support for MacOS, Linux, and Windows.</li>
<li>Supports long running jobs by reading output from jobs in real time.</li>
<li>Supports long running tasks by reading output from task in real time.</li>
<li>Interval callback times.</li>
<li>Simple file based configuration.</li>
<li>Embedded files.</li>
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -88,12 +88,14 @@ github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/urfave/cli v1.22.5 h1:lNq9sAHXK2qfdI8W+GRItjCEkI+2oR4d+MEHy1CKXoU=
github.com/urfave/cli v1.22.5/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/urfave/cli/v2 v2.24.4 h1:0gyJJEBYtCV87zI/x2nZCPyDxD51K6xM8SkwjHFCNEU=
github.com/vektah/gqlparser/v2 v2.5.1 h1:ZGu+bquAY23jsxDRcYpWjttRZrUz07LbiY77gUOHcr4=
github.com/vektah/gqlparser/v2 v2.5.1/go.mod h1:mPgqFBu/woKTVYWyNk8cO3kh4S/f4aRFZrvOnp3hmCs=
github.com/vmihailenco/msgpack/v5 v5.3.5 h1:5gO0H1iULLWGhs2H5tbAHIZTV8/cYafcFOr9znI5mJU=
github.com/vmihailenco/msgpack/v5 v5.3.5/go.mod h1:7xyJ9e+0+9SaZT0Wt1RGleJXzli6Q/V5KbhBonMG9jc=
github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 h1:bAn7/zixMGCfxrRTfdpNzjtPYqr8smhKouy9mxVdGPU=
github.com/zclconf/go-cty v1.13.0 h1:It5dfKTTZHe9aeppbNOda3mN7Ag7sg6QkBNm6TkyFa0=
github.com/zclconf/go-cty v1.13.0/go.mod h1:YKQzy/7pZ7iq2jNFzy5go57xdxdWoLLpaEp4u238AE0=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
Expand Down
Loading

0 comments on commit f122e76

Please sign in to comment.