Skip to content

Commit

Permalink
Release 1.3 to main (#332)
Browse files Browse the repository at this point in the history
* Thoughts laid out on Models.

* Clarifying Sets.

* Forgot carrot.

* Update models.md

* Adding the object model.

* Polishing object, to allow nulls.

* Comment typo.

* Adding an item and itemset.

* Adding init for models folder.

* Adding some stubs.

* More models v2 being added.

* Running without custom validators.

* Adding views.

* First pass at v2 logic.

* Conditionally adding admin when debug is true.

* Adding magic item type.

* Structural changes to models

* Running.

* Keys.

* Progress on export.

* Dump is now structured correctly.

* symmetric loading and unloading.

* cleanup.

* Adding licenses output, single file.

* Root definitions.

* Document items have a key.

* Renaming and ruleset.

* refactoring to weapon and armor.

* Adding some basics.

* whitespace.

* More refactoring to get rid of Org

* Pulling magic fields into item.

* Adding licenses.

* One document, multiple licenses.

* Removing prefix.

* Adjustments.

* Adding additional data.

* Whitespace, and url.

* Adding filtersets.

* Comment changes.

* Adding appropriate item filters.

* Removing unnecessary props.

* Bad merge

* Adding import and export.

* Making some progress on magicitems.

* Progress

* Adding many more magic items.

* Adding figurines.

* Added the Ioun Stones.

* Adding potions of giant strength.

* Adding spell scrolls and removing old magic items.

* Adjusting quicksetup to add v2 data

* Adding artifact rarity.

* Reordering for a cleaner export.

* Fixing output message.

* Stubbing out Itemset.

* Added trade goods and itemsets.

* Tweaking itemsets.

* Reverse pk doesn't  make any sense.

* Adjusting items to simple/full standard.

* Adding properties and filters

* Tweaking responses.

* Adding in rulesets.

* Adding filters.

* Missed the word Name in fields.

* Missed description as well.

* Adding test router.

* Added poisons.

* Adding adventuring gear.

* Fixing magic and adventuring gear added.

* Adding items and itemset for focuses and others.

* Adding packs.

* Fixing sort order.

* Collapsing migrations.

* Documenting the Data for v2.

* Update README.md

blockquotes -> code blocks in readme

* Adding Kobold Press Vault of Magic.

* Updating key to be long.

* Tweaks.

* Adjusting folder name.

* Readme Improvements (#289)

* docs: simplify installation

* docs: readme logo and styling

* docs: fix centering

* docs: install, build, deploy, ToC

* docs: bash code tags

* docs: expand contributing

* docs: simplify installation

* docs: readme logo and styling

* docs: fix centering

* docs: install, build, deploy, ToC

* docs: bash code tags

* docs: expand contributing

* docs: minor corrections

* cleanup document titles to remove "ogl", standardize on "5e" (#293)

* cleanup document titles to remove "ogl", standardize on "5e"

* add ™ where needed

* Data/284 update adventuring equipment (#294)

* added missing copy: equipment list

* added missing table to adventuring gear section

* fix: update srd madness/traps section to fix table header display (#298)

* Add raw size field to all races (#299)

* models: add raw size to races

* migrations

* parsing fun.

* fixes.

* Removing the partials, consolidating into item.

* Collapsing migrations.

* iterating versions.

* Removing an unecessary package from lock.

* Adding support for /version endpoint.

* Adjustments to use REF and SHA.

* 158 search with empty text string literally returns all results which is expensive (#303)

* Now there is no response to empty query on search.

* Updating env.

---------

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>

* Whitespace.

* Updating pipfile seems to fix.

* Correction of Goblin Specialists CR to 1/2 from 1 (#307)

See A5E MM Page 250 for details.

* Add missing bracket (#306)

* Replace unicode bullets with asterisks

Bulleted lists in monster ability descriptions are now markdown-formatted.

* Create "Misellaneous Creatures" and "NPCs" groups

Assign monsters from "Appendix MM-A: Miscellaneous Creatures" and
"Appendix MM-B: Nonplayer Characters" in the 5e SRD to their
respective groups.

* Refactor view filtersets:
Extract CommonFilterSet base class for all filtersets to extend;
Add document__slug__not_in field filter to CommonFilterSet;
Move filtersets to seperate filters module;

* Bugfix for array properties becoming an empty string when empty (there might be more properties that need the same fix, needs a second look)

* Adding key and reusable method.

* Large changes to support depth.

* More work with Depth.

* Removing unnecessary setting.

* v2 flag

* Registering routers if v2 is on.

* Setting v2 enabled to debug.

* This should work for dev and production.

* Found one more array to default to "null"

* Simplifying how key and url are sent.

* Maintaining depth, but forcing props.

* Created DynamicFieldsHyperlinkedModelSerializer base class

* Inheritance from DFHMS for classes that previously
inherited from both DFMS and HMS, as well as classes that only
inherited from HMS, in order to provide dynamic field serialization
on those classes.

* Revert "Inheritance from DFHMS for classes that previously"

This reverts commit 70a19d7.

* Inheritance from DFHMS for classes that previously
inherited from both DFMS and HMS, as well as classes that only
inherited from HMS, in order to provide dynamic field serialization
on those classes.
Subrace is left out due to a bug in the DFHMS, which will be fixed in the following commit.

* Fix bug in DFMS (indexing with key that may not exist)

* Implement DFHMS on Subrace

* Merge branch 'staging' into api_v2_items

* Just doing an update.

* Updating tests so that #309 is taken into account.

* Update build_and_deploy.yml

* Backfill from Main

* a5e - spells

* a5e - spell classes

* a5e: add spell slugs

* option to import slug from spell json

* a5e: add spell saving throw and shape

* a5e: add attack roll to spells

* Fixed more Hellfire armor issues

* Got through A-Cs. Made some global changes.

* Got through the Es. Fixed some other formatting issues.

* Bone devil had the  wrong cr.

* a5e: use numerical ranges to fit new schema

* spells slug import v2

* a5e: add self as default range

* task: Add pytest as the test runner for the project.

* task: Update README installtion instructions to install pipenv dev dependencies.

* Up through Hs. Ended at Hallowed Effigy.

* Up through the Ns. Ended at Octopus Bracers

* Adding descriptions based on the SRD.

* Up to Rod of Verbatim

* Up to Shark Tooth Crown

* Added MM-B npcs.

* Importing and exposing.

* Adding parameters.

* Editing a weird leading underscore in desc for tob

* Corrected up to Slimeblade..

* Up to Spider Staff.

* Corrected and updated many Vault of Magic items. Did a full skim-through comparing it to the PDF. Oof!

* Fixed JSON error.

* Found a bunch of other fixes to "varies" items.

* Added Barbarian subclasses

classes.json & document.json

* Update classes.json

Added Errata 02/01/2023 changes to Barbarian's "Roar of Defiance"

* Update classes.json

1. Added Bard & Cleric subclasses.
2. Italicized the bold text for player choice options to match the standardized formatting.
3. Italicized spells, the 'x' multiplier symbol, and other words as indicated in the source material.
4. Added Errata 02/01/2023 changes for Bard & Cleric subclasses.

* Update classes.json

1. Fixed a few formatting errors.
2. Added Druid subclasses.
3. Added Errata 02/01/2023 changes for Druid subclasses.

* Update classes.json

1. Added Fighter, Monk, & Paladin subclasses.
2. No Errata 02/01/2023 changes for Fighter, Monk, & Paladin subclasses.

* Update classes.json

1. Added Ranger subclasses

* Removed underscores around spell names.

* fixed wand of vapors

* task: Update the README with better instructions to run the test suite with pytest.

* Update classes.json

1. Added Rogue, Sorcerer, Warlock, and Wizard subclasses.
2. Removed all subclasses not found in Tome of Heroes.
3. Removed class-level fields.
4. Added Errata 02/01/2023 changes for remaining subclasses.

* Update README.md

* Updated classes.json

Updated with cleaner formatting.

* Adding many filters.

* Charclass

* Added the rest of them.

* Cleaning up a few missed items.

* Everything except the markdown.

* maybe all of them in? Errors probably.

* Adjusting ability.

* Removing a couple more dqs.

* Removing some funky escapes.

* Adding lots of newlines, removing links.

* Added in support for just subclass files.

* Create CODE_OF_CONDUCT.md

* Monster environments (#257)

* exposed monster enviroments to the api

* Removed environment duplicates

* storing json, exposing object.

* Exposing a list.

* adding environments to monster test.

---------




* Create races.json

Added races & subraces + Errata 02/01/2023

* Create spells.json

Added all spells + Errata 02/01/2023

* Introduce versioning to DRF API (#248)

* Updating dependencies to current. (#249)

* Create backgrounds.json (#260)

* Create backgrounds.json

Added backgrounds

* Update backgrounds.json

Fixed some formatting issues.

* Create feats.json (#261)

Added feats from Tome of Heroes.

* Create magicitems.json (#262)

Added magic items from Tome of Heroes by Kobold Press

* task: Add basic tests and fixtures for the spells view. (#263)

* task: Add basic tests and fixtures for the spells view.

* task: Use pytest as the test runner on github actions.

* Handling a blank blank_environment_handling (#266)



* some fixes, not all resolved

* removed subrace-only entries

currently the system cannot ingest subraces-only. Removed the gnome, dwarf, and halfling subrace additions. Once we solve this, they could be added in a future version.

* Update spells.json

"Tree Heal" had no class specified. I contacted KP and they clarified it was Druid only.

* Create races.json (#259)

* Create races.json

Added races & subraces + Errata 02/01/2023

* some fixes, not all resolved

* removed subrace-only entries

currently the system cannot ingest subraces-only. Removed the gnome, dwarf, and halfling subrace additions. Once we solve this, they could be added in a future version.

---------



* A few QOL enhancements

Striving for similarity across formatting.

* Api v2 items (#271)

* Thoughts laid out on Models.

* Clarifying Sets.

* Forgot carrot.

* Update models.md

* Adding the object model.

* Polishing object, to allow nulls.

* Comment typo.

* Adding an item and itemset.

* Adding init for models folder.

* Adding some stubs.

* More models v2 being added.

* Running without custom validators.

* Adding views.

* First pass at v2 logic.

* Conditionally adding admin when debug is true.

* Adding magic item type.

* Structural changes to models

* Running.

* Keys.

* Progress on export.

* Dump is now structured correctly.

* symmetric loading and unloading.

* cleanup.

* Adding licenses output, single file.

* Root definitions.

* Document items have a key.

* Renaming and ruleset.

* refactoring to weapon and armor.

* Adding some basics.

* whitespace.

* More refactoring to get rid of Org

* Pulling magic fields into item.

* Adding licenses.

* One document, multiple licenses.

* Removing prefix.

* Adjustments.

* Adding additional data.

* Whitespace, and url.

---------



* Add telemetry and hide v2 apps (#272)

* add newrelic telemetry

* hide v2 paths while in development

* clear API key for test

* try to get it to pick up our license key

* push up correct config for logging

* oops

* trying something crazy

* maybe?

* welp not sure

* Additions for ToH & fixes for WOTC + Open5e_original files (#269)

* Added ToH content + fixes

Fixes:
backgrounds.json

New:
armor.json
spells.json
weapons.json

* Few formatting fixes

open5e_original -> backgrounds.json
WOTC_5e_SRD_v5.1 -> armor.json

* Added adventuringgear.json

This is broken down from sections.json to be more like other files where multiple sources can be appended on one webpage and alphabetized if needed. Original design is a large paragraph in sections.json.

* Update adventuringgear.json

Removed the text "(described later in this section)" because it does not apply for this.

* Monster type normalization (#264)

* Normalized monster types across Kobold Press books

* Normalized monster types across Kobold Press books

* Update data/WOTC_5e_SRD_v5.1/monsters.json

* Update data/WOTC_5e_SRD_v5.1/monsters.json

* Update data/WOTC_5e_SRD_v5.1/monsters.json

* Update data/WOTC_5e_SRD_v5.1/monsters.json

* Update data/WOTC_5e_SRD_v5.1/monsters.json

* Apply suggestions from code review

* Apply suggestions from code review

* Apply remaining "swarm"s

---------



* add serializer methods to avoid errors on json-based fields (#275)

* add serializer methods to avoid errors on json-based fields

* fix copy/paste error

---------

Co-authored-by: Sturlen <43607012+Sturlen@users.noreply.github.com>
Co-authored-by: mshea <mike@mikeshea.net>
Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
Co-authored-by: guizesilva <31866341+guizesilva@users.noreply.github.com>
Co-authored-by: Jonathan Lawrence <jonathanlawrence813@outlook.com>
Co-authored-by: August Johnson <augustjohnson@users.noreply.github.com>
Co-authored-by: Tim Basten <tbasten@gmail.com>
Co-authored-by: David Prothero <david@prothero.com>
Co-authored-by: Jonathan L <47407031+JonathanELawrence@users.noreply.github.com>

---------

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
Co-authored-by: August Johnson <augustjohnson@users.noreply.github.com>
Co-authored-by: Sturle Spetland <43607012+Sturlen@users.noreply.github.com>
Co-authored-by: calum <47755775+calumbell@users.noreply.github.com>
Co-authored-by: A. Stadler <adamv.stadler@gmail.com>
Co-authored-by: Namtrah Kcirt <13903735+namtrah@users.noreply.github.com>
Co-authored-by: jakdevmail <140275384+jakdevmail@users.noreply.github.com>
Co-authored-by: Michael Groufsky <m.groufsky@gmail.com>
Co-authored-by: Gresh <>
Co-authored-by: denizaydin <deniznaydin@gmail.com>
Co-authored-by: mshea <mike@mikeshea.net>
Co-authored-by: guizesilva <31866341+guizesilva@users.noreply.github.com>
Co-authored-by: Jonathan Lawrence <jonathanlawrence813@outlook.com>
Co-authored-by: Tim Basten <tbasten@gmail.com>
Co-authored-by: David Prothero <david@prothero.com>
Co-authored-by: Jonathan L <47407031+JonathanELawrence@users.noreply.github.com>
  • Loading branch information
16 people authored Oct 2, 2023
1 parent 3ce9b38 commit 5836ad5
Show file tree
Hide file tree
Showing 63 changed files with 36,360 additions and 2,047 deletions.
5 changes: 4 additions & 1 deletion .github/workflows/build_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set Docker image tag
run: |
run: |
echo "GITHUB_REF = $GITHUB_REF" > version.py
echo "GITHUB_SHA = $GITHUB_SHA" >> version.py
if [[ $GITHUB_REF == refs/heads/main ]]; then
echo "DOCKER_IMAGE_TAG=latest" >> $GITHUB_ENV
elif [[ $GITHUB_REF == refs/heads/staging ]]; then
Expand Down
226 changes: 109 additions & 117 deletions Pipfile.lock

Large diffs are not rendered by default.

181 changes: 132 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,99 +1,182 @@
![API status](https://img.shields.io/website?down_message=Down&label=Open5e%20API&up_message=Up&url=https%3A%2F%2Fapi.open5e.com)
<p align="center">
<img src="logo.png" width="200px" align="center" alt="Open5e logo" />
<h1 align="center">Open5e API</h1>
<p align="center">
<a href="https://open5e.com">https://open5e.com</a>
<br/>
A JSON API for the D&D 5e ruleset
</p>
</p>
<br />

<p align="center">
<a href="https://api.open5e.com" rel="nofollow"><img src="https://img.shields.io/website?down_message=Down&label=Open5e%20API&up_message=Up&url=https%3A%2F%2Fapi.open5e.com" alt="API"></a>
<a href="https://open5e.com" rel="nofollow"><img src="https://img.shields.io/website?down_message=Down&label=Open5e&up_message=Up&url=https%3A%2F%2Fopen5e.com" alt="homepage"></a>
</p>

<div align="center">
<a href="https://api.open5e.com">API</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://discord.gg/9RNE2rY">Discord</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://www.patreon.com/open5e">Patreon</a>
</div>

<br/>

# Table of contents

- [Table of contents](#table-of-contents)
- [Introduction](#introduction)
- [Installation](#installation)
* [Requirements](#requirements)
* [Modules](#modules)
- [Development](#development)
* [Build](#build)
+ [Search Indexing](#search-indexing)
* [Run](#run)
* [Building the OAS file](#building-the-oas-file)
- [Contributing](#contributing)
* [Editing existing sources](#editing-existing-sources)
* [Adding a new sorce](#adding-a-new-sorce)
* [Change existing models](#change-existing-models)
- [Tests](#tests)
- [Deployment](#deployment)
* [DigitalOcean](#digitalocean)
* [Railway.app](#railwayapp)
* [Docker](#docker)

<small><i><a href='http://ecotrust-canada.github.io/markdown-toc/'>Table of contents generated with markdown-toc</a></i></small>

# Introduction

Open5e is a community project driven by a small number of volunteers in their spare time. We welcome any and all contributions! Please join our Discord to help out: https://discord.gg/9RNE2rY or check out the issue board if you'd like to see what's being worked on!

The Django API uses Django REST Framework for its browsability and ease of use when developing CRUD endpoints. It uses django's default SQLite database, and pulls the data from the /data directory.
The API uses the Django REST Framework for it's browsability and ease of use when developing CRUD endpoints. It uses django's default SQLite database, and pulls the data from the `/data` directory.

# Install Prerequisites
# Installation

1. Install the sqlite3 development package. On Ubuntu, the package is called
`sqlite3-devel`. On Debian-based systems, it's called `libsqlite3-dev`.
## Requirements

1. This project currently uses python3.8 configured with loadable sqlite
extensions. If you don't have python3.8, or if you aren't sure that your
python3.8 installation has loadable sqlite extensions enabled, download and
install the python3.8 source
[here](https://www.python.org/downloads/release/python-3816/). Installation
instructions are in the README found in the source tarball. When you
configure it, be sure to to use
`./configure --enable-loadable-sqlite-extensions`.
- [Python 3.11](https://www.python.org/downloads/)

1. We use pipenv to manage our Python dependencies. Installation instructions
are on the [pipenv website](https://pipenv.readthedocs.io/en/latest/).
- [Pipenv](https://pipenv.pypa.io/en/latest/installation/)

1. Once pipenv is installed, you can install all of the project dependencies
defined in the Pipfile via `pipenv install --dev`.
## Modules

## Quick Setup
Pipenv is used to install all required packages from the `Pipfile` at the project root. Use the following command after cloning the project or switching branches.

To do any python development on the django application itself, I would suggest using django's built-in server as it allows for various things (such as debug mode and quick reloads). Here's the general process for getting that up and running.
```bash
pipenv install --dev
```

If you want to work with existing data sources and just get working you can quickly stand up the server with
# Development

> pipenv run python manage.py quicksetup
## Build

followed by
Crate a local database and import game content.
```bash
pipenv run python manage.py quicksetup --noindex
```

> pipenv run python manage.py runserver
To make sure the API is always using your updated code, this command must be run again if:
- You add/remove/edit Game Content
- You edit Python code
- You switch git branches

This will stand up the server with full content and search index at http://localhost:8000.

## Manual Setup Steps
### Search Indexing

If you want to customize your setup, particularly useful if adding new content sources, then you will need to use the built-in django migration function to define your database, making sure to run it within the pipenv environment.
> pipenv run python manage.py migrate
To use the search function, you must build the search index by running the above command without the `--noindex` flag.
```bash
pipenv run python manage.py quicksetup
```

You will then need to collect the static files (this makes django-resk-framework look presentable when viewing it in html).
> pipenv run python manage.py collectstatic --noinput

Finally, you will need to load the SRD data from the json files in the /data folder. This is using the custom populatedb command.
> pipenv run python manage.py populatedb --flush ./data/WOTC_5e_SRD_v5.1/
## Run

At that point, you will be able to run the django server normally (within the pipenv environment).
> pipenv run python manage.py runserver
Run the server locally. This server is only for development and shall __not__ be used in production. The server will be available at `http://localhost:8000`.

And your server should be available at http://localhost:8000.
```bash
pipenv run python manage.py runserver
```

## Tests
If you need to run the server on another port, add the port number as an argument.

### To run the test suite:
```bash
pipenv run python manage.py runserver $PORT
```

First, install the prerequisites as described above

Then, install dev requirements:
## Building the OAS file

```
pipenv install --dev
After completing a build, you can generate an OAS file to be used by another application.
```bash
pipenv run ./manage.py generateschema --generator_class api.schema_generator.Open5eSchemaGenerator > openapi-schema.yml` to build the OAS file.
```

Then, run the test suite:
```
# Contributing

Before making any changes, you should fork the Ope5e-api repository. This will make a copy on your account, which can be freely edited. Once your edits are done you can open a Pull Request to have your changes reviewed by a maintainer, which may ask for changes or clarification before approving it. Once merged the changes go live on [Beta Site](https://beta.open5e.com) before being pushed live.

Smaller edits such as spelling mistakes can be edited directly in Github. For larger edits, it is recommeded that you make changes in a full editor, such as [VS Code](https://code.visualstudio.com) with the [Github Extenstion](https://code.visualstudio.com/docs/sourcecontrol/github).

## Editing existing sources

Game Content is stored in the `data` directory. It is first split according to which document/source books it originated from and further into JSON files split by category e.g. "monsters.json", "spells.json". These can be edited directly. You can also add new categories to existing sources by creating the required JSON file. See an existing source, such as the 5.1 SRD to see how these should be structured.

## Adding a new sorce

To add a new source, create new directory inside `data` and a `document.json` file that credits the source and links to the license it was published under. An example of this can be found [here](/data/a5e_srd/document.json). You can then add a json file for each category of content. See an existing source, such as the 5.1 SRD to see how these should be structured.

To load this new source, it must be added to the `SOURCE_DIRS` in [quickload.py](/api/management/commands/quickload.py). Rebuild the project to see the new Game Content.
## Change existing models

Models such as Monsters and Classes are stored in the [api/models](/api/models) directory. These define fields (hp, str, speed) and how they are output. The import of Game Content from `data` is handled by an [ImportSpec](/api/management/commands/importer.py)

# Tests

Tests are located in the `api/tests` directory. These should be run before pushing new changes to the main repository.
```bash
pipenv run pytest
```

## Starting up a droplet
# Deployment

The API is normally deployed via [Docker](https://docs.docker.com/get-started/). You can either build and host it yourself, or use one of the tested providers below:


## DigitalOcean

This deployment has been tested using DigitalOcean Apps with Docker Hub.
This deployment has been tested using [DigitalOcean Apps](https://www.digitalocean.com/go/cloud-hosting) with Docker Hub.

To start up the server from scratch on a droplet:

```
```bash
git pull https://github.com/open5e/open5e-api
export SECRET_KEY=a_new_secret_key
export SERVER_NAME=whatever.yourdomain.com
cd open5e-api/
docker-compose up
```

## Deploying on Railway.app
1. Create a fork on github.com This is used to automatically deploy when you make a change.

## Railway.app
1. Create a fork on Github. This is used to automatically deploy whenever you make a change.
2. Login with your Github account on [Railway.app](https://railway.app) and give it access to manage your forked repository.
3. Create a new Project and choose 'Deploy from GitHub repo'. Select your fork in the list.
4. Keep all settings default and deploy. Accept when Railway asks to copy variables existing variables from the repository.
5. Add the variable `PORT` with the value `8888`.
6. Add the variable `SERVER_NAME` with the [Railway-provided domain](https://docs.railway.app/deploy/exposing-your-app#railway-provided-domain) or add your own.
7. Push a commit to Github and watch your open5e-api redeploy in minutes!

## Building the OAS file

Once you have everything set up, run `pipenv run ./manage.py generateschema --generator_class api.schema_generator.Open5eSchemaGenerator > openapi-schema.yml` to build the OAS file.
## Docker

With docker installed, you can build the project with provided Dockerfile

```bash
docker build
```

This docker app can then be deployed with any provider.
Loading

0 comments on commit 5836ad5

Please sign in to comment.