🌊 River Charts 📈📉

River Charts is a Python, Django, Plotly, and Pandas web application that visualizes river data for a specific river/site/location.

The line graph is driven by data pulled using an API from the United States Geological Survey (USGS).
- The data is captured by the USGS using a gage height sensor every 15 minutes.
The data is updated with the most recent river height data every time the application is loaded.
- Please be patient, the loading time of the application is around 30+ seconds (depending on your internet connection), due to the large API JSON response and the amount of data being processed from the USGS.
- The application can be configured to pull live data in via the API or use a static .json file to display a snapshot of data.
The loading screen screen contains fun wave and tube graphics as well as 20 "Fun Facts" regarding Tubing and Rivers on a 7 second interval to keep the user occupied while the data is loading in the background.
I recommend using the application on a desktop since the chart is interactive has a wider view, but it can be used on a mobile device as well.
Visit the application here.

Features

Responsive: The application is responsive and can be used on a mobile device.
Interactive: The application uses Plotly to create an interactive line graph.
Data Visualization: The application visualizes river data for a specific river/site/location.
Data Source: The application sources data using an API that returns JSON output.
Data Processing: The application processes the JSON output and converts it to a Pandas DataFrame.
Data Manipulation: The application manipulates the DataFrame to create a Plotly line graph.
Data Update: The application updates the data with the most recent river height data every time the application is loaded.
Data Capture: The data is captured by the USGS using a gage height sensor every 15 minutes.

Background Story

Every year, my friends and I float 2 miles down the Susquehanna River in NEPA on river tubes (a 2 hour float). I wanted to create a web application that would allow us to visualize past river data in order to see the river height on the days we floated down the river. Some float dates, we still got together, but we didn't float due to the dangerous river levels.

Below is the key to use when viewing the plots on the graph for Float dates:

Color	Hex	Floated Status	Floated Key
Green	`#008000`	Yes (Met up and did Float)	Y
Red	`#FF0000`	No (Met up, but did not Float)	N
Orange	`#FFA500`	Skipped (Skipped the event altogether)	S

Note

The closest river gage to our float location is the Susquehanna River at Meshoppen, PA, which is the default site code in the application. You can change the site code in the application config.py file to visualize data for a different river/site code.

West Branch, Susquehanna River.

The application uses gage height data to plot the river height in feet on a given date.

Definitions

Here are some definitions to help you understand the terminology used in this document:

USGS: The United States Geological Survey. The USGS is a science organization that provides impartial information on the health of our ecosystems and environment, the natural hazards that threaten us, the natural resources we rely on, the impacts of climate and land-use change, and the core science systems that help us provide timely, relevant, and useable information.
- USGS
Gage Height: The height of the water surface above the gage datum (zero point). Gage height is often used interchangeably with the more general term, stage, although gage height is more appropriate when used with a gage reading. Stage is more appropriate when used with a recorded or calculated gage height.
- Gage Height
- Why does the USGS use the spelling "gage" instead of "gauge"?

Getting Started

Dependencies

This project makes use of several libraries and frameworks:

Python: For the application logic.
Django: For web application functionality.
Plotly: For creating interactive visualizations.
Pandas: For data manipulation and analysis.
Requests: For making API calls.
Python-Decouple: For storing sensitive information in a .env file.
See requirements.txt for a full list of dependencies.

Configuration

Edit config.py to add your own USGS API (and other) information.
- USGS API
- USGS API Documentation
Toggle USE_DUMMY_DATA to True in config.py to use dummy data instead of the API.
- This is useful for testing the application without making API calls.
Toggle USE_SNAPSHOT_DATA to True in config.py to use snapshot/static data instead of the API.
- This is useful for avoiding API calls and using a static data set in the data/snapshot.json file.
- You will need to populate the file with JSON by calling the "Historical River Levels" API call in Postman, included in this project (docs/api/River Charts.postman_collection.json); see Calling the API below for more information.
- Your app will esentially be a snapshot, and not live updates using this method, but it can be helpful if you want to avoid server costs making API calls.
The float data plots are driven from a .csv file located in static/data/river_charts.csv.
- This file can be edited to add/remove float dates.
- The file is read in views.py and passed to the template as a context variable.

Installation

To install and run the project locally, follow the steps below:

Clone the repository:

git clone https://github.com/scottgriv/River-Charts

Cloning a repository

Navigate to the project directory:
```
cd [YOUR PROJECT DIRECTORY]
```
Create a virtual environment:
```
python -m venv venv
```
- Creating Virtual Environments
Activate the virtual environment:
```
source venv/bin/activate
```
- Activating a virtual environment
Install the required packages using requirements.txt:
```
pip install -r requirements.txt
```
- Installing Packages
Note: If you wany to generate a new requirements.txt file, run the following command:
```
pip freeze > requirements.txt
```
- Requirements Files
Run the Django server:
```
python manage.py runserver
```
- Running the Django server

Now, you can visit http://127.0.0.1:8000/ in your browser to access the application.

Deployment

The application is hosted here on PythonAnywhere.
The application is deployed using a WSGI configuration file.
- WSGI Configuration
First, make sure you adjust your settings.py file ALLOWED_HOSTS to include your deployment host.
- Deploying Django
Second, make sure you adjust your settings.py file DEBUG to False for production.
- Django Settings
Finnaly, be sure to create a .env file where you host your application to store your sensitive information (excluded from this repository).
- Python-Decouple

What's Inside?

Below is a list of the main files and folders in this repository and their specific purposes:

River-Charts # Root folder
├─ .github # GitHub related files such as CHANGELOG, CREDITS, etc.
├─ docs # Included resources.
│   ├─ api # API resources.
│   │   └─ River Charts.postman_collection.json # A Postman Collection used to manually call the API, and to gather data if you want to use this as a static site.
│   ├─ assets # Misc. resources.
│   │   └─ stage versus flow slide.pdf # A PDF file that explains stage vs. flow slide.
│   └─ images # Image resources.
├─ River_Charts # The Django project directory.
│   ├─ __init__.py # An empty file that tells Python that this directory should be considered a Python package.
│   ├─ asgi.py # An entry-point for ASGI-compatible web servers to serve your project.
│   ├─ settings.py # Settings/configuration for this Django project.
│   ├─ urls.py # The URL declarations for this Django project.
│   └─ wsgi.py # An entry-point for WSGI-compatible web servers to serve your project.
├─ rivercharts # A directory for the rivercharts app.
│   ├─ templates # A directory for HTML templates.
│   │    ├─ rivercharts # A directory for HTML templates specific to the river_charts app.
│   │    ├─ error.html # An HTML template that displays an error message.
│   │    └─ index.html # An HTML template that displays the application.
│   ├─ __init__.py # An empty file that tells Python that this directory should be considered a Python package.
│   ├─ admin.py # A file that registers models to be displayed in the Django admin site.
│   ├─ apps.py # A file that contains the application configuration.
│   ├─ config.py # A file that contains sensitive information (excluded from this repository).
│   ├─ models.py # A file that contains the database models.
│   ├─ tests.py # A file that contains the tests for the application.
│   ├─ urls.py # A file that contains the URL declarations for the application.
│   └─ views.py # A file that contains the application logic.
├─ static # A directory for static files that are used in this Django project.
│   ├─ assets # A directory images used throughout the application.
│   ├─ css # A directory for CSS files.
│   │   └─ styles.css # A CSS file that contains the styles for the application.
│   ├─ data # A directory for data files.
│   │   ├─ fun_facts.json # A JSON file used to display Fun Facts while the application is loading, add or remove more facts here.
│   │   ├─ snapshot.json # A JSON file used for static data when USE_SNAPSHOT_DATA is set to True in config.py.
│   │   └─ river_charts.csv # A CSV file that contains the float dates for the application.
│   ├─ fonts # A directory fonts used throughout the application.
│   └─ js # A directory for JavaScript files.
│       └─ script.js # The JavaScript file used to render the loading screen. 
├─ requirements.txt # A list of Python packages required to run this project.
├─ db.sqllite3 # A database file Django uses for this project (Do not delete).
├─ manage.py # A command-line utility that lets you interact with this Django project in various ways.
├─ .github # GitHub folder
├─ .gitignore # Git ignore file
├─ .gitattributes # Git attributes file
├─ PRG.md # PRG Connection File
├─ LICENSE # A file that contains the license for this project.
├─ VERSION # A file used to keep the repository release and the PythonAnywhere deployment in sync.
└─ README.md # This file.

API Documentation

Below is the documentation for the API used in this application.

Calling the API

To call the API and retrieve the data:

Make a GET request to: http://nwis.waterservices.usgs.gov/nwis/... (based on your requirements).
- ex. https://waterservices.usgs.gov/nwis/iv?format=json&sites=01533400&startDT=2015-07-01&endDT=2023-08-16&parameterCd=00065&siteStatus=active&siteType=ST

Pass the necessary parameters in the request.

ex.

params = {
    "format": "json", # Set your interchange format.
    "sites": "01533400", # Site Code: Susquehanna River at Meshoppen, PA.
    "startDT": "2015-07-01", # Set the date you want to start collecting data from.
    "endDT": "2023-09-14", # This is based on the current date in the application.
    "parameterCd": "00065", # Parameter Code: Gage height, ft.
    "siteStatus": "active", # Selects sites based on whether or not they are currently active. Each USGS Water Science Center determines whether a site is active or inactive. The default is all (show both active and inactive sites).
    "siteType": "ST", # ST = A body of running water moving under gravity flow in a defined channel. The channel may be entirely natural, or altered by engineering practices through straightening, dredging, and (or) lining. An entirely artificial channel should be qualified with the "canal" or "ditch" secondary site type.
}

Process the JSON response as demonstrated in the example above.

Notes:

The application uses the API to source data for the graph. If the API is down, the graph will not render.
Errors are handled in the application by redirecting the user to an error page with the appropriate error message.
- HTTP errors are handled in the application by redirecting the user to an error page with the appropriate error message.
- The application will also display an error message if the API returns an empty response or a timeout error (default is set to 90 seconds in config.py).
- The API is rate limited to 30 calls per minute. If you exceed this limit, you will receive a 429 error.

Example of an Error Page.

API Output Example

The application sources data using an API that returns JSON output. Here's an example of what the API response looks like:

{
    "name": "ns1:timeSeriesResponseType",
    ...
    "timeSeries": [
        {
            "sourceInfo": {
                "siteName": "Susquehanna River at Meshoppen, PA",
                ...
            },
            "variable": {
                "variableName": "Gage height, ft",
                ...
            },
            "values": [
                {
                    "value": [
                        {
                            "value": "15.13",
                            "dateTime": "2015-07-01T00:00:00.000-04:00"
                        },
                        ...
                    ]
                }
            ]
        }
    ]
}

(For the sake of brevity, the full output is abbreviated with ...)

Disclaimer

The data provided by this application is sourced from the USGS.
It's subject to revision, and for more information, please refer to their official disclaimer.
Software is provided as-is and no warranty is given about its usability.

Closing

Thank you for taking the time to read through this document and I hope you find it useful! If you have any questions or suggestions, please feel free to reach out to me.

Please reference the SUPPORT file in this repository for more details.

What's Next?

I'm looking forward to seeing how this project evolves over time and how it can help others with their GitHub Portfolio.

Please reference the CHANGELOG file in this repository for more details.

Project

Please reference the GitHub Project tab inside this repository to get a good understanding of where I'm currently at with the overall project.

Issues and Enhancements will also be tracked there as well.

Contributing

Feel free to submit a pull request if you find any issues or have any suggestions on how to improve this project. You can also open an issue with the tag "bug" or "enhancement".

How to contribute:

Fork the Project
Create your Feature Branch (git checkout -b feature/River-Charts)
Commit your Changes (git commit -m 'Add new feature')
Push to the Branch (git push origin feature/River-Charts)
Open a Pull Request

Please reference the CONTRIBUTING file in this repository for more details.

Resources

Below are some external resources I found helpful when creating River Charts:

Technical Resources

Python - A programming language that lets you work quickly and integrate systems more effectively.
Django - A high-level Python Web framework that encourages rapid development and clean, pragmatic design.
Plotly - A Python graphing library that makes interactive, publication-quality graphs online.
Pandas - A fast, powerful, flexible and easy to use open source data analysis and manipulation tool.
Requests - A simple, yet elegant HTTP library.
Python-Decouple - A Python library for separating the settings of your Django/Flask/FastAPI project from the source code.
Deploying Django - A tutorial on how to deploy Django.
Django Settings - A list of all settings available in Django.
Running the Django server - A tutorial on how to run the Django server.
Creating Virtual Environments - A tutorial on how to create virtual environments.
Activating a virtual environment - A tutorial on how to activate a virtual environment.
Installing Packages - A tutorial on how to install packages using pip.
Requirements Files - A file containing a list of items to be installed using pip install like so: pip install -r requirements.txt.
PythonAnywhere - A Python hosting provider with a free tier.

Note

To use external API's (like this application does), PythonAnywhere requires a basic paid tier.

Educational Resources

USGS - The United States Geological Survey.
USGS API - The USGS Instantaneous Values Web Service.
USGS API Documentation - The USGS Instantaneous Values Web Service Documentation.
USGS Site Web Service - The USGS Site Web Service.
Codes and Parameters - The USGS Codes and Parameters Documentation.
How to Get Real-time Flood Data from the USGS API - A tutorial on how to get real-time flood data from the USGS API.
USGS Client Library - Client library for interfacing with USGS datasets (GitHub repository).
USGS Client Library Repository - Client library for interfacing with USGS datasets.
USGS API Homepage - The USGS API Homepage.
USGS Rest API web services - The USGS Rest API web services.
USGS Instantaneous Water Service data URL generator - The USGS Instantaneous Water Service data URL generator.
National Water Dashboard - The National Water Dashboard.
Rivers vs. Streams vs. Creeks (USGS) - A USGS article on the difference between rivers, streams, and creeks.
How to Make Colored Country Maps with GeoPandas and Python - A tutorial on how to make colored country maps with GeoPandas and Python.
5 Ways to Use Free Data on the Internet for Science - A tutorial on how to use free data on the internet for science.
GitHub for Biologists - A tutorial on how to use GitHub for biologists.
Why does the USGS use the spelling "gage" instead of "gauge"? - Fun nerd fact.

License

This project is released under the terms of the GNU General Public License, version 3 (GNU GPLv3), which ensures that derivatives of the software remain open source.

The GNU GPLv3 is a "copyleft" license, ensuring that derivatives of the software remain open source and under the GPL.
For more details and to understand all requirements and conditions, see the LICENSE file in this repository.

Credit

Author: Scott Grivner
Email: scott.grivner@gmail.com
Website: scottgrivner.dev
Reference: Main Branch

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

🌊 River Charts 📈📉

Table of Contents

Features

Background Story

Definitions

Getting Started

Dependencies

Configuration

Installation

Deployment

What's Inside?

API Documentation

Calling the API

API Output Example

Disclaimer

Closing

What's Next?

Project

Contributing

Resources

Technical Resources

Educational Resources

License

Credit

About

Releases 10

Packages

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 108 Commits
.github		.github
River_Charts		River_Charts
docs		docs
rivercharts		rivercharts
static		static
.gitattributes		.gitattributes
.gitignore		.gitignore
LICENSE		LICENSE
PRG.md		PRG.md
README.md		README.md
VERSION		VERSION
db.sqlite3		db.sqlite3
manage.py		manage.py
requirements.txt		requirements.txt

License

scottgriv/River-Charts

Folders and files

Latest commit

History

Repository files navigation

🌊 River Charts 📈📉

Table of Contents

Features

Background Story

Definitions

Getting Started

Dependencies

Configuration

Installation

Deployment

What's Inside?

API Documentation

Calling the API

API Output Example

Disclaimer

Closing

What's Next?

Project

Contributing

Resources

Technical Resources

Educational Resources

License

Credit

About

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases 10

Packages 0

Languages

Packages