Deployed at https://esk8bst.netlify.com
A small webapp that scans the monthly reddit /r/ElectricSkateboarding Buy / Sell / Trade thread and displays high signal posts in a more digestable format. Allows for sorting and filtering on a number of fields.
Future updates are likely to include stats on the number of boards available, as well as email notifications if a board you want comes up for sale.
The site is all in-browser and doesn't require a back-end server. Reddit's API is used to collect thread information when you load the page.
This is a project built with TypeScript 3 and React. Shout out to Microsoft's TypeScript-React Starter for making it easy to get started with TypeScript.
npm install
npm run start
- client/
- models/
- react/
- components/
- pages/
- router/
- services/
- types/
Most of the magic happens in the Services, specifically Services/FindThread.ts
and Services/GetNewestData.ts
, which contain the fetcher and the parser.
Google's Cloud Firestore is the datastore. It is structured thusly:
- Databases/
- Scan/
- Matches/
- Preconfirmed/
The scan collection contains a single document with id ScanData
structured like:
{
LastScanDate: Timestamp
}
The matches object contains one collection for each user, where the id is their email. Each document contains an array of Match objects, each describing one potential search query the user wants to search for over the Buy Sell Trade thread:
{
matches: [
{
bst: string,
currency: string,
price: number,
companies: [
string
]
}
]
}
Although both the Datastore and the Lambda's support an arbitrary number of matches, the front end presently only supports one match per user.
Stores a sha256 of each email that has previously opted-in so that additional subscription requests in the futrue, for example, to edit their match object, aren't locked behind another opt-in email.
Each object is simply of the format:
sha256(email)
{
exists: true
}
where the document id is the hash and the only field in the document is an exists field set to true. They are non-empty to avoid Firebase automatically pruning them.
This project uses AWS Lambda to host 4 functions:
- Subscribe
- Confirm Subscribe
- Unsubscribe
- Scan
Email is quite tricky, finnicky, and dependant upon, among a hundred other things, a senders 'reputation'. One way for a sender to main good reputation is to support double opt-in. In our architecture, a user requests to be subscribed by hitting the Subscribe
endpoint, the first Lambda sends a confirmation email to the user, the user clicks a link that email which pings the Confirm Subscribe
endpoint, and the second lambda registers their email.
Unsubscribe
is another lambda function, which starts when a user hits the unsubscribe endpoint. This removed their email from Firestore.
And finally, the Scan
lambda does the hard work of fetching subscribers, fetching the reddit thread, parsing into machine readable formats, determining if any user had any matching new posts, and sending out daily emails. This Lambda is only triggerable by a scheduled cronjob from AWS every hour. It is not available via any HTTP endpoint.
Email is handled with the Mailgun API.
- Tracking-Protection Enabled browsers will refuse to load the reddit json.
- Posts for items designated in Canadian Dollars can parse weirdly at times
- Known Boards / Companies is static and hard to update, meaning obvious brands like OneWheel are bucketed into the
?
, also known as theother / unknown
category. - Small screens have trouble displaying the Filter Zone
- Some links run out of their post box if they are long enough