spend time thinking about strategy for this project

[stevekrouse]

• pick date to end season 1
• pick date to begin season 2
• think about better way to organize things than time-series journal
• continue working on /about and /thesis

[stevekrouse]

Podcast seasons

Clearly, anything I decide here will be arbitrary. There doesn't seem to be any clear demarcation point between seasons for what I do. I guess whenever I want to take more than two weeks off, we'll call that the end of season 1. But that isn't for a little while, potentially that will be thanksgiving and/or new years.

Website organization

The benefits of the time-series /journal structure:

1. Very easy to add thoughts every day.
2. Easy for people who want to follow my journey

The problem:

1. Hard to find things when you're looking for them.
2. Ideas are scattered, centered on when they were thought, not what they were thought about

The solution: wouldn't it be great if I could have my cake and eat it to? I think the solution to this problem is commit messages:

1. The commit log can be a journal of sorts as long as I create better commit messages and potentially add commit descriptions.
2. For updates that don't fit into commit messages, I can git commit --allow-empty for just the message.

Moving away from /journal to /log

Thus in the future, I will no longer add content to /journal (RIP). However, I will keep the page as it is so as to not break the many links I've made to it (an annoying reminder that web content is location-identified instead of identified by a hash of contents). I will add a message in bold at the top that explains that the page has been deprecated and link visitors to /log, which initially could just redirect to https://github.com/stevekrouse/futureofcoding.org/commits/master but eventually I could pull the data myself (using my clay service https://www.clay.run/services/steve/github-project-data/code) to make it prettier.

/ideas

Going forward, when I have various ideas for more work, I will work on those in the /ideas/ folder. The existing /ideas page will be renamed index.md inside that folder. This page will have two sections.

1. The top of the page will explain the purpose of the ideas folder and link to all of the ideas in the folder. For now, I can create that list manually but it'd be nice if it could be generated automatically on the client side by pulling that data from github eventually (although this would make it harder to add high-level descriptions for each idea).

2. The bottom of the page will be called "Archived" and explain the structure of the old /links.md file and how this section is so as to not break the old links (I'm realizing that I really need a more salable structure like IPFS if I both want to be able to 1) never break old links and 2) be able to iterate on the structure of this site, let's make an issue for this... better platform #53)

Each idea will have it's own markdown page to start. If it becomes bigger, it can graduate to it's own folder where it'd markdown file is renamed to index.md inside it.

/links

Similar problems exist with my /links page. Will think more on those on #47.

/about and master plan

Will think more about how to attack this problem on #32

/thesis

And this one on #9

[stevekrouse]

Plan of attack

I got 99 issues and too many issues is one

Ok, ok, ok! I have a lot of thoughts here but I feel like they're now scattered around various issues. I wan to consolidate my thoughts and issues into a small, key number of issues so I can figure out my plan of attack:

First things first, build a structure for scale

Before I start iterating on the structure for this site, it probably makes sense to build myself a structure that I can iterate on indefinitely in case I don't like this new structure. It took me some thinking but I think I came up with a way for this structure to work in the "Git commit hashes" section of this comment.

Cases

There are a few different cases I need to handle.

a. The incoming URL is futureofcoding.org so requesting the index.html page with no commit hash. In this case, I merely have to return the current version of index.html but then I must insert the current commit hash into the url bar so that it now reads: futureofcoding.org/as33fd

b. The incoming URL has no commit hash and is requesting a page other than index.html and this page exists in the current version of the project. In this case, I do the same as above. I return the current page and insert the current commit hash into the URL, such as futureofcoding.org/as33fd/episodes/13... (In reality, case (a) above is really a sub-case of this case (b) but it feels better to make it its own case because we know that index.html will always exist while every other path may not.)

c. Same as above -- the incoming URL has no commit hash and is requesting a page other than index.html -- but this page does not exist in the current version of the project. In this case, github pages will redirect to 404.html where will will handle this error. The ideal way to handle this error would be to find the most recent version of the project that includes this file and redirect to that version of the file (while explaining that this file no longer exists in the project and showing the commit message that deleting the file which possibly will explain why it was deleted and where the content was moved). However that might be difficult because it would involve searching through potentially hundreds of versions of the project (which of course we could optimize with a binary search or something). It makes me wonder if we could optimize this by creating a data structure that persists somewhere on the internet of a mapping of every file path every created in this directory to the most recent hash in which that files exists. That way, this problem would be a simple lookup! However, in the meanwhile, we can simply error and say that the file you are looking for no longer exists in the project but you can look for it in the commit messages if you want to do the digging yourself and link to the commit log...

d. Same as above -- the incoming URL has no commit hash and is requesting a page other than index.html -- but while the page does exist, the project links to an anchor tag on the page that does not exist on that page in the current version of the project. The short term solution could be doing nothing in this case. The medium term solution is saying "the anchor tag you are looking for does not exist in the current version of this page. You may be able to find it by looking at previous versions of this page here." The long term solution is to extend the mapping discussed above to also map headings to the most recent commit that they exist in that file. Quite the project!

e. An incomming request does have a commit hash. In this case we return the version of the file at that commit hash. It would be great if we could find out the most recent version of the file and if it's not the same, notify the user that a more recent version exists and link to it.

f. An incomming request does have a commit hash and also has a link to an anchor tag. In this case, we could look for the most recent version of the page that also contains this anchor tag which we'd really need this content to hash mapping for.

Solutions

There are a few things I need to build:

1. Mapping from content to hash

As discussed above, a mapping from each path ever created in this directory to the value of the most recent hash in which it exists in the project is key. The obvious way to do this is that on each commit, I could get a post-commit hook on github where I go through and update this structure for each path that still exists, while leaving the paths that do not alone. However, it would be neat if I could figure out a way to run this in such a way that it's not so brittle. That is, if I could run it from scratch every time as a pure function as opposed to have this global mutable state.

1b. Mapping from anchor tag to commit hash

An added bonus would be if we could also map the anchor tags in each file as well to the most recent hash in which they exist in each file.

2. The rewrite code on 404.html

TODO

3. The rewrite code on every other page

TODO

4. The notification code that tells you about more recent versions

TODO

5. (potentially) the history page for any file that doesn't link to github

Like /log.md (which I haven't yet built), it could be a better interface than github history because it could show the full commit message, not just the summary (although I never use the main commit message part so I should text what github shows if I do...)

Fully deprecate /journal

Once this system is in place, I imagine it won't be terribly hard to deprecate the journal fully and move to /log as discussed above. Let's play it out to see how well the system above works to handle this use case.

1. Create /log. The first version of this could be merely redirecting to https://github.com/stevekrouse/futureofcoding.org/commits/master. However we need to see what happens to longer commit messages in this interface. Either way, we'll probably want to build our own interface to this data.

1a. Old journal entries

TODO

1b. Linkability of commit messages

TODO and the linkability of their anchor tags?

1c. Formatting of commit messages

TODO markdown? their own page? link to the github commit?

2. Delete /journal from the repo. In the commit that we delete the journal, we explain why it was deleted and link to /log. The new system should take all links to /journal and redirect them to the commit that it last existed in. It should also have a notice that contains the commit explaining why it was deleted right on the page.

Master plan, /about and /thesis

Will need to think on all this, how it relates to Streamsheets #54 and Reactive WoofJS #55. The thinking for this will mostly take place on #32 but will also relate to #9.

/links

This problem is also tricky but I can't really focus on it until the other problems above are address. Will be thought about on #47.

[stevekrouse]

Github issues don't keep history, so I'm going to continue this on a new issue. Apologies for the duplication...

---

Plan of attack (continued on 10/6/17)

I got 99 issues and too many issues is one

Ok, ok, ok! I have a lot of thoughts here but I feel like they're now scattered around various issues. I wan to consolidate my thoughts and issues into a small, key number of issues so I can figure out my plan of attack:

First things first, build a structure for scale

Before I start iterating on the structure for this site, it probably makes sense to build myself a structure that I can iterate on indefinitely in case I don't like this new structure. It took me some thinking but I think I came up with a way for this structure to work in the "Git commit hashes" section of this comment.

Cases

There are a few different cases I need to handle.

a. The incoming URL is futureofcoding.org so requesting the index.html page with no commit hash. In this case, I merely have to return the current version of index.html but then I must insert the current commit hash into the url bar so that it now reads: futureofcoding.org#as33fd

b. The incoming URL has no commit hash and is requesting a page other than index.html and this page exists in the current version of the project. In this case, I do the same as above. I return the current page and insert the current commit hash into the URL, such as futureofcoding.org/episodes/13...#as33fd (In reality, case (a) above is really a sub-case of this case (b) but it feels better to make it its own case because we know that index.html will always exist while every other path may not.)

c. Same as above -- the incoming URL has no commit hash and is requesting a page other than index.html -- but this page does not exist in the current version of the project. In this case, github pages will redirect to 404.html where will will handle this error. The ideal way to handle this error would be to find the most recent version of the project that includes this file and redirect to that version of the file (while explaining that this file no longer exists in the project and showing the commit message that deleting the file which possibly will explain why it was deleted and where the content was moved). However that might be difficult because it would involve searching through potentially hundreds of versions of the project (which of course we could optimize with a binary search or something). It makes me wonder if we could optimize this by creating a data structure that persists somewhere on the internet of a mapping of every file path every created in this directory to the most recent hash in which that files exists. That way, this problem would be a simple lookup! However, in the meanwhile, we can simply error and say that the file you are looking for no longer exists in the project but you can look for it in the commit messages if you want to do the digging yourself and link to the commit log...

d. Same as above -- the incoming URL has no commit hash and is requesting a page other than index.html -- but while the page does exist, the project links to an anchor tag on the page that does not exist on that page in the current version of the project. The short term solution could be doing nothing in this case. The medium term solution is saying "the anchor tag you are looking for does not exist in the current version of this page. You may be able to find it by looking at previous versions of this page here." The long term solution is to extend the mapping discussed above to also map headings to the most recent commit that they exist in that file. Quite the project!

e. An incomming request does have a commit hash. In this case we return the version of the file at that commit hash. It would be great if we could find out the most recent version of the file and if it's not the same, notify the user that a more recent version exists and link to it.

f. An incomming request does have a commit hash and also has a link to an anchor tag. In this case, we could look for the most recent version of the page that also contains this anchor tag which we'd really need this content to hash mapping for.

Solutions

There are a few things I need to build:

1. Mapping from content to hash

As discussed above, a mapping from each path ever created in this directory to the value of the most recent hash in which it exists in the project is key. The obvious way to do this is that on each commit, I could get a post-commit hook on github where I go through and update this structure for each path that still exists, while leaving the paths that do not alone. However, it would be neat if I could figure out a way to run this in such a way that it's not so brittle. That is, if I could run it from scratch every time as a pure function as opposed to have this global mutable state.

1b. Mapping from anchor tag to commit hash

An added bonus would be if we could also map the anchor tags in each file as well to the most recent hash in which they exist in each file.

2. The rewrite code on 404.html

3. The rewrite code on every other page

4. The notification code that tells you about more recent versions

5. (potentially) the history page for any file that doesn't link to github

Like /log.md (which I haven't yet built), it could be a better interface than github history because it could show the full commit message, not just the summary (although I never use the main commit message part so I should text what github shows if I do...)

Fully deprecate /journal

Once this system is in place, I imagine it won't be terribly hard to deprecate the journal fully and move to /log as discussed above. Let's play it out to see how well the system above works to handle this use case.

1. Create /log. The first version of this could be merely redirecting to https://github.com/stevekrouse/futureofcoding.org/commits/master. However we need to see what happens to longer commit messages in this interface. Either way, we'll probably want to build our own interface to this data.

1a. Old journal entries

There are probably a lot of commit messages that aren't so well polished but I guess they do deserve a place in the log. We can also include old journal entries in the log because that's really how we're thinking about this transition. Whatever code generates /log can dynamically pull old journal entries in according to their timestamp.

1b. Formatting of commit messages

It's currently not so pretty. We can eventually improve this with our own markdown renderer...

2. Delete /journal from the repo. In the commit that we delete the journal, we explain why it was deleted and link to /log. The new system should take all links to /journal and redirect them to the commit that it last existed in. It should also have a notice that contains the commit explaining why it was deleted right on the page.

Master plan, /about and /thesis

Will need to think on all this, how it relates to Streamsheets #54 and Reactive WoofJS #55. The thinking for this will mostly take place on #32 but will also relate to #9.

/links

This problem is also tricky but I can't really focus on it until the other problems above are address. Will be thought about on #47.

Let's do it!

Do first things really need to come first?

At the end of the day, do I really need to build a structure for scale before I change this website? If I don't things will break in the short term (or I could just put a notice on /journal and not delete it). But in the long term, I can always come back and fix the broken links. The real key is now that I know things can be saved forever, I don't have to worry as much about changing things -- however I can only really change things with impunity after this system is in place.

What I'm trying to say is: is building this system currently my highest priority? The answer is clearly no. My highest priority is continuing to articulate my goal, developing my strategy to achieving that goal, getting feedback on the goal and strategy, while also developing my current thesis within my strategy. The website organization should all be subservient to that.

Let's now go work on my master plan #32 and thesis #9!

I think I'm going to close this issue in deference to #53