Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resources #1

Open
dariusk opened this issue Oct 25, 2015 · 48 comments
Open

Resources #1

dariusk opened this issue Oct 25, 2015 · 48 comments
Labels

Comments

@dariusk
Copy link
Owner

dariusk commented Oct 25, 2015

This is an open issue where you can comment and add resources that might come in handy for NaNoGenMo.

There are already a ton of resources on the old resources threads for the 2013 edition and the 2014 edition.

@javierarce
Copy link

Maybe someone finds my collection of libraries and APIs useful: https://github.com/javierarce/toolbox

@cpressey
Copy link

After last year's NaNoGenMo I said I would try to extract useful things from the NaNoGenLab and package them in a more useful way, and, so, yes, this is what I extracted:

  • T-Rext cleans up spacing and punctuation in a text file.
  • Guten-gutter attempts to strip Project Gutenberg boilerplate from a text file (and succeeds more often than gutenizer does)
  • seedbank makes any Python script record and be able to replay the random seeds it chooses, with a one-line change. Useful for improving reproducibility.

They all require Python 2.x (tested on Python 2.7.6 but will probably still work on some earlier versions) but only seedbank requires that you write your script in Python - the other two can be used as stand-alone tools.

They're all in the public domain, as is all the stuff in the NaNoGenLab. So, please steal, fold, spindle, and mutilate as you see fit.

@MichaelPaulukonis
Copy link

Conceptually, you might want to stumble around The Living Handbook of Narratology - for some overviews of narrative theory, and some alternate takes on what might constitute narrative.

@ikarth
Copy link

ikarth commented Oct 26, 2015

ProcJam is more visually oriented, but it does have some useful resources that come out of it, like Tracery, a JavaScript library for generating stories from expanding grammars.

@MichaelPaulukonis
Copy link

Tracey was recently mentioned in Generative Text - the mailing list/google-group that sprouted from the end of NNGM2014. Lately, there's been a lot of partisan ranting about templates. I may resemble some of that remark.

@enkiv2
Copy link

enkiv2 commented Oct 26, 2015

I'd like to point out that the BotAlly slack group has a NaNoGenMo room,
and that the GenArt slack group that sprouted from the Generative Text
mailing list is mostly full of NaNoGenMo2014 people.

On Mon, Oct 26, 2015 at 11:40 AM Michael Paulukonis <
notifications@github.com> wrote:

Tracey was recently mentioned in Generative Text
https://groups.google.com/forum/#!forum/generativetext - the mailing
list/google-group that sprouted from the end of NNGM2014. Lately, there's
been a lot of partisan ranting about templates. I may resemble some of that
remark.


Reply to this email directly or view it on GitHub
#1 (comment)
.

@ikarth
Copy link

ikarth commented Oct 26, 2015

Since we're speaking of visual novels, I should probably mention the Yarn dialog editor. I don't know that'll be all that useful directly for generative stuff, but in case someone is trying to procedurally generate a dialog tree, it might be handy. Or you could always generate a Twine file.

@ikarth
Copy link

ikarth commented Oct 26, 2015

@enkiv2 Where's the signup for the GenArt group? I need to check that out.

@enkiv2
Copy link

enkiv2 commented Oct 26, 2015

The genart group is at https://generativeart.slack.com ; I think it might
be invite-only. Mike Paulukonis controls the group & can probably send
invites.

On Mon, Oct 26, 2015 at 12:15 PM Isaac Karth notifications@github.com
wrote:

@enkiv2 https://github.com/enkiv2 Where's the signup for the GenArt
group? I need to check that out.


Reply to this email directly or view it on GitHub
#1 (comment)
.

@enkiv2
Copy link

enkiv2 commented Oct 26, 2015

As for botally, the public signup is here: http://t.co/OiOB7197BH ; people
periodically check for registrations and send invites out.

On Mon, Oct 26, 2015 at 12:18 PM John Ohno john.ohno@gmail.com wrote:

The genart group is at https://generativeart.slack.com ; I think it
might be invite-only. Mike Paulukonis controls the group & can probably
send invites.

On Mon, Oct 26, 2015 at 12:15 PM Isaac Karth notifications@github.com
wrote:

@enkiv2 https://github.com/enkiv2 Where's the signup for the GenArt
group? I need to check that out.


Reply to this email directly or view it on GitHub
#1 (comment)
.

@dariusk
Copy link
Owner Author

dariusk commented Oct 26, 2015 via email

@MichaelPaulukonis
Copy link

Yeah, there's not a lot going on in https://generativeart.slack.com - and if there's a way to allow people to request an invite, I'm all ears. @dariusk - what's the source for the #botALLY registration doc+script?


UPDATE: he's the GenerativeArt invite page: http://tinyurl.com/genartslackrequest

@ikarth
Copy link

ikarth commented Oct 26, 2015

@MichaelPaulukonis I've seen people use a Google Form to manage signups for Slack groups.

@enkiv2
Copy link

enkiv2 commented Oct 26, 2015

So far as I can tell, the signup just is a web form that sends an email to
the slack maintainer. Maybe google forms has this functionality by itself?

On Mon, Oct 26, 2015 at 12:29 PM Isaac Karth notifications@github.com
wrote:

@MichaelPaulukonis https://github.com/MichaelPaulukonis I've seen
people use a Google Form to manage signups for Slack groups.


Reply to this email directly or view it on GitHub
#1 (comment)
.

@dariusk
Copy link
Owner Author

dariusk commented Oct 26, 2015 via email

@ikarth
Copy link

ikarth commented Oct 26, 2015

There's been a lot of work done with Neural Networks this year, including text-based output (such as the generated Magic: the Gathering cards). That might be an interesting avenue to pursue.

Abulafia uses MediaWiki for its generators. Not directly useful, but I keep forgetting to mention it here.

Here's a method of procedurally generating a wilderness in a text adventure format. People who are doing a Swallows type simulation approach may find its ideas useful.

@vijithassar
Copy link

Multiverse JSON is a light JSON syntax/spec and Python compiler script with which to store editorial projects in small logical units so they can be quickly reconfigured according to build parameters you define.

@aparrish
Copy link

Since last year's NaNoGenMo I've made a few libraries that might be of interest:

I also made Context-Free GenGen, which is sort of like a mini-Tracery except driven by Google Sheets (a la the original GenGen). Probably not great for making a whole novel, but maybe cool for prototyping ideas with context-free grammar generation.

@mgiraldo
Copy link

@enkiv2
Copy link

enkiv2 commented Oct 26, 2015

I have a few tools I've made in the past year too:

Some novelty filters:

And, some novelty generators:

On Mon, Oct 26, 2015 at 2:21 PM Allison Parrish notifications@github.com
wrote:

Since last year's NaNoGenMo I've made a few libraries that might be of
interest:

I also made Context-Free GenGen http://cfgg.decontextualize.com/, which
is sort of like a mini-Tracery except driven by Google Sheets (a la the
original GenGen http://tinysubversions.com/gengen/). Probably not great
for making a whole novel, but maybe cool for prototyping ideas with
context-free grammar generation.


Reply to this email directly or view it on GitHub
#1 (comment)
.

@dariusk dariusk added the admin label Oct 26, 2015
@ikarth
Copy link

ikarth commented Oct 27, 2015

If you're looking for a large corpus of internet comments, the mostly complete Reddit corpus is available.

@coleww
Copy link

coleww commented Oct 27, 2015

i have been pretty heavily tooling up for nanogenmo this year by way of publishing a ton of poetic node modules to npm.

Another resource i have created is the weirdly-specific-corpora project, which is a fork of dariusk/corpora for any lists that are too weirdly specific to be included in that project. I expect I will end up making a lot of weird lists while generating novels.

@dariusk
Copy link
Owner Author

dariusk commented Oct 27, 2015

Someone (I believe @JKirchartz) posted this link to an actual spam blog comment generator template that was accidentally posted as a comment on a guy's blog.

@enkiv2
Copy link

enkiv2 commented Oct 27, 2015

There is a generator for hilariously poorly written sex scenes
http://www.fiftyshadesgenerator.com/. The source is embedded in the page
there.

On Tue, Oct 27, 2015 at 4:14 PM Darius Kazemi notifications@github.com
wrote:

Someone (I believe @JKirchartz https://github.com/jkirchartz) posted
this link to an actual spam blog comment generator template
http://alexking.org/blog/2013/12/22/spam-comment-generator-script that
was accidentally posted as a comment on a guy's blog.


Reply to this email directly or view it on GitHub
#1 (comment)
.

@MichaelPaulukonis
Copy link

See sample code in various languages via the Language Survey, issue #17

@ikarth
Copy link

ikarth commented Oct 30, 2015

@enkiv2 suggested that a short introduction to some historical techniques might be useful in the resources thread to point out possible approaches for people who might not be familiar with some of the precedents of text generation. So I thought I'd write about a few of them.

Dada incorporated several techniques that involved randomly assembling prior texts into new poetry. The cut-up technique takes an existing text, cuts it to pieces, and then reassembles the bits in a new order. This list of Surrealist techniques might also be inspirational.

Oulipo is a group of writers who use constraints to define their writing. Relevant examples include Queneau's Cent Mille Milliards de Poèmes (A Hundred Thousand Billion Poems), which combines ten-line sonnets line-by-line; Perec's Life a User's Manual, which is structured like a knight's tour of a chessboard; Queneau's Excercises in Style, 99 retellings of the same story; Calvino's The Castle of Crossed Destinies, stories interperted via Tarot cards; and Queneau's "A Story as You Like It".

Even where they aren't directly generative, the approaches used by Oulipo writers often point to alternative forms a novel can take. For example, Calvino's Invisible Cities and If on a winter's night a traveler.

Oulipo is also significant because many of its members wrote about systems for generating stories and about incorporating computers into writing. Italio Calvino's essay "Prose and Anticombinatorics," about using the computer to to find the constraints for a murder mystery; Paul Fournel's "Computer and Writer: The Centre Pompidou Experiment"; and Claude Berge's "For a Potential Analysis of Combinatory Literature" are particularly relevant.

Lastly, Jorges Luis Borges tends to crop up a lot, particularly for his short stories: "The Library of Babel", "The Book of Sand", "An Examination of the Work of Herbert Quain", "The Garden of Forking Paths", "Pierre Menard, Author of the Quixote", and "Tlön, Uqbar, Orbis Tertius".

@ikarth
Copy link

ikarth commented Oct 31, 2015

After @rbechtel mentioned the source code for Micro-Talespin, I went poking around to see what other early AI storytelling systems had source code availible.

Micro-Talespin in Common Lisp, by Warren Sack
another source
[About Talespin](http://lispm.de/mts https://grandtextauto.soe.ucsc.edu/2006/09/13/the-story-of-meehans-tale-spin/)

For Tale-Spin itself: In 2008 Meehan found a copy of the original MLisp source code for Tale-Spin (sans the data files). There's an extensive discussion of the source code in Wardrip-Fruin's Expressive Processing, though I'm not aware of any copies online.

Eliza/Doctor:
In Java
Also in Java
BASIC
JavaScript
Python
The original Lisp source code- github repo

Skald: a Scala reimplementation of Minstrel Remixed, which is itself a Scala reimplementation of Scott Turner's Minstrel (originally in Lisp). Scott Turner on MINSTREL.

@enkiv2
Copy link

enkiv2 commented Oct 31, 2015

As an example of exercises in minimalism and apophenia, Nick Monfort has a
collection of one kilobyte story generators in python.
https://grandtextauto.soe.ucsc.edu/2008/11/30/three-1k-story-generators/

On Sat, Oct 31, 2015 at 11:20 AM Isaac Karth notifications@github.com
wrote:

After @rbechtel mentioned the source code for Micro-Talespin
#59, I went poking
around to see what other early AI storytelling systems had source code
availible.

Micro-Talespin in Common Lisp, by Warren Sack
http://lispm.de/source/misc/micro-talespin.lisp
another source http://eliterature.org/images/microtalespin.txt
About Talespin
http://lispm.de/mts%20https://grandtextauto.soe.ucsc.edu/2006/09/13/the-story-of-meehans-tale-spin/

For Tale-Spin itself
https://grandtextauto.soe.ucsc.edu/2006/09/13/the-story-of-meehans-tale-spin/:
In 2008 Meehan found a copy of the original MLisp source code for Tale-Spin
(sans the data files). There's an extensive discussion of the source code
in Wardrip-Fruin's Expressive Processing, though I'm not aware of any
copies online.

Eliza/Doctor: https://en.wikipedia.org/wiki/ELIZA
In Java
https://code.google.com/p/simple-semantic-desktop/source/browse/trunk/Progs2/Eliza/eliza.java?r=4
Also in Java http://www.chayden.net/eliza/Eliza.html
BASIC https://www.jesperjuul.net/eliza/ELIZA.BAS
JavaScript https://www.jesperjuul.net/eliza/
Python http://www.jezuk.co.uk/cgi-bin/view/software/eliza
The original Lisp source code http://elizagen.org/\- github repo
https://github.com/jeffshrager/elizagen

Skald https://sites.google.com/a/soe.ucsc.edu/eis-skald/: a Scala
reimplementation of Minstrel Remixed, which is itself a Scala
reimplementation of Scott Turner's Minstrel (originally in Lisp). Scott
Turner on MINSTREL
https://grandtextauto.soe.ucsc.edu/2007/10/30/scott-turner-on-minstrel/.


Reply to this email directly or view it on GitHub
#1 (comment)
.

@MichaelPaulukonis
Copy link

@ikarth et alia - this year I have been looking at Tale-Spin a bit, even though I don't think I will be using it as a model for what I will be doing. However, I do have some more resources in my NMGM repo and will be adding more there as the month progresses.

@jemisa
Copy link

jemisa commented Nov 8, 2015

neural-storyteller is a recurrent neural network for generating little stories about images. Code is at https://github.com/ryankiros/neural-storyteller
and samples https://medium.com/@samim/generating-stories-about-images-d163ba41e4ed

@ikarth
Copy link

ikarth commented Nov 8, 2015

Linked Data has links to a large number of datasets, many of which are also Open Data.
Datahub lists a large number of datasetes, and can be filtered by license, if you're only interested in public domain or Creative Commons datasets.
Pleiades is a gazetter of ancient geographic places.

@ikarth
Copy link

ikarth commented Nov 10, 2015

Baba - a Javascript-based designer for text generators. A generator that outputs text generators should be a useful resource for non-coders, but it looks like the live web demo is down so it'll take a little bit of work to get it to run if you download it.

@ikarth
Copy link

ikarth commented Nov 11, 2015

I've created a thread for beginner's questions and tutorials, as a supplement to the main Resources thread: #152

@ikarth
Copy link

ikarth commented Nov 28, 2015

Working in Python? Having unicode issues? Even if you properly take care of the proper encoding and decoding on your end, your source data may be incorrectly encoded. Or, worse, Windows might have mucked it up. If you've got “ or — all over your text, you might want to consider using ftfy. Fixes bad unicode and mojibake; doesn't convert non-unicode to unicode.

@hugovk
Copy link
Collaborator

hugovk commented Nov 29, 2015

If you want to avoid certain words in your output text, here's a number of things. These are mainly aimed at bots, but should still be generally useful.

Here's a JavaScript, Python, Ruby and PHP word filter:
https://github.com/dariusk/wordfilter

Here's a headline filter:
https://github.com/molly/CyberPrefixer/blob/master/offensive.py

Tips on transphobic joke detection:
http://tinysubversions.com/notes/transphobic-joke-detection/

Some lists of bad words:
https://github.com/shutterstock/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words
https://gist.github.com/ryanlewis/a37739d710ccdb4b406d
http://www.bannedwordlist.com/lists/swearWords.txt

[Inactive] muted Twitter topics:
https://github.com/sjml/bot-innocence

Some general etiquette things:
http://tinysubversions.com/2013/03/basic-twitter-bot-etiquette/
http://www.crummy.com/2013/11/27/0

@greg-kennedy
Copy link

If you need to plug in some Markov Chains in Perl, you could do a lot worse than the module I built for it this year.
https://github.com/greg-kennedy/MarkovChain

@MichaelPaulukonis
Copy link

@MichaelPaulukonis
Copy link

For anyone working with a twitter-sourced corpus, myna is a "JS lib that compiles a tweet to machine speakable text". Or non-twitter-literate text.

For example, the tweet:

"OH: Myna is #SuperAwsome for #TTS and #TwitterBeginners! It is written by @kn bit.ly/ykbz6b"

compiles into plain English:

"Katsuya Noguchi overheard: Myna is Super Awesome for TTS and Twitter Beginners! It is written by Katsuya Noguchi"

@dariusk
Copy link
Owner Author

dariusk commented Oct 8, 2016

Hey all. Since this is our most popular (and most-subscribed to) issue, I just wanted to let people know I have a new issue open to discuss whether we do NaNoGenMo 2016 here on Github again, or somewhere else. Feel free to participate! #199

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests