Linux Setup

Note: We strongly encourage using Docker to set up your dev environment. Due to limited resources, we can't offer support for all local installs, so please proceed with the understanding that you'll have to do most of the troubleshooting on your own.

This guide is intended for people who who want to set up the archive software for development purposes and who are experienced with Linux; i.e. know how to use the package manager and the command line. It was tested on Ubuntu Xenial; package names might differ slightly for other Linux distributions.

Getting the source code

First of all, we need to get the source code; it is located at GitHub and managed by a version control system which is called git. To download the source code, you need to install a git client. As root, execute

aptitude install git

Now you can download (check out) the source code by typing in the command line (as a normal user)

git clone git://github.com/otwcode/otwarchive.git otwarchive-read-only

You will see what files will be downloaded, it will take a while until the whole repository is completely downloaded to your hard drive. When the command is finished, you will have a directory otwarchive-read-only which contains the source code. If you want, you can already look around!

Getting the required software to run the archive

Now for the interesting part: Getting the archive to run on our own machine! We will need a full Ruby >= 3.1 and MySQL >= 5.0 setup, as well as some additional libraries.

Note: unless you know what you are doing, use current versions of Ruby (specified in Gemfile) and MySQL (search the wiki for the newest supported version). If you use too old or too new versions of Ruby and MySQL, you might end up with incompatible dependencies and schema errors.

The following packages need to be installed (as root):

aptitude install git ruby3.1 ruby3.1-dev mysql-server libmysqld-dev libmysqlclient-dev libxml2-dev libxslt-dev redis-server

(If your distribution's default Ruby version is 3.1 or higher, you can omit the version number and go with whatever your distribution provides. On Ubuntu Xenial however, 2.3 is the default and you might not be able to uninstall it if other installed programs depend on it. In this case, you have to be careful always to use the right Ruby version, i.e. >= 3.1)

(If for some reason your Linux distribution doesn't provide a suitable Ruby version, it might be possible to use a 3rd party repository (this was last tested with the Brightbox one), or you'll have to compile Ruby yourself, see Appendix: Compiling Ruby.)

It should ask you for a password you want to use for the MySQL root (not to be confused with the the root of your Linux system), and it will start the MySQL server in the background. From now on, the MySQL server will automatically start at boot time.

The Rails framework will not be installed via the Linux package management, but via Ruby's package management, gem. Still as root, type

gem3.1 install rails bundler

(Again, you can omit the version number if your default Ruby version is the right one.)

You might want to create links pointing to the rails and bundler executables, e.g.

ln -s /var/lib/gems/3.1.4/gems/rails-3.0.3/bin/rails /usr/bin/rails
ln -s /var/lib/gems/3.1.4/gems/bundler-1.17.3/bin/bundle /usr/bin/bundle

Adjust the path names to your rails and bundler version number.

(If your default Ruby version is not the right one, you will have to adjust those two files: The first line should then read

#!/usr/bin/env ruby3.1.4

instead of

#!/usr/bin/env ruby

)

Now change to the otwarchive source directory and install the needed Ruby plugins:

cd otwarchive-read-only
bundle install

Configuring the archive

As normal user, go to the otwarchive-read-only directory. You should see a file config/database.example, open it with an Editor and save it to config/database.yml. At the top of the file there's a line where you should enter the root password you had to choose when installing MySQL; the line should look something like this:

password: secret

If you have stuck to default settings, it won't be necessary, but if you run into difficulties you may need to specify the socket in this file as well.

You'll also need to create a direct copy of other config files:

cp config/config.yml config/local.yml
cp config/redis.example config/redis.yml

And adjust so that the redis config is not pointing at dev.ao3.org; for now do: open config/redis.yml and change redis.ao3.org and dev.ao3.org to 0.0.0.0.

Now you need to create the database schema in the database:

rake12.3.3 db:create:all

(Again, you can omit the version number if your default Ruby version is the right one. You may need to run it as bundle exec rake db:create:all.)

To start with an empty database, use

rake db:schema:load
rake db:migrate

Note: MySQL 5.7 does not support null defaults for primary keys. This means that if your local MySQL version is 5.7 or higher, you'll need to override the default for primary keys, as described in this StackOverflow link. Placing the file in the directory described by that answer (there will not be a preexisting file) should allow the schema load to run without issues.

The seed data should still work as of 12.10.2019.

To create an admin account, run

rails runner script/create_admin.rb

(If you are not using the default Ruby version, you need to edit script/create_admin.rb beforehand so that the first line points to the right ruby executable:

#!/usr/bin/env ruby3.1.4

)

Finally, start Redis (the ampersand will start it as a background process):

redis-server &

Edit config/redis.yml to point to localhost:[the port Redis is running on] for all development instances of Redis.

(Note: the Redis process can be killed and the archive will still load; however, it does need an initial local start, or you'll see a controller error on launch.)

Skins

Were you to start the archive now, it would run, but without much in the way of formatting. ImageMagick is required to use the archive's skins. On Ubuntu, ImageMagick can be installed via the package manager, using

sudo apt install imagemagick

It can also be installed from source.

After install, create a user with username 'lim' using, again, script/create_admin.rb. Run the following commands:

bundle exec rake skins:load_site_skins
bundle exec rake skins:load_user_skins

Select 'N' for 'replace existing skins'. You may see an iteration error during the second rake; this will not prevent the default skin from being used.

Starting the archive

Now, still as normal user and in the otwarchive-read-only directory, start the archive

rails server

You should see roughly the following output:

=> Booting WEBrick
=> Rails 3.0.3 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2010-12-30 12:48:50] INFO  WEBrick 1.3.1
[2010-12-30 12:48:50] INFO  ruby 1.9.2 (2010-08-18) [x86_64-linux]
[2010-12-30 12:48:50] INFO  WEBrick::HTTPServer#start: pid=3789 port=3000

You will also likely see errors related to Elasticsearch and Memcached connectivity - both of which are beyond the scope of this page, but neither of which should prevent basic launch.

Now open up a browser and visit http://localhost:3000/, you should see the archive's front page! You can login to your new admin account using the URL http://localhost:3000/admin/login. BTW, most of the archive configuration can be found in config/config.yml, which you should now have copied into config/local.yml.

Note that the archive will run in development mode by default. This will give you additional debug information and will cause Rails to reload the source code on every request so that you don't have to restart the archive on code changes. On the downside, it will slow down the archive quite a bit.

Take a look at: https://github.com/otwcode/otwarchive/wiki/Development-Data for tips on how to add some test user data to your instance.

If you want to check the mails the archive would send (for example to create new accounts), you can find a copy of each mail in otwarchive-read-only/log/development.log (among lots and lots of other stuff ;-)

If you made it thus far, we would love for you to join us! ;-)

Updating

In the future, the source code will be changed by the developers, and you, of course, want to have those updates, too. Change to the source directory and run the update command to check for new stuff (as normal user):

cd otwarchive-read-only
git pull upstream master

To update your database after you have pulled changes from git, run:

rake12.3.3 db:migrate
rake12.3.3 after