Update README.md

[sofiaclara93]

Make sure these boxes are checked before your pull request is ready to be reviewed and merged. Thanks!

• all tests pass -- rake test:all
• code is in uniquely-named feature branch, and has been rebased on top of latest master (especially if you've been asked to make additional changes)
• pull request is descriptively named with #number reference back to original issue
• if possible, multiple commits squashed if they're smaller changes
• reviewed/confirmed/tested by another contributor or maintainer
• schema.rb.example has been updated if any database migrations were added

Please be sure you've reviewed our contribution guidelines at https://publiclab.org/wiki/contributing-to-public-lab-software

We have a loose schedule of reviewing and pulling in changes every Tuesday and Friday, and publishing changes on Fridays. Please alert developers on plots-dev@googlegroups.com when your request is ready or if you need assistance.

Thanks!

[sofiaclara93]

@egott @melaniegershman

• Adjust workspace naming to account for errors with former syntax (yourname/plots2 does not work with symbols) in cloud9
• Add additional steps necessary to run the app for the first time

[ryzokuken]

I thought running ./install_cloudnine.sh took care of running stuff like bundle install and rake db:setup

[sofiaclara93]

It took care of the bundle install but then raised an error after running 'rails s -b $IP -p $PORT' and opening the URL because there were was no setup. We had to run 'rake db:setup' then 'rails s -b $IP -p $PORT'. Another two pairs were having the same issue at a hacker event.

[ryzokuken]

Looking into it, I think the shell script has some serious issues. Eg: It does not copy config/database.yml.sqlite.example as config/database/yml, and thus rake db:setup throws an error.

[jywarren]

Indeed - I'd just noticed the lack of copying database.yml -- so sorry you had trouble with the cloud9 install script! We've found it difficult to keep it perfectly set up as the code changes; it's not tested regularly and we should find a way to do better. I'll test out the process start to finish right now to see.

Thanks for your patience and for the extra README steps!

[jywarren]

Hi, just reading through the changes carefully, "yourname/plots2 does not work with symbols" -- was that in reference to the workspace name? I had meant it to be for the "Git or Mercurial URL", so what about expanding it to:

3. Name your project `plots2` under "Workspace name".
4. Enter `your_github_username/plots2` in the "Clone from Git or Mercurial URL" field, and press **Create Workspace** (don't choose a template!)

[jywarren]

I believe @ryzokuken is correct that this step should be run in install_cloudnine.sh, but that script is failing, probably due to lack of a database.yml copy step. I'm testing this out right now and will confirm, and add to this PR or open a new one.

[jywarren]

Confirmed -- a change was made in 67dc936#diff-a3e84a54cf5d4cea38c7b29a7dd76ac8L9 that broke this. Adding the cp config/database.yml.sqlite.example config/database.yml line in now.

[jywarren]

OK - I added the changes on top of yours, and manually merged the final README.md and install_cloudnine.sh files in 917e45b.

I noticed also, though, that you submitted a pull request from your master branch. That can be an issue if publiclab/plots2's master branch goes out of sync with your master branch, because you can't rebase your changes on top or pull in the latest changes (with nowhere left to put them, since you've added to your own master branch). It's not a huge deal, but there's a simple way around this, which is doing any new work of your own in a named feature branch.

There's a guide to this workflow here and if you need to rewind your master branch to get it back in sync with publiclab's, you can follow this procedure -- I'm happy to help you with either, and can also log into your cloud9 environment to help if you run into trouble (you'd have to invite me).

Thanks for noticing this, it's unfortunate that errors crept into our initial startup sequence script, and we're grateful for your having caught them.

Were you running the event last night? I'm curious, how did folks at the event find us?

[ryzokuken]

With the setup process being so complex, I had been thinking of making a Makefile so someone could just run a little make setup or something similar to setup the entire project. But the problem was, a lot of stuff like using a package manager is very platform-dependent, and I just noticed that the entire README.md uses apt-gets in order to install dependencies. This is very Debian-specific and might pose a problem to people on other environments and newer versions of Debian/Ubuntu (New Debian uses just apt, Older RPM systems used yum, Newer ones use dnf, OSX uses homebrew, I guess). Also, the README.md involves installing the openjdk-7-jre and openjdk-7-jdk packages which don't exist on the Ubuntu packages list anymore (I guess), so that needs to be updated too. @jywarren maybe we could talk sometime about the options in order to make the setup process more platform-agnostic and easy?

[jywarren]

I'd like that, yes. I think we have to strike a balance between showing
exact steps for each environment, and acknowledging that things change and
sometimes it may be best to link to others' documentation instead of
maintaining all our own... thanks for thinking about this!

Another strategy could be to use Docker to standardize the environment, and
then we have to provide docker install steps... Anyway yes. And I'd be
happy to talk on the upcoming OpenHour about this if you make it, or at a
later date.

Thanks!!

[ryzokuken]

Docker was definitely what I had in mind. If we use Docker, we can also make a standardized Makefile that sets up the project for the docker ecosystem. Also, every PR that adds a required step for setup could make the needed changes in the Makefile, thus keeping it fresh and relevant.

[icarito]

Hi!
I am responsible for keeping plots2 running in production and have interest in this discussion.
Indeed every step should be taken to simplify setting up the app both for development and deployment for testing and production.
You will notice we already have a Dockerfile which indeed is used by our continuous integration setup for automated testing.
Plots2 being a standard server side application, it requires a database. Docker specifies one service per container which means we have to run two containers, one for the database. The way to specify that, is by using docker-compose. We also have a docker-compose.yml in the repo. It specifies how to create an empty database. The way to use it is encoded into the .travis.yml file which is the script the CI system uses.

How to make it friendlier?
Beyond documenting the above really nicely, I think exploring the possibility of creating a Sandstorm bundle is a great idea!

Currently I am adding another container to the docker-compose.yml file to deal with the new Solr indexer. With this I hope to put a testing instance in the development/testing pipeline, and finally put containers in production once monitoring questions have been resolved.

I am happy to discuss this work and get your suggestions with regard to the road ahead. I expect to take part in OpenHour as it is my favourite subject!

[ryzokuken]

@icarito When you guys ported the app to Rails, did you keep the database as-is? Because the database always seems to be running some really complex hybrid Drupal-Rails sorcery in the background and that's perhaps a reason why we need a separate container for the database. Maybe if we used the Rails-y solution involving ActiveRecord, we wouldn't need two containers.

I'm new with everything so I must've got stuff wrong, please feel free to correct me.

P.S. I guess it has become imperative for me to attend this month's Open Hour considering such wonderful stuff would be discussed.

[icarito]

@ryzokuken good question! I haven't been involved in development beyond very simple patches I have made out of necessity in deployment and sent upstream.

I have noticed the legacy tables from Drupal as well. While the legacy database schema is not so great, it's not really sorcery ;-) , and it doesn't change the fact that a database requires a separate container, and so does an indexer. By the way, to access this legacy schema you are using ActiveRecord.

[ryzokuken]

yes, but the data is being stored using DrupalNodes and stuff rather than the normal way rails handles data on the database (Using MyModel.new and MyModel.create etc)

[icarito]

@ryzokuken I can't comment on the schema itself as I haven't used it. For sure dropping legacy stuff is good!

[david-days]

Hello, all!

I, too, am interested in this conversation. The make and docker approaches are both something I'm interested in and would like to contribute to.

One possibility would be to have multiple installation paths. Since the README for installation is getting complex (which kicked off this conversation), we could split out various installation methods into separate X_INSTALL.md files. We already have a Cloud9 process and a manual process (Cloud9_INSTALL.md and Manual_INSTALL.md), so adding a Make_INSTALL.md and a Docker_INSTALL.md would not be that difficult.

To push it a little further, I think that doing so would have several benefits: First, it would simplify the front-page README.md by quite a bit. When first looking into plots2, it's possible that some potential contributors are intimidated by the level of complexity just to get it up and running. Reworking the setup processes into separate installation methods by which people could choose might give them a little extra boost to stick with it.

Second, it would make the installation process choosable by the contributors based on their resources, experience, and comfort level. For example, I enjoy going through the manual install process (yes--I actually meant the word "enjoy") because it forces me to think about all the intricate parts necessary to make it run. However, I'm neck-deep in a Makefile-based personal project, so automating the setup accordingly would help me stretch those muscles, too.

Lastly, and related to the previous point, different contributors have different backgrounds and requirements. A new contributor who has used Docker would feel very good about going to the Docker_INSTALL.md steps, and any changes/contributions there would help @icarito (and vice-versa). OTOH, I'm more from a make-based background when working with Linux, so using a makefile-installation would make (no pun intended) life a little simpler for me.

(Another aside--one of the make targets could be to update the local code from the fork and/or upstream. Just a thought...)

We'd have to have a set of fairly dedicated maintainers for each _INSTALL.md file, but that can be a roving list of who has expressed interest and experience. It would also allow for future methods of installation. If someone came up with an setup called TELEKENESIS (psychic software installation), for example, we just add the required support steps to TELEKENESIS_INSTALL.md and add an entry to the main README.md)

Just my long-winded, inflation-adjusted two cents...

[jywarren]

Hi, I think it'd be best to move this discussion either to another issue, or to the dev list at plots-dev@googlegroups.com -- just so we don't clutter up Sofia's PR. But thank you for your input!

Just to respond to a couple things quickly --

https://github.com/FalkorCloud/falkor is an interesting open source clone of cloud9 which could let us run it directly off of a docker config, rather than in its own custom install script.

exploring the possibility of creating a Sandstorm bundle is a great idea!

+1!!!

More in a second (since I see others chipping in here) but please do move this to a new issue!

[jywarren]

(And i'm happy to copy my comments into the new issue)

really complex hybrid Drupal-Rails sorcery in the background and that's perhaps a reason why we need a separate container for the database

The legacy Drupal stuff is almost eliminated -- mostly at this point it's:

1. the word Drupal on some table names
2. non-standard table names associated with ActiveRecord models (needs some /app/models/ changes and a migration)
3. users managed across two models (DrupalUsers and User), which would not be too hard to merge

I think we could/should make issues for these, and tackle them -- the huge # of tests from the past summer's work, and the TravisCI service, will make it easier to ensure this goes smoothly in a way that didn't used to be the case.

As to install paths, I prefer paths which are themselves tested regularly, as the Docker process is -- that way it can't go stale. But open to ideas, and I like what David said about breaking out more complex, more obscure, less-common install steps into separate files.

[ryzokuken]

I'm very interested in simplifying the app by porting stuff to the natural Rails side. I think removing unneeded dependencies and cleaning up code would be more instrumental than any other in making the codebase friendlier and simpler to comprehend.

For eg: Solr is a great searching solution and I, by no measure have any intention of scrutinizing a GSoC scholar's work, but installing Java for a Rails application could give me the creeps any day. There would definitely be a more Rails-y solution to indexing/searching and if not, then atleast a Ruby-y (not sure if that even makes sense) solution that would not involve downloading Java or some other additional dependencies but would rather solve problems in our own existing tech stack.

[ryzokuken]

Also, anyone of you on the IRC server right now?

@jywarren told you we need a Gitter chatroom for the project. IMO, it'd serve as yet another way to make the devs approachable and the project friendlier.

[jywarren]

Hi, thanks for your input -- we haven't had time to work on a chat
solution, but I'll bring it up again with our community team (@ebarry and
@steviepubliclab).

As for Solr, we've put a lot of work into making it optional and there
isn't a reason for new coders (or actually most coders, i believe) to need
to install Java at this point -- it was a concern a month ago, but I
believe we're in a much better place now. Thanks!

On Fri, Nov 4, 2016 at 12:50 PM, Ujjwal Sharma notifications@github.com
wrote:

Also, anyone of you on the IRC server right now?

@jywarren https://github.com/jywarren told you we need a Gitter
chatroom for the project. IMO, it'd serve as yet another way to make the
devs approachable and the project friendlier.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#950 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AABfJ9WC2PG0CgGS3cnMpjPL_N_mfkUFks5q62JAgaJpZM4Kn5LF
.

[david-days]

I just added a plots-dev topic for this.

https://groups.google.com/forum/#!topic/plots-dev/dwqeItVUJl8

My apologies, @sofiaclara93, for hijacking your PR!

[jywarren]

Thanks, David -- moved the Drupal legacy discussion to #956.