Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DairyBox Next #71

Open
18 tasks
mikekasprzak opened this issue Feb 9, 2022 · 5 comments
Open
18 tasks

DairyBox Next #71

mikekasprzak opened this issue Feb 9, 2022 · 5 comments

Comments

@mikekasprzak
Copy link
Member

mikekasprzak commented Feb 9, 2022

Well it's been a few years and much of the software we using is out-of-date or unsupported (see #69).

This issue collects some notes and thoughts on what to do next, and what we need.

Post Scotch/Box

In the absence of a replacement for Scotch/Box, we could make a new setup script that starts from a stock Ubuntu vagrant box, and installs/configures everything as we need it. The downside is setup time would be much longer, but frankly we've added so much to Scotch/Box anyway that it might not be that much worse.

Vagrant works best with images that don't need to do much after they've been instanced. That's the opposite of what we've done.

It would be nice if DairyBox only contained Jammer Core setup specifics, and didn't need all the hacked-in software updates.

JuiceBox vs DairyBoxSource

We could create a monolithic script that not only does the system setup, but all the Jammer Core setup as well. This would solve a lot of our problems, and potentially make upgrading to newer Ubuntu versions far simpler (just change the base-box). The downside is every fresh build would be slow, and it would be proprietary. I call this the "DairyBoxSource" approach.

Alternatively we could create a more work for ourselves, isolate the system setup from the Jammer Core setup, and make a general purpose box. The box would be driven by our needs, but anyone could use it for their own projects (even us). Scotch.io did us a huge favour creating and sharing Scotch/Box, and it would be cool if we did the same.

I've dabbled with this in the past, taking Scotch/Box and upgrading it. I also intended to do this with the original JuiceBox, but in practice Scotch/Box was good enough. I still like the name JuiceBox, so I'll call this the "JuiceBox" approach. 😋

I'm conflicted. I think the JuiceBox approach is better, but it would be much easier to get the DairyBoxSource approach working. That said, gutting DairyBoxSource and splitting it into JuiceBox and DairyBox later could also work. It would be nice to just skip the DairyBoxSource approach, but deadlines are looming.

Needs

Here's a rough list of what the new "box" needs.

Server Needs

  • Latest Ubuntu (20.04 today, 22.04 after April)
  • Nginx (instead of Apache)
  • PHP 8.x with APCu
  • Redis (NOTE: only used as a better PHP session cache)
  • MariaDB
  • ImageMagick, FFMPEG, and other image optimizers

Tooling needs

  • GNU Make
  • Nodejs
  • JavaScript linter and other build tools
  • Less and other CSS build tools
  • SVG build tools
  • Typescript tools (future use)
  • Rustup and Cargo (future use)
  • [ ] Sass (future use)

Other needs

  • Something for database hacking (PHPMyAdmin?)
  • Something for outgoing email capturing and review
  • Something capable of emulating Amazon S3 object storage (CEPH?)
  • Search database (Sphinx?)
  • Something to make PHP easier to debug
@mikekasprzak
Copy link
Member Author

On Vagrant boxes: If I recall, creating a Vagrant box isn't actually that difficult, but there are some gotchas.

An Ubuntu installation becomes quite fat once you do something with it. I believe Scotch/Box was only a few hundred MB, but my first boxes were several GB (I have a much faster Internet connection today, so that might be less of a concern). I later found out there are recommendations for "files that can be deleted" from an Ubuntu installation, which greatly helps with file size.

I can't remember the specifics, but I remember there is something really tricky about getting the networking right. Something to do with fixed MAC addresses or interfaces. The problem is that you couldn't safely run 2 instances of the same box without properly changing certain setup files before imaging, and you need to remember to do this every time before you image.

I think there might be something about hosting the boxes, but I can't remember the specifics. I believe Hashicorp runs the servers that host free boxes, and it might have had something to do with them.

Last time I touched this was nearly 6 years ago, so it's possible these are non issues today. Either way there is some research ahead to be sure boxes are being packaged and pushed correctly.

@zwrawr
Copy link
Contributor

zwrawr commented Feb 19, 2022

DareBox


DareBox (dárɛ box) is a pre provisioned LAMP vagrant box in the vein of scotchbox, but up to date.
The name isn't great

This gets Dairybox running on ubuntu 20.04 and replaces Scotch Box. This repo is purely a test and doesn't meet the objectives set out above. The goal is to gain an understanding of how to build, provision, package and upload a Box. Currently, this can be done by changing the version number and running a single command ./darebox.sh build upload. It gets the website more or less running on ubuntu 20.04. The simulator (with my fixes PR) runs and generates test data. MailHog works, PHPMyAdmin works (but has some errors). I haven't tested everything yet, but it's probably not worth doing that till we have decided on a final approach.

So what you would need to look at this are these three branches.

On the points above

  • ScotchBox 3.5 is 1.2GB DareBox v0.0.8 is 1.68GB. There may be some more cleanup we can do but the reality is that newer Ubuntu releases are bigger than old ones. This is why ScotchBox went from 800MB to 1.2GB between 3.0 and 3.5. There are some cleanup steps I have turned off because they were uninstalling the software I just installed. We might be able to reduce this a bit.
  • I believe the MAC address problem is solved by the cleanup scripts I am running
  • This is more of the JuiceBox style
  • app.vagrantup.com seems to allow free hosting of publicly available boxes
  • Switching to php8.x would actually simplify the scripts a lot since it is the default in ubuntu 20.
  • I'm using Hashicorps packer as well as vagrant. I don't think this is really what it is meant for. It does help in some ways but I might try not using it and just use the darebox.sh script.
  • Installing other software should be easy
  • Switching to Nginx shouldn't be too bad

Now I have an idea of how to create a base box it shouldn't be too hard to make something that meets a specific set of requirements.

https://app.vagrantup.com/zwrawr/boxes/DareBox

@winniehell
Copy link

Don't want to criticize your approach here, just out of curiosity: What's the main difference/advantage of a Vagrant box in that context over a docker-compose file? Does it make development on a Windows machine easier?

@mikekasprzak
Copy link
Member Author

mikekasprzak commented Mar 30, 2022 via email

@mikekasprzak
Copy link
Member Author

So what you would need to look at this are these three branches.

@zwrawr Will do, and thank you for digging into this! I'll fork and send a PR once things calm down here.

FYI: I'm taking the rest of the month off after LD50 ends (April 21st). In early May, I'll get back to rebuilding the servers. Once that's done, we can start accepting PHP 8.x changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants