Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

basic setup for a rmq worker for rendering rss feeds #845

Open
wants to merge 34 commits into
base: integration
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
a78ce3a
basic setup for a rmq worker for rendering rss feeds
branch14 Feb 15, 2018
691fde9
add missing s
branch14 Feb 15, 2018
0c4569f
Create `Podcaster` class boilerplate
munen Feb 16, 2018
ac0a18c
Render feed in callback
munen Feb 16, 2018
e2b7dee
Merge branch 'integration' of github.com:munen/voicerepublic_dev into…
munen Feb 16, 2018
294503d
Rails logger works within Daemons
munen Feb 17, 2018
60675ad
Crosscut: Volatile working version
munen Feb 17, 2018
9823a18
Make rubocop happy
munen Feb 17, 2018
70f3b8a
Make rubocop happy
munen Feb 17, 2018
7034eff
Bugfix: if != unless
munen Feb 17, 2018
a84ff7b
Jot down thoughts on Podcaster's stale data
munen Feb 17, 2018
288f490
Refactor
munen Feb 19, 2018
83d7a18
Refactor
munen Feb 19, 2018
18388c0
Introduce `docker-compose.yml` for RMQ
munen Feb 19, 2018
578d070
Fixed: `render_for_talk` renders a previous version of the talk
munen Feb 19, 2018
84d0bdb
Doc for code review
munen Feb 19, 2018
0c0ed58
Refactor
munen Feb 19, 2018
1208a05
Render Series Feeds
munen Feb 19, 2018
42cdbd6
Render dependent series RSS when talk has changed
munen Feb 19, 2018
9e04ae1
Series Podcast only contains archived and ordered Talks
munen Feb 19, 2018
91c5ad3
Bugfix: Actually render associated talks
munen Feb 19, 2018
d7f8249
Bugfix: Actually call `html_escape`
munen Feb 19, 2018
e8a5186
Finish: Render Series podcast feed
munen Feb 25, 2018
f1258e5
Render UserPublished podcast feed
munen Feb 25, 2018
25d38f0
Render FeaturedTalks podcast
munen Feb 26, 2018
64b8cca
Render UserPublished podcast on Talk.update
munen Feb 26, 2018
c67e854
Add task to trigger creation of all podcast feeds
munen Mar 10, 2018
ce508b5
Place podcast feeds in `public/system`
munen Mar 10, 2018
8de113d
Create cronjob to render featured podcasts every 5min
munen Mar 13, 2018
9de43da
Add UserPinned Podcasts
munen Mar 13, 2018
7b59368
Start documenting the podcast creation flow
munen Mar 13, 2018
3fbde76
Add guard
munen Mar 13, 2018
a0fb6bb
Fix series_controller_spec
munen Mar 13, 2018
8a67ed8
Fix users_controller_spec
munen Mar 13, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ local/
.*.swo
/public/sitemap.xml

/public/feeds/**

#########################
# Emacs temporary files #
#########################
Expand All @@ -71,3 +73,4 @@ Thumbs.db
/userdata_audio_worker.sh
/userdata_icebox.sh
/userdata_jumphost.sh
/app/assets/javascripts/.tern-port
8 changes: 7 additions & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,13 @@ group :test do
gem 'ci_reporter'
gem 'database_cleaner', github: 'bmabey/database_cleaner'

#gem 'guard-rspec'
# BEGIN guard deps
gem 'guard-rspec', require: false
gem 'ruby_dep', '1.3.1' # newer versions require ruby 2.2+
gem 'listen', '3.0.8' # newer versions require ruby 2.2+
# END guard deps


gem 'launchy'
gem 'poltergeist'
gem 'selenium-webdriver'
Expand Down
34 changes: 34 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,20 @@ GEM
globalid (0.3.6)
activesupport (>= 4.1.0)
greensock-rails (1.18.4.0)
guard (2.14.2)
formatador (>= 0.2.4)
listen (>= 2.7, < 4.0)
lumberjack (>= 1.0.12, < 2.0)
nenv (~> 0.1)
notiffany (~> 0.0)
pry (>= 0.9.12)
shellany (~> 0.0)
thor (>= 0.18.1)
guard-compat (1.2.1)
guard-rspec (4.7.3)
guard (~> 2.1)
guard-compat (~> 1.1)
rspec (>= 2.99.0, < 4.0)
haml (4.0.7)
tilt
haml-rails (0.9.0)
Expand Down Expand Up @@ -382,8 +396,12 @@ GEM
letter_opener_web (1.2.3)
letter_opener (~> 1.0)
rails (>= 3.2)
listen (3.0.8)
rb-fsevent (~> 0.9, >= 0.9.4)
rb-inotify (~> 0.9, >= 0.9.7)
loofah (2.0.3)
nokogiri (>= 1.5.9)
lumberjack (1.0.12)
mail (2.6.3)
mime-types (>= 1.16, < 3)
mailgun_rails (0.8.0)
Expand All @@ -401,12 +419,16 @@ GEM
multi_json (1.11.2)
multi_xml (0.5.5)
multipart-post (2.0.0)
nenv (0.3.0)
net-scp (1.2.1)
net-ssh (>= 2.6.5)
net-ssh (3.0.2)
netrc (0.11.0)
nokogiri (1.6.7.2)
mini_portile2 (~> 2.0.0.rc2)
notiffany (0.1.1)
nenv (~> 0.1)
shellany (~> 0.0)
oauth2 (1.0.0)
faraday (>= 0.8, < 0.10)
jwt (~> 1.0)
Expand Down Expand Up @@ -483,6 +505,9 @@ GEM
thor (>= 0.18.1, < 2.0)
raindrops (0.15.0)
rake (10.5.0)
rb-fsevent (0.10.3)
rb-inotify (0.9.10)
ffi (>= 0.5.0, < 2)
rbvmomi (1.8.2)
builder
nokogiri (>= 1.4.1)
Expand All @@ -496,6 +521,10 @@ GEM
http-cookie (>= 1.0.2, < 2.0)
mime-types (>= 1.16, < 3.0)
netrc (~> 0.7)
rspec (3.2.0)
rspec-core (~> 3.2.0)
rspec-expectations (~> 3.2.0)
rspec-mocks (~> 3.2.0)
rspec-core (3.2.3)
rspec-support (~> 3.2.0)
rspec-expectations (3.2.1)
Expand All @@ -513,6 +542,7 @@ GEM
rspec-mocks (~> 3.2.0)
rspec-support (~> 3.2.0)
rspec-support (3.2.2)
ruby_dep (1.3.1)
ruby_parser (3.7.3)
sexp_processor (~> 4.1)
rubyzip (1.1.7)
Expand All @@ -534,6 +564,7 @@ GEM
rubyzip (~> 1.0)
websocket (~> 1.0)
sexp_processor (4.6.1)
shellany (0.0.1)
simple_form (3.2.1)
actionpack (> 4, < 5.1)
activemodel (> 4, < 5.1)
Expand Down Expand Up @@ -646,13 +677,15 @@ DEPENDENCIES
foundation-rails (= 6.2.0.1)
friendly_id
greensock-rails
guard-rspec
haml-rails
inifile
jquery-rails
jquery-ui-rails
launchy
letter_opener
letter_opener_web (~> 1.2.0)
listen (= 3.0.8)
mailgun_rails
meta_request
omniauth-facebook (= 4.0.0)
Expand All @@ -669,6 +702,7 @@ DEPENDENCIES
redcarpet
rspec-rails (= 3.2.1)
rspec_junit_formatter!
ruby_dep (= 1.3.1)
sass-rails
sdoc (~> 0.4.0)
selectize-rails
Expand Down
82 changes: 64 additions & 18 deletions Guardfile
Original file line number Diff line number Diff line change
@@ -1,24 +1,70 @@
# A sample Guardfile
# More info at https://github.com/guard/guard#readme

guard 'rspec', zeus: true, bundler: false do
watch(%r{^spec/.+_spec\.rb$})
watch(%r{^lib/(.+)\.rb$}) { |m| "spec/lib/#{m[1]}_spec.rb" }
watch('spec/spec_helper.rb') { "spec" }

# Rails example
watch(%r{^app/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
watch(%r{^app/(.*)(\.erb|\.haml)$}) { |m| "spec/#{m[1]}#{m[2]}_spec.rb" }
watch(%r{^app/controllers/(.+)_(controller)\.rb$}) { |m| ["spec/routing/#{m[1]}_routing_spec.rb", "spec/#{m[2]}s/#{m[1]}_#{m[2]}_spec.rb", "spec/acceptance/#{m[1]}_spec.rb"] }
watch(%r{^spec/support/(.+)\.rb$}) { "spec" }
#watch('config/routes.rb') { "spec/routing" }
watch('app/controllers/application_controller.rb') { "spec/controllers" }

# Capybara request specs
watch(%r{^app/views/(.+)/.*\.(erb|haml)$}) { |m| "spec/requests/#{m[1]}_spec.rb" }

## Uncomment and set this to only include directories you want to watch
# directories %w(app lib config test spec features) \
# .select{|d| Dir.exists?(d) ? d : UI.warning("Directory #{d} does not exist")}

## Note: if you are using the `directories` clause above and you are not
## watching the project directory ('.'), then you will want to move
## the Guardfile to a watched dir and symlink it back, e.g.
#
# $ mkdir config
# $ mv Guardfile config/
# $ ln -s config/Guardfile .
#
# and, you'll have to watch "config/Guardfile" instead of "Guardfile"

# Note: The cmd option is now required due to the increasing number of ways
# rspec may be run, below are examples of the most common uses.
# * bundler: 'bundle exec rspec'
# * bundler binstubs: 'bin/rspec'
# * spring: 'bin/rspec' (This will use spring if running and you have
# installed the spring binstubs per the docs)
# * zeus: 'zeus rspec' (requires the server to be started separately)
# * 'just' rspec: 'rspec'

guard :rspec, cmd: "bin/rspec" do
require "guard/rspec/dsl"
dsl = Guard::RSpec::Dsl.new(self)

# Feel free to open issues for suggestions and improvements

# RSpec files
rspec = dsl.rspec
watch(rspec.spec_helper) { rspec.spec_dir }
watch(rspec.spec_support) { rspec.spec_dir }
watch(rspec.spec_files)

# Ruby files
ruby = dsl.ruby
dsl.watch_spec_files_for(ruby.lib_files)

# Rails files
rails = dsl.rails(view_extensions: %w(erb haml slim))
dsl.watch_spec_files_for(rails.app_files)
dsl.watch_spec_files_for(rails.views)

watch(rails.controllers) do |m|
[
rspec.spec.call("routing/#{m[1]}_routing"),
rspec.spec.call("controllers/#{m[1]}_controller"),
rspec.spec.call("acceptance/#{m[1]}")
]
end

# Rails config changes
watch(rails.spec_helper) { rspec.spec_dir }
watch(rails.routes) { "#{rspec.spec_dir}/routing" }
watch(rails.app_controller) { "#{rspec.spec_dir}/controllers" }

# Capybara features specs
watch(rails.view_dirs) { |m| rspec.spec.call("features/#{m[1]}") }
watch(rails.layouts) { |m| rspec.spec.call("features/#{m[1]}") }

# Turnip features and steps
watch(%r{^spec/acceptance/(.+)\.feature$})
watch(%r{^spec/acceptance/steps/(.+)_steps\.rb$}) { |m| Dir[File.join("**/#{m[1]}.feature")][0] || 'spec/acceptance' }
watch(%r{^spec/acceptance/steps/(.+)_steps\.rb$}) do |m|
Dir[File.join("**/#{m[1]}.feature")][0] || "spec/acceptance"
end
end

2 changes: 1 addition & 1 deletion app/controllers/api/oembed_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def show
thumbnail_url: talk.image.url,
thumbnail_width: talk.image.width,
thumbnail_height: talk.image.height )
response[:url] = vrmedia_url(talk) if talk.archived?
response[:url] = Podcaster.vrmedia_url(talk) if talk.archived?
end

render json: response.to_json
Expand Down
13 changes: 11 additions & 2 deletions app/controllers/reminders_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,17 @@ class RemindersController < BaseController
# GET /users/:user_id/reminders
def index
@user = User.find(params[:user_id])
talks = Talk.remembered_by(@user)
@podcast = OpenStruct.new(talks: talks)

respond_to do |format|
# TODO: Discuss during Code Review
# `format.html` probably is not needed, because there's no view
# templates. I put it here in case I'm missing something and the
# index action is not only used to render podcast feeds.
format.html
format.rss do
render file: Rails.root.join(Settings.feeds.path, 'user_pinned', "#{@user.id}.rss")
end
end
end

# POST /talks/:talk_id/reminders
Expand Down
2 changes: 1 addition & 1 deletion app/controllers/root_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ def index
@talks_upcoming = Talk.upcoming.limit(12)
end
format.rss do
@podcast = OpenStruct.new(talks: Talk.recent.limit(10))
render file: Rails.root.join(Settings.feeds.path, 'featured/index.rss')
end
format.xml
end
Expand Down
5 changes: 3 additions & 2 deletions app/controllers/series_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,11 @@ def show
@total_plays = @series.talks.sum(:play_count)
end
format.json { render json: @series }

format.rss do
talks = @series.talks.archived.ordered
@podcast = OpenStruct.new(talks: talks)
render file: Rails.root.join(Settings.feeds.path, 'series', "#{@series.id}.rss")
end

format.ics
end
end
Expand Down
2 changes: 1 addition & 1 deletion app/controllers/talks_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ def show
format.png { send_file @talk.flyer.path(true) }
format.ics
format.rss do
@podcast = OpenStruct.new(talks: [@talk])
render file: Rails.root.join(Settings.feeds.path, 'talks', "#{@talk.id}.rss")
end
end
end
Expand Down
3 changes: 1 addition & 2 deletions app/controllers/users_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,7 @@ def show

end
format.rss do
talks = @user.talks.archived.order('updated_at DESC')
@podcast = OpenStruct.new(talks: talks)
render file: Rails.root.join(Settings.feeds.path, 'users', "#{@user.id}.rss")
end
format.js do
@talks =
Expand Down
15 changes: 0 additions & 15 deletions app/helpers/application_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,6 @@ def body_classes
list * ' '
end

def itunes_image_url(image)
image.thumb('1400x1400#', format: 'png').url(name: 'image.png')
end

def default_content(locale, key)
keys = [locale.to_s] + key.split('.')
keys.reduce(sections) do |r, k|
Expand Down Expand Up @@ -78,12 +74,10 @@ def section(key, interpolations={})
end
alias_method :s, :section


def blog_url(path)
"http://blog.voicerepublic.com#{path}?lang=#{I18n.locale}"
end


def icon_tag(topic, opts={})
title = opts[:title] || topic
"<div class='svg-icon' title='#{title}'><svg><use xlink:href='#icon-#{topic}'></use></svg></div>".html_safe
Expand All @@ -93,15 +87,10 @@ def naked_icon(topic, opts={})
"<svg><use xlink:href='#icon-#{topic}'></use></svg>".html_safe
end


def test_talk_path
new_talk_path talk: { dryrun: true }
end

def vrmedia_url(talk, fmt='mp3')
root_url + 'vrmedia/' + talk.id.to_s + '.' + fmt
end

def rss_link_tag(title, url=nil)
url ||= url_for(format: 'rss')
tag :link, rel: "alternate",
Expand Down Expand Up @@ -154,10 +143,6 @@ def strip_html(str)
document.text
end

def unsecured_url(url)
url.gsub(/https:\/\//, 'http://')
end

def render_social_meta_tags(opts)
render partial: 'shared/social_meta_tags', locals: opts
end
Expand Down
17 changes: 17 additions & 0 deletions app/models/series.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ class Series < ActiveRecord::Base
# RECORDINGS_ARCHIVE_PATH = "#{Rails.root}/public/system/recordings_raw_archive"
RECORDINGS_ARCHIVE_PATH = Settings.rtmp.archive_path

# See `emit_render_feed` in Talk model
attr_accessor :emit_render_feed

acts_as_taggable

# TODO: rename to host
Expand Down Expand Up @@ -49,6 +52,9 @@ class Series < ActiveRecord::Base
before_save :process_description, if: :description_changed?
before_save :set_image_alt, unless: :image_alt?

after_save :remember_to_render_feed
after_commit :render_feed!, if: :emit_render_feed

accepts_nested_attributes_for :talks

serialize :options
Expand Down Expand Up @@ -113,4 +119,15 @@ def inherit_penalty
self.penalty = user.penalty
end

def remember_to_render_feed
should_render_feed = title_changed? || description_changed? ||
teaser_changed?

self.emit_render_feed = should_render_feed
end

def render_feed!
Emitter.render_feed(:series, id: id)
end

end
Loading