Skip to content
Barry Allard edited this page Mar 23, 2016 · 9 revisions

Sequel + Parallel

Especially with the sqlite db backend, it doesn't really work because there's some unresolved threading issues.

A temporary, hacky, slow workaround (which basically defeats Parallel) is to wrap db accesses in .synchronize blocks like so:

DB.synchronize do
  s = some_model.find_or_create blah: foo
  s.add_whatever
  s.save
end

The following currently results in a Sequel::DatabaseError: SQLite3::BusyException: database is locked error at sqlite3-1.3.11/lib/sqlite3/statement.rb:108:in 'step'

require 'sequel'
require 'parallel'
DB = Sequel.connect('sqlite://wontwork.sqlite3')

DB.create_table!(:users) do
  primary_key :id
  String :name, null: false
  index :name, unique: true
end
DB.create_table!(:posts) do
  primary_key :id
  Integer :user_id, null: false, index: true
end

class User < Sequel::Model(DB)
  one_to_many :posts
end

class Post < Sequel::Model(DB)
  many_to_one :user
end

users = %w[geroge william joe bob jane mary sue sally wilma]

Parallel.each(users, in_threads: 5) do |name|
 u =  User.find_or_create(name: name)
 Parallel.each(1000.times, in_threads: 10) { u.add_post({}) }
end  
Clone this wiki locally