NeverBlock and ActiveRecord, Concurrent DB Access Without Threads

A new neverblock_postgresql activerecord adapter has been released. This new adapter makes it possible for applications using activerecord to utilize multiple connections to do queries in parallel in single threaded applications. This results in much higher performance and better CPU utilization.

What do I need to do?

If you were using the vanilla postgresql adapter you would’ve initialized the connection as follows:

ActiveRecord::DB.establish_connection( :adapter => “postgresql”, :host => host, :database => database, :username => user, :password => pass )

Now, to use NeverBlock all you need to do is:

ActiveRecord::DB.establish_connection( :adapter => “neverblock_postgresql”, # neverblock_postgresql instead of neverblock :host => host, :database => database, :username => user, :password => pass, :connections => 8 # The number of connections to use, optional, defaults to 4 )

Note: you must have the pg, neverblock, neverblock-pg and the activerecord-neverblock-postgresql-adapter gems installed

This way the AR library will pick the gem and use neverblock_postgresql instead of the postgresql adapter

Is that all?

To be able to utilize NeverBlock, you need to be running in an EventLoop, namely EventMachine's event loop. Establishing a connection should happen in the EM.run block or in a later event. The other thing is that you need to be running within a fiber, using NeverBlock's FiberPool class. Does it sound confusing? don't worry, soon we will release an end to end solution where you wont worry about any of this to reap all the benefits of NeverBlock.

Example

require 'neverblock' require 'activerecord' class User < ActiveRecord::Base # assuming a users table in the db end def test_pg(count) t = Time.now count.times do db_block end puts "Blocking time: #{Time.now - t}" end def test_nb_pg(count) i = 0 pool = FiberPool.new(10) t = Time.now count.times do pool.spawn do db_block i = i + 1 puts "NeverBlock time: #{Time.now - t}" if i == count end end end def db_block User.transaction do user_count = User.count user = User.create({:name=>'xyz',:password=>'abc'}) raise "user not inserted" if User.count != user_count + 1 user.destroy end end EM.run ActiveRecord::DB.establish_connection( :adapter => "neverblock_postgresql", :host => "localhost", :database => "evented", :username => "postgres", :connection => 8 ) count = 500 test_pg(count) test_nb_pg(count) end

We benchmarked the new adapter against the vanilla postgresql adapter. This time our workloads consisted of normal AR operations repeated for a 1000 times. We increased the workload by increasing the count and complexities of the queries we ran in each fiber.

Running the work loads against postgresql-adapter vs neverblock-postgresql-adapter

NeverBlock advantage for different workloads

Active Record Connection Scaling With NeverBlock

Testing NeverBlock with different connection counts for slow and fast queries

The benchmarks page has been updated with the NeverBlock::ActiveRecord performance results.

What's next? I think you already guessed it!