eSpace Blog RSS
23Mar

Thinking Sphinx on Windows

8

In one of my Ruby on Rails projects, I needed a full-text search engine to integrate. After some research, I decided that Thinking Sphinx is the way to go. It seemed simple, fast and well documented. Unfortunately, I had some issues when tried to get it running on Windows.

The first step is installing Sphinx itself. While this looks simple, it took me five hours to move to the next step!

 

When I checked Sphinx documentation, it said that I needed Microsoft Visual C/C++ Studio to compile the source code! What? This is another project! After some (read: a lot of) research, some people suggested that I just needed to download the binaries, not the source code, and add the bin folder to the Windows Path environemnt variable. I tried that, but got errors later when tried to index data. So here is what works:

  1. Go to Sphinx downloads page and download the Win32 release binaries with MySQL support. At the time of writing it is sphinx-0.9.8-win32.zip. Of course, you should get the one with PostgreSQL support if you need it.

  2. Unzip the downloaded file and open the bin folder. Select and copy all files and paste them in your ruby/bin folder. If some files already exist in your ruby/bin folder, don't replace them.

  3. Install the Thinking Sphinx plugin:

ruby script/plugin install git://github.com/freelancing-god/thinking-sphinx.git

       4. To set a model to be indexed, add some fields and attributes (refer to the primer):

class Post < ActiveRecord::Base  
  define_index do   
    #sphinx fields
    indexes title
  end  
end

 

      5. Tell Sphinx to index the data:

rake thinking_sphinx:index

Now, you may get something like this in your console:

 

 

rake thinking_sphinx:index
Generating Configuration to F:/InstantRails/rails_apps/community/config/development.sphinx.conf
indexer --config F:/InstantRails/rails_apps/community/config/development.sphinx.conf --all

It seems that it couldn't continue. To solve this, make sure you copied all the .exe files from
sphinx-0.9.8-win32/bin to ruby/bin. If you keep getting the same result, you may need to copy the libmySQL.dll file from mysql/bin to ruby/bin (check this post for details). Now, you should get something like this

Generating Configuration to F:/InstantRails/rails_apps/community/config/development.sphinx.conf
indexer --config F:/InstantRails/rails_apps/community/config/development.sphinx.conf --all
Sphinx 0.9.8-release (r1371)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file 'F:/InstantRails/rails_apps/community/config/development.sphinx.conf'...
indexing index 'post_core'...
collected 2 docs, 0.0 MB
collected 0 attr values
sorted 0.0 Mvalues, 100.0% done
sorted 0.0 Mhits, 100.0% done
total 2 docs, 27 bytes
total 0.030 sec, 904.37 bytes/sec, 66.99 docs/sec
distributed index 'post' can not be directly indexed; skipping.

Also, note the newly generated files:

  • config/development.sphinx.conf
  • db/sphinx/development/post_core.spa
  • db/sphinx/development/post_core.spd
  • db/sphinx/development/post_core.sph
  • db/sphinx/development/post_core.spi
  • db/sphinx/development/post_core.spm
  • db/sphinx/development/post_core.spp

Great! But what was that last line about? Don't worry, Pat says it's OK

A distributed index is made up of other indexes. It doesn't need to be
indexed, but Sphinx tries to anyway, and then complains when it can't.
I'm not sure about the reasoning for this, but it's not important: You
can ignore the error, it doesn't stop anything working, searching and
indexing will happen as expected.

          6.Start a Sphinx searchd daemon:

rake thinking_sphinx:start

You should get something like this:

 

searchd --config F:/InstantRails/rails_apps/community/config/development.sphinx.conf
[Wed Oct 29 14:26:44.390 2008] [ 2796] WARNING: forcing --console mode on Windows
[Wed Oct 29 14:26:44.390 2008] [ 2796] using config file 'F:/InstantRails/rails_apps/community/config/development.sphinx.conf'...
[Wed Oct 29 14:26:44.406 2008] [ 2796] creating server socket on 127.0.0.1:3312
[Wed Oct 29 14:26:44.406 2008] [ 2796] accepting connections

Now, you are ready to search. To test it, you can modify the index action as follows:

class PostsController < BaseController
  def index
    @posts = Post.search params[:search]
  end  
end

And your view may look like this:


<% for post in @posts %>
  <h2><%=h post.subject %></h2>
  <%=h post.body %>
<% end %>

Now, navigate to /posts?search=myword, where myword is a string that can be found in the title of some of your posts. You should see that the results are filtered according to the keyword sent.

Congratulations! You are thinking Sphinx on Windows.

 

Written By:

Hatem Mahmoud (www.expressionlab.com)

Comments

1 Chris wrote: on Thursday, April 16, 2009:

Hi Hatem,

I've tried to follow your tutorial, but I'm afraid I'm missing a step. Once I download sphinx, and copy the bin files, do I need to do anything else with the rest of the files? When I try to run "rake thinking_sphinx:index" I get an error that says....

Don't know how to build task 'thinking_sphinx:index".

I am running Windows Vista, and Instantrails. Any advice is welcome.

Chris

2 chris wrote: on Thursday, April 16, 2009:

This tutorial was great. I apologize if you see 2 basically identical posts. I found why it wasn't working. I needed to use the mysygit program from github to install the plugin on my Windows machine. For some reasong the "ruby script/install plugin" wasn't working from the console. Once I did that everything worked great. Many thanks for this tutorial. I'm sure many others will be thankful as well.

3 chris wrote: on Thursday, April 16, 2009:

BTW, I'm using the following

"Instant"Rails 2.3.2
Ruby 1.8.6
Windows Vista

4 Hatem wrote: on Wednesday, April 22, 2009:

Congrats Chris. Don't forget to check my other post on ThinkingSphinx: http://www.expressionlab.com/2008/11/19/thinking-sphinx-in-arabic-unicode

5 chimere wrote: on Tuesday, June 02, 2009:

Am using Vista and am getting a bad file descriptor notice when i run "rake thinking_sphinx: index". here's the output "druby://localhost:9010 - #<Error::EBADF: Bad file descriptor - connect (2)>". What could be wrong?

6 Aninda wrote: on Tuesday, July 07, 2009:

I wonder what I am going to have to make to deploy to a linux box from a windows dev box. Any pointers? Thanks.

7 Hatem wrote: on Monday, July 13, 2009:

@Aninda there is nothing special about that. Just upload your code and follow the quickstart: http://freelancing-god.github.com/ts/en/quickstart.html

8 Hatem wrote: on Tuesday, July 14, 2009:

@Aninda there is nothing special about this, just upload your code and follow the quickstart: http://freelancing-god.github.com/ts/en/quickstart.html

Post a Comment

eSpace podcast Prodcast

RSS iTunes

Archive

Latest Comments