Joe@Nitobi

September 2nd, 2008

One thing that I’ve been using a lot is will_paginate and it’s derivatives, such as merb_paginate, ultraminx and ultrasphinx. One problem that I noticed with merb_paginate was that it didn’t want to use the restful paths in the merb app that I was using. This was rather annoying, since I was trying to make my merb app almost entirely restful, except for the home controller. So, instead of using paginator, I decided that I wanted to add ajax pagination.

The first step that needs to be done is to create a create and delete js view. This is extremely basic. I have some code that does this in one of my github repositories. Namely, the sparty_server code.

In the controller, uncomment this line as usual. We are going to be sending back json:

 provides :xml, :yaml, :js

Then create the following *.js.erb files:

index.js.erb

var events = "";
<%= partial "widgets", :with => @widgets %>
<%= partial "shared/paginate", :items => @widgets %>
$jQ(”#main_eventlist”).empty();
$jQ(”#main_eventlist”).append(events);
$jQ(”.pagination”).empty();
$jQ(”.pagination”).append(navigation);

Note that you should really make this so that it uses your jQuery variable. Here, we’re not using noConflict. noConflict should almost always be used, since you most likely will be using a non-jQuery item with your software. We also created two partials, namely one that just lists the lists. This could be used for anything that will update inside a div. The other thing that we created is a pagination partial, which is just a string definition.

This is pretty obvious. This just uses the methods present in all will_paginate derived objects to determine which page is present. One thing that we found is that we don’t always have access to all the methods and attributes, which is why it looks rather odd.

Here’s the javascript code that does the pagination:

var prevPaginate = function(id)
{
  items = id.split('_')[1];
  current_page = id.split(’_')[2];
  next_page = parseInt(current_page) - 1;
  next_page_key = “#page_” + next_page;
  uri = parseMerbUri(window.location.href);
  good_uri = window.location.href.split(”#”)[0];
  next_page_key = good_uri + next_page_key;
  hist.add(next_page_key);
  query_string = uri + “?page=” + next_page;
  if(next_page > 0)
  {
    jQuery.get(query_string, function(data){eval(data);});
    return false;
  }
}
var nextPaginate = function(id)
{
  items = id.split(’_')[1];
  current_page = id.split(’_')[2];
  next_page = parseInt(current_page) + 1;
  uri = parseMerbUri(window.location.href);
  good_uri = window.location.href.split(”#”)[0];
  next_page_key = good_uri + “#page_” + current_page;
  hist.add(next_page_key);
  query_string = uri + “?page=” + next_page;
  jQuery.get(query_string, function(data){eval(data);});
  return false;
}

Of course, we have a method that parses the URI so that we are going to the right resource. This is important for nested resources to do this, since we want to display the correct results. We also use the Nitobi history in this case because we want the back button to use the pagination, and not go back to the previous page in the history.

I’m going to make this more generic in the next couple of weeks and post the code to the blog, however this should be rather trivial for anyone to do once they know how will_paginate works and how to use the json data that it sends back.

Posted in Ruby, UI | No Comments »

June 3rd, 2008

Recently, someone gave me some server space and root. I could do what I wanted to the server, as long as I kept the PHP and Drupal configurations working. While I at first was somewhat annoyed with it, being a fan of nginx and mongrel, I tried to install an Apache + Mongrel configuration, similar to what we deploy on other severs over and over again. The problem is that this was an Ubuntu server, therefore mod_php needs apache2-mpm-prefork to work, while mod_proxy needs apache2-mpm-worker. This will probably explain the oh-so-misleading error log:

[Mon Jun 02 18:01:26 2008] [warn] proxy: No protocol handler was valid for the URL /. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.

and the even worse

[Mon Jun 02 18:08:44 2008] [error] proxy: BALANCER: (balancer://mongrel_cluster). All workers are in error state


Of course, Google has told me that I wasn’t running the right modules, while Ubuntu’s documentation told me that I needed apache2-mpm-worker. After apt-getting the stuff, and realizing that the two are mutually exclusive, I decided to give passenger a try.

The setup is way simpler, but there are definitely some downsides to it. It appears that mod_rails is about as fast as mongrel so far (although I am not using it on a live environment), but when things go wrong, going through the trace stack is definitely a lot more painful. Also, another thing that I noticed is that with every deploy change, that I have to restart Apache now, which disrupts the other apps on the server. This is probably more of the case of getting your own box, but of course if I had my own box to play with, I wouldn’t be using Apache.

Overall, if you are stuck sharing a box with some Drupal users, Passenger is your friend, since you won’t have to explain why their PHP is now using FastCGI. It’s clearly the path of least resistance as far as some Rails deployments go. This also means though that things like Capistrano will have to change to reflect this. I guess this is my assignment to lazyweb. Go find Cap scripts that deploy to Passenger. :P

Posted in Linux, Rails, Ruby, drupal | No Comments »

May 20th, 2008

I’m currently working on a project in Merb, and I’m liking it. So far, I’ve just done some basic stuff, like added merbful_authentication, and simple things like that, but overall I’m finding that it’s not much different from rails. The main thing that I’m liking the most about it is getting rid of prototype. After the last couple of projects that I’ve worked on with Prototype, it felt like I only had a hammer and everything that I was looking to solve were nails. I was able to beat the crap out of it to get it to work, but it wasn’t as elegant as it should have been.

I may hack something together with Merb at Hackday, but I haven’t decided yet. I could just as easily play with Arduino again or try out Django or play with another Python app.

However, here’s the list of events that are interesting to me:

• May 24th - Nitobi Hackday the Third - At Nitobi HQ!
• May 26th - VanLUG presentation - From Embedded to Web, the Tech behind FreeTheNet
• May 29th to June 1st - Something in Portland (TBD!)
• June 2nd - van.rb presentation - From Embedded to Web, the Tech behind FreeTheNet

Posted in FtN, Linux, Rails, Ruby, VONIC | No Comments »

April 3rd, 2008

The last couple of weeks, I’m back spending 100% of my dev time working on Ubuntu, which is good, since I’m going to be presenting DogOnRails and WifiDog at Open Web Vancouver. I’m not sure what exactly I’ll be demoing at Open Web, but it will definitely be a refresh of the current DogOnRails interface at the very least. Hopefully, after tonight’s FreeTheNet meeting, we’ll have a good idea of what exactly we’ll be presenting. Depending on what goes down, it will definitely impact the VONIC presentation.

You can check the schedule here.

Also, the Vancouver Open Network Initiatives Cooperative has sent its paperwork into Victoria for incorporation. It’s going to be interesting to see what happens next.

Posted in Rails, Ruby, openwebvan | No Comments »

January 9th, 2008

I noticed that I’ve been referring to DogOnRails a lot in my Ruby Examples (because I don’t like exposing our clients’ code to the outside world if I can avoid it), so I better talk about what it is other than it being an abstract example I use here!

What is Dog On Rails?

DogOnRails is a WifiDog Captive Portal Authentication Server. Unlike the WifiDog server, it is written in Ruby on Rails (hence the name). It originally started out as a small hobby project so I could use it to run on a Linksys WRT54G at my apartment and have it redirect to my Dreamhost account. Then some other people saw that I was working on the project on Google Code and started to help me out with it. Then I started doing this Meraki FreeTheNet stuff, which gave me a new platform to put WifiDog client, and this was the perfect opportunity to de-mystify the whole process.

So, why is it cool?

The original concept of WifiDog is what drew me to it. The original WifiDog project was designed so that Captive Portals, which were only being used by WISPs at the time could be used to display both user-centric and location-centric information to the user so that they would learn more about the area around them. The reason I started working with DogOnRails was because I am able to add more features faster with Rails than I would be able to with PHP, and the fact that I dislike the Smarty Templating Engine, which made adding larger features a major hassle.

There is also the fact that the project seemed to cater more and more to the WISP community than its original purpose, so I felt the server no longer was interesting.

The thing that I like about it is the fact that this allows anyone who can hack Ruby on Rails to add more features onto something that normally would require that you write in C or Shell Script. By moving the authentication off the device, you can do much more with the process than simple authentication, you could redirect the user to what you want to see before they are on their way. And since they’re using YOUR bandwidth anyway. The analogy here is showing a visitor to your house around before they sit down and watch television. It’s not always necessary, but it’s a good courtesy for those who haven’t been to your house before.

Oh yeah, did I mention that it was voted the Best of HackDay 1. I think that bars it from competing in Hack Day 2. I have a whole new killer app for that. :P

So, what features did you add during HackDay?

I added the following features:

• User-Agent detection for mobile devices
• GoogleMaps Functionality
• GoogleEarth Functionality

The User-Agent detection was much easier than I thought. I borrowed Alexei’s iPhone to test the final design, and at the end I was able to get the UserAgent detection to show the view easily, the code looks like this:


user_agent = request.user_agent.downcase
mobile = false

The rest is pretty self-explanitory at this point! After that was added, it was just a matter of doing some iPhone-based CSS. I used Facebook as an example of how to do this, and after cursing WebKit/Safari’s existance, I managed to get something that didn’t require a LOT of resizing to get working.

Of course, the following features were added post-hackday:

• ROBIN/Open-Mesh Update Recieving - (Can’t update settings yet, can get the Mesh status)
• Improved GoogleMaps Functionality - Looks more like the Merkai Map
• Per Node Auditing
• Graphs using gruff
• MAC Address Blocking
• Facebook Functionality

That’s right, DogOnRails is now a Facebook application as well. The idea behind this is to encourage people to grow Wifi networks like they would grow their own garden. Make it so that there’s a certain level of pride for having the nodes up and working. It’s very, very early alpha stages, but it exists and it currently looks like this:

It needs a lot more polish, but it’s going to be used by the FreeTheNet group in the coming month, and will be the replacement to the Meraki Dashboard that we have been looking it. There will probably be more changes to this, and to other things like this. But, when I refer to DogOnRails, I’m referring to a real app, and not some abstract thing like in a textbook.

And I will keep using it as an example of what to do and what NOT to do for many posts to come.

Posted in FtN, Linux, Rails, Ruby, mesh | 1 Comment »

October 14th, 2007

I was talking to someone about a new rails project, and we asked about which server would be the best one to work on. Of course, me being a person who loves apt-get and hates RPM, I said one word.

Debian

But then I had to point him to a place to go to make things easier. The problem is that deploying a Rails app is rarely easy, especially one that actually is meant to not die. Not only that, but I don’t always agree with all the tutorials. So I’m going to write a series of blog posts about how I’m going to do it. I’d recommend reading this post when it comes to installing ruby, rails and gems on Debian/Ubuntu but forget any of that Capistrano/Apache stuff that they talk about. I recommend vlad an nginx for setting up and deploying rails apps. We’re going to talk about nginx in thsi post.

So, the first thing that is done when setting up a production server is to choose the webserver. Now, conventional wisdom says that when you’re running Linux, you will most likely use Apache. Conventional Wisdom is very wrong, since Apache is a giant 800lb Gorilla of a webserver that has more features than you’ll ever possibly need. It’s great if you want to load things like mod_php or mod_python (which you would do with django, but that’s the topic for another post), but it sucks if you want to use it for Ruby, since we’re going to be forwarding everything to the mongrels anyway.

So, what do we use? We’re going to use the Big Red Webserver from Russia, nginx. nginx is a nice http/reverse proxy server, with small, human readable files. The first thing that we’re going to do is install it on Debian. Sudo as root and do this:


apt-get install nginx


See, isn’t apt-get the coolest thing ever! Beats the crap out of yum! Anyway, what this just did was installed nginx, so in /etc/nginx, you are now going to have to delete your stock nginx file and create a new one. The first thing that you do is specify the user. It’s best to create a user for this such as www-data.


user www-data;
worker_processes 1;
error_log /var/log/nginx/error.log debug;


Note, we also set the log files. Now, we have to set some basic settings, such as the mime-type includes, the connections that we will accept, and gzipping your data. Simple, commonsense stuff. This begins the http configuration block:


http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_min_length 1100;
gzip_buffers 4 8k;
gzip_types text/plain;


OK, so far so good. Now, let’s specify some mongrel clusters. Depending on your app, you may want more or less clusters to balance the load. I’d ideally say at least 2 per processor, but sometimes you may want to run less of these for some weird reason. So, here’s what I have setup for a dual-processor machine.


upstream mongrel {
server 127.0.0.1:3000;
server 127.0.0.1:3001;
server 127.0.0.1:3002;
server 127.0.0.1:3003;
}


We’re going to show how to setup this in mongrel later. This is what we have currently. Now, we have to specify the server.


server {
listen 80;
server_name www.dogsridingrails.com;
root /var/www/dogonrails/current/public;
index index.html index.htm;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}


Not much to see here. We’re using nginx as a proxy to the mongrel servers. We point to public just like how we would in any rails application that’s going to production, and we specify what we do in the case of a filename request. In the case that we request index, and index exists, we show the index.html page. Otherwise, we pass it all to mongrel. Then we use a closing brace to finish the scope.


}


Now, that was MUCH simpler than the beasts of Apache logs that you’d have to wade through to do the same thing. It’s interesting to note that nginx is a lightweight proxying server, and is actually designed to do this, as opposed to Apache which is more general purpose, and is meant to load web apps using shared libraries which is always much faster than doing something like using mongrel.

I’m not saying that nginx is the right tool for every job, in fact, I would use think seriously about using Apache for a Python/Django project, but that’s the topic of another post entirely. Stay tuned for my next post about Vlad the Deployer!

Posted in General, Linux, Rails, Ruby | 2 Comments »

October 5th, 2007

In the life of a web application, there comes a point where that shared hosting account just isn’t good enough (and you found out because your provider kicked you off), or your server just isn’t able to pull the queries from the database fast enough. Then one day, you finally get the filesystem error EMLINK, which you have a VERY hard time googling.

This is simple, you just created the maximum number of subdirectories that you can have in a directory. This is suprisingly not a common issue with file_column, acts_as_attachhment or attachment_fu, although I’m shocked as why it’s not. So, what do you do when you’re faced with scalability issues, and you’re image handling plugin is broken!

THROW IT ALL AWAY!

That’s what I had to do. Recently we worked on a site and we decided that because it was getting too hammered, that we would put the images on S3. Then we found the ultimate weakness of S3, which is that it’s not able to easily handle batch processing. We used the AWS:S3 library for most of the movement of the files, but we found that if we made a mistake, it would cost us hours to get these back.

Eventually, the day was saved with jetS3t, and Cockpit. Using jetS3t, we were finally able to actually get through all the S3 issues, and it saved the day at the end. (Actually, Dave saved the day at the end, my computer kept running out of memory). But we managed to get S3 support into it, and all we had to do was sacrifice File Column and replace it with this:


def user_image=( blob )
# establish S3 connection
AWS::S3::Base.establish_connection!(:access_key_id => AWS_ACCESS_KEY_ID, :secret_access_key => AWS_SECRET_ACCESS_KEY)
datestamp = Time.now.strftime(’%d%m%Y’)
identifier = UUID.random_create.to_s
object_path = “images/” + datestamp + ‘/’ + identifier + ‘/’
object_key = object_path + blob.original_filename
self.image = blob.original_filename
self.image_dir = ‘http://s3.amazonaws.com/bucket/images/’ + datestamp + ‘/’ + identifier + ‘/’
image_data = blob.read

AWS::S3::S3Object.store(thumb_key, thumbnail.to_blob , ‘bucket’, :access => :public_read)
end


However, if you have to do S3, I would highly recommend using a long key so that you can sort your re.sults better based on this key! However, the biggest gotcha I found when adding S3 integration to my rails app was including AWS/S3. If you include and require it, it will break your routing, this is something that can cause hours of headaches, especially if you are doing something else. At the end, we learned that S3 is a misnomer. For a large number of files, it’s far from simple.

Posted in Linux, Rails, Ruby, Uncategorized | 1 Comment »