Joe@Nitobi

June 3rd, 2008

Recently, someone gave me some server space and root. I could do what I wanted to the server, as long as I kept the PHP and Drupal configurations working. While I at first was somewhat annoyed with it, being a fan of nginx and mongrel, I tried to install an Apache + Mongrel configuration, similar to what we deploy on other severs over and over again. The problem is that this was an Ubuntu server, therefore mod_php needs apache2-mpm-prefork to work, while mod_proxy needs apache2-mpm-worker. This will probably explain the oh-so-misleading error log:

[Mon Jun 02 18:01:26 2008] [warn] proxy: No protocol handler was valid for the URL /. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.

and the even worse

[Mon Jun 02 18:08:44 2008] [error] proxy: BALANCER: (balancer://mongrel_cluster). All workers are in error state


Of course, Google has told me that I wasn’t running the right modules, while Ubuntu’s documentation told me that I needed apache2-mpm-worker. After apt-getting the stuff, and realizing that the two are mutually exclusive, I decided to give passenger a try.

The setup is way simpler, but there are definitely some downsides to it. It appears that mod_rails is about as fast as mongrel so far (although I am not using it on a live environment), but when things go wrong, going through the trace stack is definitely a lot more painful. Also, another thing that I noticed is that with every deploy change, that I have to restart Apache now, which disrupts the other apps on the server. This is probably more of the case of getting your own box, but of course if I had my own box to play with, I wouldn’t be using Apache.

Overall, if you are stuck sharing a box with some Drupal users, Passenger is your friend, since you won’t have to explain why their PHP is now using FastCGI. It’s clearly the path of least resistance as far as some Rails deployments go. This also means though that things like Capistrano will have to change to reflect this. I guess this is my assignment to lazyweb. Go find Cap scripts that deploy to Passenger. :P

Posted in Linux, Rails, Ruby, drupal | No Comments »

May 20th, 2008

I’m currently working on a project in Merb, and I’m liking it. So far, I’ve just done some basic stuff, like added merbful_authentication, and simple things like that, but overall I’m finding that it’s not much different from rails. The main thing that I’m liking the most about it is getting rid of prototype. After the last couple of projects that I’ve worked on with Prototype, it felt like I only had a hammer and everything that I was looking to solve were nails. I was able to beat the crap out of it to get it to work, but it wasn’t as elegant as it should have been.

I may hack something together with Merb at Hackday, but I haven’t decided yet. I could just as easily play with Arduino again or try out Django or play with another Python app.

However, here’s the list of events that are interesting to me:

• May 24th - Nitobi Hackday the Third - At Nitobi HQ!
• May 26th - VanLUG presentation - From Embedded to Web, the Tech behind FreeTheNet
• May 29th to June 1st - Something in Portland (TBD!)
• June 2nd - van.rb presentation - From Embedded to Web, the Tech behind FreeTheNet

Posted in FtN, Linux, Rails, Ruby, VONIC | No Comments »

April 23rd, 2008

I’m certain by now that we’ve heard throughout the blogosphere about what happened to Blaine at Twitter. Regardless of the cries of “Rails doesn’t scale”, it seems that this is the biggest thing to hit the Rails community since Zed’s pointless rant. Normally I don’t pay attention to most of the blogosphere, however since I know Blaine personally, and he’s someone who I can honestly say that I’ve looked up to over the past couple of years, as both a developer and as a friend, I think that the personal attacks based on a presentation where as a representative of a company, it is expected that you put a positive spin on what you are working on, this “He said scaling was easy! Twitter is DOWN!” nonsense is uncalled for.

The thing that I find with this speculation is that we don’t know what happened to Twitter. I haven’t spoken to Blaine personally in years, but I’m glad to see that other people are calling Michael Arrington out on this article, and that this is better left to Valleywag and other blogs that I don’t bother reading, which TechCrunch has joined the ranks of. Since this happened a couple of weeks ago, I’m suprised that none of our mutual friends up here has heard that he had left Twitter. However, I wish him the best, and he is definitely one of the smartest people that I have probably ever talked to about tech in this town. It makes me sad at the state of the industry, even when it’s gossip, to see this sort of story pass as a critique.

Posted in Rails, professionalism | No Comments »

April 3rd, 2008

The last couple of weeks, I’m back spending 100% of my dev time working on Ubuntu, which is good, since I’m going to be presenting DogOnRails and WifiDog at Open Web Vancouver. I’m not sure what exactly I’ll be demoing at Open Web, but it will definitely be a refresh of the current DogOnRails interface at the very least. Hopefully, after tonight’s FreeTheNet meeting, we’ll have a good idea of what exactly we’ll be presenting. Depending on what goes down, it will definitely impact the VONIC presentation.

You can check the schedule here.

Also, the Vancouver Open Network Initiatives Cooperative has sent its paperwork into Victoria for incorporation. It’s going to be interesting to see what happens next.

Posted in Rails, Ruby, openwebvan | No Comments »

January 9th, 2008

I noticed that I’ve been referring to DogOnRails a lot in my Ruby Examples (because I don’t like exposing our clients’ code to the outside world if I can avoid it), so I better talk about what it is other than it being an abstract example I use here!

What is Dog On Rails?

DogOnRails is a WifiDog Captive Portal Authentication Server. Unlike the WifiDog server, it is written in Ruby on Rails (hence the name). It originally started out as a small hobby project so I could use it to run on a Linksys WRT54G at my apartment and have it redirect to my Dreamhost account. Then some other people saw that I was working on the project on Google Code and started to help me out with it. Then I started doing this Meraki FreeTheNet stuff, which gave me a new platform to put WifiDog client, and this was the perfect opportunity to de-mystify the whole process.

So, why is it cool?

The original concept of WifiDog is what drew me to it. The original WifiDog project was designed so that Captive Portals, which were only being used by WISPs at the time could be used to display both user-centric and location-centric information to the user so that they would learn more about the area around them. The reason I started working with DogOnRails was because I am able to add more features faster with Rails than I would be able to with PHP, and the fact that I dislike the Smarty Templating Engine, which made adding larger features a major hassle.

There is also the fact that the project seemed to cater more and more to the WISP community than its original purpose, so I felt the server no longer was interesting.

The thing that I like about it is the fact that this allows anyone who can hack Ruby on Rails to add more features onto something that normally would require that you write in C or Shell Script. By moving the authentication off the device, you can do much more with the process than simple authentication, you could redirect the user to what you want to see before they are on their way. And since they’re using YOUR bandwidth anyway. The analogy here is showing a visitor to your house around before they sit down and watch television. It’s not always necessary, but it’s a good courtesy for those who haven’t been to your house before.

Oh yeah, did I mention that it was voted the Best of HackDay 1. I think that bars it from competing in Hack Day 2. I have a whole new killer app for that. :P

So, what features did you add during HackDay?

I added the following features:

• User-Agent detection for mobile devices
• GoogleMaps Functionality
• GoogleEarth Functionality

The User-Agent detection was much easier than I thought. I borrowed Alexei’s iPhone to test the final design, and at the end I was able to get the UserAgent detection to show the view easily, the code looks like this:


user_agent = request.user_agent.downcase
mobile = false

The rest is pretty self-explanitory at this point! After that was added, it was just a matter of doing some iPhone-based CSS. I used Facebook as an example of how to do this, and after cursing WebKit/Safari’s existance, I managed to get something that didn’t require a LOT of resizing to get working.

Of course, the following features were added post-hackday:

• ROBIN/Open-Mesh Update Recieving - (Can’t update settings yet, can get the Mesh status)
• Improved GoogleMaps Functionality - Looks more like the Merkai Map
• Per Node Auditing
• Graphs using gruff
• MAC Address Blocking
• Facebook Functionality

That’s right, DogOnRails is now a Facebook application as well. The idea behind this is to encourage people to grow Wifi networks like they would grow their own garden. Make it so that there’s a certain level of pride for having the nodes up and working. It’s very, very early alpha stages, but it exists and it currently looks like this:

It needs a lot more polish, but it’s going to be used by the FreeTheNet group in the coming month, and will be the replacement to the Meraki Dashboard that we have been looking it. There will probably be more changes to this, and to other things like this. But, when I refer to DogOnRails, I’m referring to a real app, and not some abstract thing like in a textbook.

And I will keep using it as an example of what to do and what NOT to do for many posts to come.

Posted in FtN, Linux, Rails, Ruby, mesh | 1 Comment »

October 14th, 2007

I was talking to someone about a new rails project, and we asked about which server would be the best one to work on. Of course, me being a person who loves apt-get and hates RPM, I said one word.

Debian

But then I had to point him to a place to go to make things easier. The problem is that deploying a Rails app is rarely easy, especially one that actually is meant to not die. Not only that, but I don’t always agree with all the tutorials. So I’m going to write a series of blog posts about how I’m going to do it. I’d recommend reading this post when it comes to installing ruby, rails and gems on Debian/Ubuntu but forget any of that Capistrano/Apache stuff that they talk about. I recommend vlad an nginx for setting up and deploying rails apps. We’re going to talk about nginx in thsi post.

So, the first thing that is done when setting up a production server is to choose the webserver. Now, conventional wisdom says that when you’re running Linux, you will most likely use Apache. Conventional Wisdom is very wrong, since Apache is a giant 800lb Gorilla of a webserver that has more features than you’ll ever possibly need. It’s great if you want to load things like mod_php or mod_python (which you would do with django, but that’s the topic for another post), but it sucks if you want to use it for Ruby, since we’re going to be forwarding everything to the mongrels anyway.

So, what do we use? We’re going to use the Big Red Webserver from Russia, nginx. nginx is a nice http/reverse proxy server, with small, human readable files. The first thing that we’re going to do is install it on Debian. Sudo as root and do this:


apt-get install nginx


See, isn’t apt-get the coolest thing ever! Beats the crap out of yum! Anyway, what this just did was installed nginx, so in /etc/nginx, you are now going to have to delete your stock nginx file and create a new one. The first thing that you do is specify the user. It’s best to create a user for this such as www-data.


user www-data;
worker_processes 1;
error_log /var/log/nginx/error.log debug;


Note, we also set the log files. Now, we have to set some basic settings, such as the mime-type includes, the connections that we will accept, and gzipping your data. Simple, commonsense stuff. This begins the http configuration block:


http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_min_length 1100;
gzip_buffers 4 8k;
gzip_types text/plain;


OK, so far so good. Now, let’s specify some mongrel clusters. Depending on your app, you may want more or less clusters to balance the load. I’d ideally say at least 2 per processor, but sometimes you may want to run less of these for some weird reason. So, here’s what I have setup for a dual-processor machine.


upstream mongrel {
server 127.0.0.1:3000;
server 127.0.0.1:3001;
server 127.0.0.1:3002;
server 127.0.0.1:3003;
}


We’re going to show how to setup this in mongrel later. This is what we have currently. Now, we have to specify the server.


server {
listen 80;
server_name www.dogsridingrails.com;
root /var/www/dogonrails/current/public;
index index.html index.htm;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://mongrel;
break;
}
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}


Not much to see here. We’re using nginx as a proxy to the mongrel servers. We point to public just like how we would in any rails application that’s going to production, and we specify what we do in the case of a filename request. In the case that we request index, and index exists, we show the index.html page. Otherwise, we pass it all to mongrel. Then we use a closing brace to finish the scope.


}


Now, that was MUCH simpler than the beasts of Apache logs that you’d have to wade through to do the same thing. It’s interesting to note that nginx is a lightweight proxying server, and is actually designed to do this, as opposed to Apache which is more general purpose, and is meant to load web apps using shared libraries which is always much faster than doing something like using mongrel.

I’m not saying that nginx is the right tool for every job, in fact, I would use think seriously about using Apache for a Python/Django project, but that’s the topic of another post entirely. Stay tuned for my next post about Vlad the Deployer!

Posted in General, Linux, Rails, Ruby | 2 Comments »

October 5th, 2007

In the life of a web application, there comes a point where that shared hosting account just isn’t good enough (and you found out because your provider kicked you off), or your server just isn’t able to pull the queries from the database fast enough. Then one day, you finally get the filesystem error EMLINK, which you have a VERY hard time googling.

This is simple, you just created the maximum number of subdirectories that you can have in a directory. This is suprisingly not a common issue with file_column, acts_as_attachhment or attachment_fu, although I’m shocked as why it’s not. So, what do you do when you’re faced with scalability issues, and you’re image handling plugin is broken!

THROW IT ALL AWAY!

That’s what I had to do. Recently we worked on a site and we decided that because it was getting too hammered, that we would put the images on S3. Then we found the ultimate weakness of S3, which is that it’s not able to easily handle batch processing. We used the AWS:S3 library for most of the movement of the files, but we found that if we made a mistake, it would cost us hours to get these back.

Eventually, the day was saved with jetS3t, and Cockpit. Using jetS3t, we were finally able to actually get through all the S3 issues, and it saved the day at the end. (Actually, Dave saved the day at the end, my computer kept running out of memory). But we managed to get S3 support into it, and all we had to do was sacrifice File Column and replace it with this:


def user_image=( blob )
# establish S3 connection
AWS::S3::Base.establish_connection!(:access_key_id => AWS_ACCESS_KEY_ID, :secret_access_key => AWS_SECRET_ACCESS_KEY)
datestamp = Time.now.strftime(’%d%m%Y’)
identifier = UUID.random_create.to_s
object_path = “images/” + datestamp + ‘/’ + identifier + ‘/’
object_key = object_path + blob.original_filename
self.image = blob.original_filename
self.image_dir = ‘http://s3.amazonaws.com/bucket/images/’ + datestamp + ‘/’ + identifier + ‘/’
image_data = blob.read

AWS::S3::S3Object.store(thumb_key, thumbnail.to_blob , ‘bucket’, :access => :public_read)
end


However, if you have to do S3, I would highly recommend using a long key so that you can sort your re.sults better based on this key! However, the biggest gotcha I found when adding S3 integration to my rails app was including AWS/S3. If you include and require it, it will break your routing, this is something that can cause hours of headaches, especially if you are doing something else. At the end, we learned that S3 is a misnomer. For a large number of files, it’s far from simple.

Posted in Linux, Rails, Ruby, Uncategorized | 1 Comment »

July 5th, 2007

Even though we’re doing a lot of work in Ruby on Rails, and we blog about it a lot, we don’t sell a back-end for it. So the question that people may ask is how do we use our Components with Rails?

Here’s some information on how I’m using Rails with the current version of Grid. In Rails, you can use Builder to create XML, so the first thing that I am going to create is a grid controller. In that grid controller, I define a getHandler method, which looks like this:


def getHandler
if params[:SortColumn].nil?
@tblcustomer_pages, @tblcustomers = paginate :tblcustomers, :per_page => params[:PageSize].to_i
else
@tblcustomer_pages, @tblcustomers = paginate :tblcustomers, :per_page => params[:PageSize].to_i , :order => params[:SortColumn] + ” ” + params[:SortDirection]
end
render :layout => false
end


What this does is use the paginate function to do some pagination of the table. We set the PageSize using the Nitobi Grid’s PageSize paramater, and we use the SortColumn and the SortDirection coming into the function from the grid to determine the column sorting.

I then have a view (rxml) that looks like this:


xml.instruct! :xml, :version => “1.0″
xml.root(:fields => “CustomerName|ContactName”, :keys => “CustomerName|ContactName”) {
for tblcustomer in @tblcustomers
xml.e(:a => tblcustomer.CustomerName,:b =>tblcustomer.ContactName, :xk => tblcustomer.CustomerID)
end
}

This is very basic. I’m just starting out with builder, so I don’t have the Error Handler here. I suppose that I should have that there, but I don’t know what will happen if its nil. (I suspect it will break). That’s definitely something I’m going to have to ask around about.

And, I make sure that I have the getHandler set in the Grid Definition like this:


gethandler=”getHandler/”


Of course, I make sure to include grid and toolkit. Once this is all done, I have a grid that can load. I still have to get the savehandler working on Rails, but this should be enough to get started with Rails. If I find any interesting interactions with Grid, Toolkit and Prototype, I’ll post them here.

Posted in Grid, Rails, UI | 1 Comment »

July 4th, 2007

Well, this weekend I had to write some Rails stuff. Once I did the work stuff that I had to do on Saturday morning while I’m sure some kids watched some cartoons. I then decided to revisit something that I used to contribute to.

The result of a couple hours of hacking is this:

http://code.google.com/p/dogonrails/

That’s right. DogOnRails! I used to advocate running WifiDog hotspots back in the day. The problem that I had with Wifidog is that it was hard to run unless you had a dedicated server running Postgres. There was no way to run it on Shared hosting for just your one hotspot. So, I decided to hack this thing together.

It’s pretty much just written for my old WRT54G running OpenWRT, but I might do something interesting with it. It’s under GPLv2, so check it out!

Posted in Linux, Rails | No Comments »

June 25th, 2007

I really like doing Rails work, but if there is one pet peeve that I have, it’s moving between Ruby on Rails versions. There are a fair number of Ruby apps out in the wild, and when handed a Rails app to start maintaining, you need to move from version to version. Of course, what’s interersting is how to check for the different rails versions. For example, some people think doing this will work:


rails -v


That does work, IF you are not using the vendor/rails version of rails. However, this means that you have to know what rails version you’re developing for. This becomes a major pain for certain tags that are simply ignored in Rails 1.1 and are rendered in Rails 1.2. Days can be done developing in Rails 1.2 before deploying on another site, and if you don’t know that you’re running Rails off of vendor/rails, this could be a major problem.

So, how do you determine which rails you’re running?

One way to do it is to go into the console, and run this:

Rails::VERSION::STRING


This will load what the application environment sees as its version of rails, which is much more useful than the previous command. However, if you run this command:


script/about


You’ll see the Ruby version of phpinfo(), which will give you all the information that you need to rock and roll. There are cases where you may want to downgrade to a prior version, namely if you’re using a plugin that requires a certain syntax in a previous version. However, there’s arguments for and against this behaviour. I would always use the latest version of rails, mainly because it’s easier to give to someone else who’s not as experienced with Rails, and it’s better to have them learn how to use the latest version as opposed to trying to find documentation for previous versions of rails.

So far, I have to say that going between versions of rails is one of the most frustrating things that I’ve found so far with Ruby on Rails.

Posted in Rails | No Comments »