Joe@Nitobi

January 12th, 2009

Recently, Nitobi got an Android Dev Phone, which is a copy of the T-Mobile G1 phone that Brock got earlier in the year. I’m currently using it as my primary phone while I figure out whether I should get an Android Dev Phone or the OpenMoko. I’m still leaning towards Android, but I have noticed that there are a few major usability and other bugs that I’ve found while being a casual user, and not a dev.

If the Android phones are going to beat the iPhone, they have to do everything the iPhone does. That means that they have to be able to sync with my computer. Now, the Google G1 is a dumb device as far as this is concerned, and it acts as an SD card reader. I originally thought that this was awesome, and copied my music to my 4GB MicroSD card that I put in the phone. However, I also decided to install another application, and BOOM. SD Card is Unreadable.

The Android Phone for some reason handles the mounting in software, and can’t deal with IO from the phone and copying files at the same time. If this is the case, it should lock the user out of the touch screen with a USB Connected screen, keeping functionality for an emergency phone only, BUT instead we can access most of the features. Only the RingDroid application is able to detect this operation. I’m most likely going to add this check to the Android version of PhoneGap, as soon as we get some serious I/O features. I was lucky and I was able to reformat the device with my digital camera. A lot of people won’t realize that it needs to be reformatted and will throw away bad storage. This can get fairly costly, since 4GB and 8 GB Micro SD cards aren’t disposable yet.

This seems to me like it’s usability fail. I’m going to load the new Cupcake Build on the device with the Multitouch patch on the Java Stack in a couple of weeks to see if the new build fixes the issue. I definitely consider this a major bug, and I’ll probably file this if it’s not resolved in trunk.

Other than this, (and the Battery issue that’s been blogged about EVERYWHERE), I’m pretty happy with the performance of the phone, and I’m hoping that Android can take off and be a challenger against Windows Mobile for all the phones that both techs and non-techs use.

*Update* - After I first posted, I found the bugtracker. I realize that I have to upgrade the device to see if this bug still exists. I predict it does.

Posted in Uncategorized | No Comments »

January 6th, 2009

Recently, for client projects and for our own Complete UI code, we have been looking more and more towards selenium testing for functional testing of the application. However, it’s interesting to see how different people approach testing, that it seems that people forget the differences between what a Unit Test is and what a Functional Test is.

A unit test is where you test a method for a desired result. This is where each method is tested to make sure that it produces the same expected result each time. When I did QA earlier, I heard developers tell me that the main purpose of Unit Testing was just to test banking software to make sure that it produced the proper result, and that because it wasn’t banking software, it didn’t matter. This isn’t true at all, and in fact Unit Testing should be done at the beginning of a project. This is why I am partial to rspec, since it keeps things clean, and allows me to properly think about what I am trying to do with my code before I go off and do it. This saves a lot of time at the end, where looks matter.

Then there is functional testing. Normally this is testing that doesn’t get done, or testing that would traditionally be done by your beta testers or some other form of unforgiving users who are not going to use your app because it does that stupid thing where it deletes the values of the fields and doesn’t notify them whether their data was accepted. This is what selenium is for. Selenium is software that fills this gap, since it allows you to test the interface before and after it’s styled and it forces you to think about making your site easy not only for you but for other people and robots to navigate as well.

The reason I mention robots is beacuse that’s what Selenium is. It will not be able to beat the captcha, and you will be stuck testing certain things yourself. However, if you wrote unit tests for your authentication or comment systems, you would know that it does work, and this wouldn’t necessarily be where you are looking to do functional testing.

When doing functional testing, the best thing to do is to try to take a black-box approach. The questions that you need to ask is this:

• What are you testing?
• Why are you testing this?
• Is this the most efficient way to test this?

What are you testing?

This is important. Knowing the goal of the test is part of the battle. If you have to make sure that a site is serving something that the browser will use that you don’t have access to in Selenium, like browser add-ons, then you need to re-assess how you are testing it. For example, in some cases, I write code in Python using only unittest, the Expat XML parser, and httplib so that I can make sure it’s
serving XML files correctly that would otherwise be used by the web browser (or Google Earth KML).

If we are using a browser that can read KML files, it should read the header and the header should have the right mime type. We don’t test for that here, but we do test to make sure that we are
returning something that can be parsed as XML instead of bad data that can’t be parsed. This perhaps isn’t the best example of this practice, but is a good idea.

Remember that you are testing the site, and that you are NOT testing the browser. It’s possible that the browser has a bug, but in that case, it’s probably a good idea to maintain a level of browser agnosticism, and that any browser-specific stuff should be tested using jsUnit.

Why are you testing this?

Occassionally there will be tests that make no sense. They may test nothing, or they may be so trivial that testing them may be pointless, and would inflate your test results. This could also be something that has been unit tested or has been tested already. Doing the same thing over and over and expecting a different result is insane, and it’s important that we keep a level of sanity. Therefore, repeating ones self when writing tests should be avoided and if you find that a step is being repeated in numerous tests, it’s probably a good idea to abstract that out into its own test and to move up that test’s priority since this test failing will cause all tests that depend on this step to fail as well.

Is there a better way to do this?

Selenium is supposed to be a substitute for point and click testing. However, it’s a robot that will only do what you tell it to. It should not know what is on the server side, and it should not be able to get past systems that verify whether you are a robot or not, such as Captchas. This is where manual testing should happen.

Writing tests isn’t as much fun as writing code. The more time that you spend writing tests, the less time you spend writing code. However, the more tests you write, the better your code tends to get. Here are some tips to ease the pain of selenium testing:

Try to use Firefox and Safari for testing as much as possible

You are testing a web application. Selenium can not tell whether a class looks right when it’s styled a certain way. It is not subjective, it is objective. The reason that you should use Firefox or Safari for the primary browser to be tested for functional tests is the fact that these browsers have getElementsByClassName. This is probably the most useful Javascript method for gathering elements to be checked. Have a collection of elements in a table and you need to verify that they all showed up? Getting the length of the collection that this method gets is the easiest way to do it. Need to make sure that certain fields are displayed or not displayed? Checking to see if this method isn’t nil will work. This will make life so much simpler for testing.

The lack of getElementsByClassName is Internet Explorer rather irritating, and you may want to add it yourself, either with an existing framework, or a homebrew hack. However as far as testing goes, this may cause bigger problems than you think, since your version of getElementsByClassName may contaminate other JS code and cause weird results that may not actually be errors but may be caused by you trying to be clever. Don’t do this! If the app uses a framework that does this, it’s fine, but otherwise try to avoid adding things that could lead to false positives.

Elements that need to be verified should have IDs or classes assigned to them

A lot of functional testing requires navigating the DOM. The less crawling through the DOM the test does, the faster the result can be arrived at, and the sooner we can get the tests done so that we can
get the actual code fixed. This also makes the code MUCH easier to be styled if the UI is deemed ugly or wrong. This also allows you to automate IE testing easier so it’s easier to find elements using xpath or by using the DOM.

Keep the tests short. One Test per Class:

Python is full of many different test runners of various quality. If you need to write a log of what was done, it’s a good idea to make sure that they are broken down this way, otherwise no tests will be run and you will have an empty file. Also, in python there are many different runners, such as XmlTestRunner, HTMLTestRunner and TextTestRunner that work this way. I recommend XmlTestRunner for Python because of it’s compatibility with CruiseControl. This makes it easier to have Java-based tests and Python based tests to be on the same machine.

Use Selenium IDE as a guideline only:

Selenium IDE is a great add-on to Firefox. The thing is that most of the time the Python/Java/Ruby/whatever that it produces is not suitable for anything past the most basic tests. This is painfully obvious if you have something like the Nitobi Grid where IDs are dynamically generated and are progammatically selected. If you rely on this without the ability to seriously customize the tests in Python or Java, you will hit a wall hard.

Have Patience:

Selenium tests tend to take over the computer you test it on as far as workspace. This is fine. I run my selenium tests in a VM so that I can do other things and let Selenium have free reign on my browsers and screen space. If not, and you’re testing a batch of processes, then I recommend walking off for a while and coming back. The reason they call this Selenium RC is because of the remote control functionality and the fact that this automation can eventually be done when you are not in the office, usually during a build, or at some point throughout the day.

Remember that it’s important to be rigorous when doing Functional Tests, and that it’s important to test real values. I highly recommend the ID and class advice since this will more likely not break your tests if minor design and styling gets changed. Of course, this is just advice, and this is not a magic bullet to fix your app. Take whichever advice you feel is the most applicable to your situation.

Posted in Uncategorized | No Comments »

November 20th, 2008

Currently, it is possible to get our components to play nicely with jQuery, but that required that jQuery.noConflict() be used. While that is a good plan regardless, we realize the pain it is to go through all the Javascript and remove the $ and replace with with some other variable such as $ntb. So, we decided to do it for you.

Currently on the nitobi github, there is an experimental branch that is called jcui. This is a branch that we are testing against the jQuery Javascript Framework to make sure that it doesn’t matter where you put your jQuery, the components should still work.

If people want to play with the jCUI branch and test it out, they can do so by going to the same URL as they would for the other github projects:

git clone git://github.com/nitobi/completeui.git

Here’s a VERY rough Wiki page I threw up for getting started with using the Complete UI source code for development. This is a very good way to help fix bugs, since it can turn things like:

nitobi.callout.Callout.getStyleSheetUrl=function(){
  if(nitobi.callout.Callout.globalStylesheet==null){
    var _33=nitobi.html.Css.getRule("ntb"+nitobi.callout.lastStyle);
    nitobi.callout.Callout.globalStylesheet=nitobi.html.Css.getPath(_33.parentStyleSheet);
    return nitobi.callout.Callout.globalStylesheet;
  }else{
    return nitobi.callout.Callout.globalStylesheet;
  }

to look more like:

nitobi.callout.Callout.getStyleSheetUrl = function()
{
	if (nitobi.callout.Callout.globalStylesheet	== null)
	{
		var rule = nitobi.html.Css.getRule('ntb' + nitobi.callout.lastStyle);
		nitobi.callout.Callout.globalStylesheet = nitobi.html.Css.getPath(rule.parentStyleSheet);
		return nitobi.callout.Callout.globalStylesheet;
	}
	else
	{
		return nitobi.callout.Callout.globalStylesheet;
	}
}

Also, if anyone thinks this is interesting, please let us know on the Google Group.

Posted in Uncategorized | No Comments »

October 22nd, 2008

From Make Blog: What is your definition of Open Source Hardware?

The discussion is very interesting, and is relevant to those who want to know all the business ins and outs of Open Source. The thing that is very interesting is the whole non-commercial aspect. In the article that was mentioned above, people used a Creative Commons - Non-Commercial - Share Alike License for the project and because of the licence, LadyAda mentioned that it could not be Open Source Hardware. This seemed to have become an issue because people are afraid of people profiteering off of their resources and credit not being given.

The thing is that all Open Source licences protect from profiteering. The granddaddy of them all, and the one that Nitobi uses is the GPL. The GPL is really putting everything out there. What is says is that as long as we distribute code, you will be able to do the following:

• Freedom 0: The freedom to run the program for any purpose.
• Freedom 1: The freedom to study and modify the program.
• Freedom 2: The freedom to copy the program so you can help your neighbor.
• Freedom 3: The freedom to improve the program, and release your improvements to the public, so that the whole community benefits.

It doesn’t say anything about who can use it. It says the public, which would mean that anyone can use it. If you make it non-commercial, it means that Freedom 3 is gone. From an idealistic point of view, it may sound good, but in reality, it shoots you in the foot, and dooms the project to be forever a side-project and limits the opportunities for people to participate. However, what would be best practice for this case is to dual license your project under a more permissive licence. This is what Mozilla does with Firefox. Despite what people like RMS would tell you, Dual-Licensing code is a good idea, and it allows you to have dialog with the people who are using your code, whether they agree with the goals of Free and Open Source Software or not.

Still, it’s interesting seeing Open Source Hardware have these discussions, since it seems that in software, they’ve gotten rather entrenched as of late.

Posted in Uncategorized | 1 Comment »

October 17th, 2008

Recently, I’ve been looking at PhoneGap. However, unlike most people at Nitobi, I am using a PC. However, I spend most of my time running Ubuntu Hardy Heron on my dual-boot system. This means that unless I wanted to jailbreak a phone, there’s really no way that I could do iPhone Development. However, the other day I decided to start taking a look at the Android SDK when the G1 was announced. It’ll be a while since we’ll get any Android phones in Canada (for more info, read this), it’ll be good for the rest of the planet to have the ability to write once and run anywhere. Sort of like what Dave said.

One thing that is interesting is that Google embraced the idea of using HTML and Javascript inside of their apps and they have their own demo apps that use this. This definitely made it a lot easier to write the app:

Of course, I’m currently relying on the Emulator and the Eclipse plugins to test this, which is different from the real world, however if someone wants to help make the world of Javascript-based Mobile Applications better, it would be great if we had a real device of some sort that we could use for this. ;)

This also hasn’t been merged into the official repo yet, and currently lives on my GitHub: https://github.com/bowserj/gap/ It’s been a while since I’ve hacked on Java, so the code is probably cruftier than it needs to be. It’d be interesting to see where this platform goes.

Posted in Uncategorized | No Comments »

October 15th, 2008

I just got back from MerbCamp 2008, and now I’m writing a blog entry on my slow PHP WordPress application. I can say this with confidence because of the fact that every time this blog loads, it has to re-parse and re-interpret the PHP, which pulls from the database the blog entries that I’m using. The reason I’m mentioning this is because of the closing keynote.

In the closing keynote, Yehuda Katz brought up a very important point, which is that Fibonacci Numbers and Matrix Multiplication are not necessarily good benchmarks for language to be measured on as far as building a web framework. When people think that Ruby is slow, it is based on the fact that during the Programming Language shootout, it does poorly. It is tested classically against other languages with the same general purpose tasks that define a language. This means that instead of testing frameworks, and extensions and adapters, it’s testing pure Ruby against pure C, pure Java, pure C#, pure Python and pure PHP (Whatever that means).

However, given the fact that a LOT of the gems used in running a framework are natively compiled in C and work with Ruby this means that a lot of the things that Ruby is inherently bad at can be mitigated. It’s true that Ruby is not a magic bullet, nor is php or any other language. The fact of the matter is that Ruby is not as slow as people make it out to be.

On another note, the best thing about the conference IMO is the fact that it was webcast and archived. This is great because I missed part of the Sunday morning presentation about Deploying Merb. Right now, I really want to find a Web-Based Service where I can just create a ton of Virtual Machines that I can use for staging. I know that we have spent countless hours here setting up servers using Apache, Nginx, Mongrel as well as personally with mod_rails and mod_rack. If that could be automated every time we need another Rails or Merb solution, that would be a major epic win for us. I know that EngineYard has come close, but then again I’m a server geek who likes to be able to have the ability to tweak everything, but doesn’t have the time.

Another thing that was cool was the IRC. I spend a LOT of time on IRC. I used to practically live on #openwrt back when I was doing hacking on the Linksys router, and on #wireless. These days I can be found in #freegeek-van, #merb, #arduino, #vhs and in #nitobi on FreeNode. Yes, I said #nitobi. (I registered #nitobi for the official Nitobi channel on FreeNode, but more on this at a later time.) Anyway, Brian and Rob had to get back onto IRC at the sprint. It’s hard to keep track of everything that goes on, but IRC definitely has stood the test of time for one to many chat, and for getting help fast. The coolest thing about the IRC was the end where everyone used IRC to ask questions of the Merb team. The reason for this is because MerbCamp was webcasted and people from around the world were asking questions. Apparently this is commonplace at various conference, but it’s the first time I’ve experienced it professionally, which is rather cool.

Overall, it was a cool conference, and I’m looking forward to going next year.

Posted in Uncategorized | 1 Comment »

October 8th, 2008

Recently, I’ve been playing with the Android SDK. The main thing that has me pumped about Android is the fact that it’s open and the fact that I can actually write code for it without having to get a Mac and run Mac OS X. I’ve been messing around with it a couple weeks, and I found that there wasn’t a good set of code samples for developing with Android.

So, here’s a rough description of what you need to do to get a WebView to actually be useful in Android. First of all, you should already be familiar with the basics, since we’re not going to be spending too much time on them here. You need to use the following in the layout:

            < WebView android:id="@+id/appView"
            android:layout_height="wrap_content"
            android:layout_width="fill_parent" />

This should be in a layout element in your XML Layout. I use the Linear Layout similar to the tutorial, but you may want to use a different layout. Then, I started using it with the following Java code:

WebView appView;
appView = (WebView) findViewById(R.id.appView);
appView.getSettings().setJavaScriptEnabled(true);
appView.loadUrl("http://www.nitobi.com");

This code should be put in the onCreate method of the activity, since this will be loaded at the beginning of the activity. This is the most basic way to make sure that you get a full running browser. It’s something that I didn’t find straight forward from the API docs, so I figured that I’d pass this thing on. It’ll be a while since I’ll be getting one of these phones, since I’m in Canada, but I have a feeling that this will be coming up again very soon.

Posted in Uncategorized | 3 Comments »

September 26th, 2008

Recently, we had a Merb project that we needed to add search to. Now, since Merb doesn’t use plugins the same way Ruby does, I decided to look for a nice easy gem to make life easier. At the time, I found ultraminx. Ultraminx was basically ultrasphinx made into a gem with no docs. Things were good, except that it broke when I decided to upgrade to the latest version of Merb.

The problem that I found is that when we did the upgrade, Ultraminx refused to work. In fact, I was rather disturbed that the repository hasn’t been touched in a while. Therefore, I decided to look at Ultrasphinx again. I found that on GitHub that it was forked numerous times by different developers, and I eventually used Jamie’s code. However, I had to hack it as well.

However, even that is rather old. I’m tempted to do what 10 other people have done before me and fork the ultrasphinx code and to see what happens when I pull from these other repos. Of course, I haven’t taken the plunge and used DataMapper or Sequel yet, so I’m going to wait until MerbCamp to see what happens when I hack on this.

Yes, I said I’m going to MerbCamp.

Posted in Uncategorized | No Comments »

August 29th, 2008

Recently, we reached a milestone with our first Merb project. The project was interesting, since it was the largest Merb app that we had to deal with. The good thing about Merb was the fact that it’s lightweight, and gives you a sense of what you’re actually using as opposed to Rails, which is full of methods that I just don’t use. I also like the way file uploads are just handled in Merb, as opposed to the Rails way, which is far more cryptic.

The fact that whenever you post data, it just goes to the following parameter:

params[:file][:tempfile]

Is extremely handy, since this doesn’t have to be handled like the good old ruby way of handling blob data through your app. You still have to move the file, but you aren’t just being handed some weird data in memory and expected to use something to process it. This is why people use plugins like attachment_fu and things like that. The less your web app is handling that sort of data, the more requests that it can handle, and the more things it can get done.

Of course, that’s the whole point of Merb. Admittedly, we don’t take advantage of this as much as we could have in this project. We have Rails projects that handle hundreds of uploads that would probably be more suited for Merb, but of course hindsight is almost always 20/20 and it’s time to move on to talk about jQuery and Nitobi Toolkit.

Since this was an experimental project of sorts, I decided to try out jQuery to see what all the hype was about. jQuery is a nice little framework that makes doing simple things like Ajax requests easy. Attaching events to classes is handy, and that is the fundamental strength of the framework. Also, using jQuery UI was interesting as well. While making things appear and disappear was easy, some of the components tended to be ugly, and lacked functionality, so we decided to use Nitobi Calendar instead of the jQuery calendar early on.

This brought us into bringing in Nitobi Toolkit and Nitobi Calendar and making it play nice with jQuery. We learned the number one thing about jQuery, which is that you should always use noConflict if you believe that you are going to use more than one framework. In our case, we use both the Calendar and the Nitobi History object, simply because of the lack of examples of the jQuery.history object, and the fact that it relied on Anchor tags in the examples. It wasn’t clear if we could custom define the URI behind the scenes, and use that instead of something defined in the markup, which is exactly what we wanted to do.

Therefore, we used Toolkit’s history method, and it worked rather well. I may check out jQuery’s History add-on library, but the Nitobi history comes standard in Complete UI and is rather easy to use. There will be some tweaks to it in the next version of CUI.

In short, it was an interesting expeirence working on the project, and being able to work with Merb. I’m definitely going to move towards Merb, and it should be good when Merb 1.0 comes out at MerbCamp in San Diego. Unfortunately, it’s at the same time as Canadian Thanksgiving, but there may be at least one person from Nitobi present.

There may not necessarily be a MerbDog soon, but It’s definitely my first choice for any new personal web projects that I do.

Posted in Uncategorized | 3 Comments »

June 2nd, 2008

In my spare time, I like to mess around with things such as Arduino hacking, SpokePOVs and other small electronics projects. I may have commented in the past about a couple of them such as the failed attempt to build a touch screen interface. I also have accumulated stuff from the FreeTheNet Community Wifi project as well.

Well, Evil Mad Scientist Labs has created this project called “The Great Internet Migratory Box of Electronics Junk”. This is where a box of electronic parts gets mailed from place to place and it gets picked through, parts get added and removed and the box moves on to its next destination. I put my name on the wiki to get a box, but if you want it, you really should check their wiki out and put your name down. I seem to be the first Canadian link, so I’m pretty pumped about that, even if I don’t get the box. :)

It’s definitely very cool, and I recommend you check it out!

Posted in Uncategorized | No Comments »