Blogs

The Nitobi office right now is starting to empty out, and it’s not just because of the weather. It’s conference season and the other guys are at JSConf. However, I’m going to be going to some conferences this year. I’m going to be going down to the SF area Google IO this year. Feel free to ask me about PhoneGap and PhoneGap Android when I’m down there.

Also, I’m speaking at OSCON this year and if you follow this blog’s details about working on WebKit, it’s going to be more of the same, except with better slides, more detail and MUCH, MUCH more hunting for weird comments in WebKit.

I’ll probably be attending other conferences related to PhoneGap and Android Development at some point this year, but those are the ones that are confirmed. If I find out about any more, I’ll definitely be postng them here.

I finally got some free time to do something that has been nagging me for quite some time, which is attempt to actually debug WebKit. Note that I said WebKit, and not PhoneGap. The main motivation for this was the fact that everyone at Nitobi depends on WebKit to work, and sadly on certain Android devices, certain things like the Android WebKit bridge don’t quite work as expected. Now, I understand that this bug doesn’t affect the top-of-the line phones like the Nexus One, the Nexus S, the Motorola Droid/Milestone, but could show up on budget devices, such as the Motorola Spice, the HTC MyTouch4G (that cheap thing with the Keyboard from T-Mobile that looks like an HTC Magic), or pretty much every Android device that has a screen smaller than 480×800, since they seem to have the weaker processors and a smaller memory footprint.

So, while we have worked around it, it’d be good to actually fix this problem for everyone, not just for us, so I decided to take a look into this issue. Now, since this is a WebKit failure, I decided that I’m going to have to hook up an Emulator to this process. This is easier said than done, since WebKit is written in C, the build is done on an Ubuntu machine, and this is leading into gdb territory. Furthermore, WebKit is a shared library, NOT an exectuable, and PhoneGap Android is written in Java! How do we go about doing this.

I decided to ask Google, and I found this entry on the Android Platform group that details the process. Unfortunately, email isn’t the clearest way of communicating things, so I’m going to start at the beginning. The first thing you need to do is fetch the Android source, to do this, follow the steps here.

Once you have the source code, the next step in the process is setting the debug flags. To do this, first it’s a good idea to run lunch and pick which platform you want to debug this on. Once you do this, you need to copy from build buildspec.mk.default to the root directory of your android repo. Then rename it to buildspec.mk, and add two lines to it:

DEBUG_MODULE_libwebcore:=true
DEBUG_MODULE_libxml2:=true
TARGET_CUSTOM_DEBUG_CFLAGS:=-O0 -mlong-calls
ADDITIONAL_BUILD_PROPERTIES += debug.db.uid=100000

Then, since we’re doing a debug build, you need to go to external/webkit and edit Android.mk. Uncomment this line:

LOCAL_PRELINK_MODULE := false

This will allow you to have HUGE webkit files that won’t break your build. Webkit will be extra large due to the fact that you now have debugging symbols included in the libraries. Once this is done, then make the app, and go home. A debug build of WebKit in the AOSP seems to take at least three times as long as a regular Android build. Once it has built, assuming that you picked the Engineering Image for the Android Emulator, you need to specify the the ANDROID_PRODUCT_OUT, then you will be able to start the emulator and get the device going. Once you start the emulator, you need to setup port forwarding:

adb forward tcp:5039 tcp:5039

The next step is pretty important, since this is frustrating. To debug WebKit, start a WebKit process, then attach the Java Debugger to that process. I chose to do this in Eclipse, where it’s just like debugging a regular Android app. Once the Java Debugger is attached, then attach gdb to it. It’s important to run Android’s GDB, and you have a copy of this located in the AOSP tree. Due to the fact that I hate using Eclipse for anything other than Java development, I decided to go and use a tool that I haven’t used since my university days, namely DDD, which is a brutally old front-end to GDB, but it works well enough, and gives you access to the GDB command line.

Anyway, to connect GDB up you need to do the following on the device:

gdbserver 10.0.2.2:5039 --attach pid

And in GDB/DDD/whatever, you need to run these commands:

set solib-absolute-prefix /home/(yourdir)/aosp-official/out/target/product/generic/symbols
set solib-search-path /home/(yourdir)/aosp-official/out/target/product/generic/symbols/system/lib
file /home/(yourdir)/aosp-official/out/target/product/generic/symbols/system/app_process

Once you have both debuggers, you need to run cont! Then the app will work until you crash it, where it will show you where it crashed. This is as far as I got with this process. I think it’s pretty ridiculous that you need to run two different debuggers with this, but it makes sense, since there’s WebKit and there’s the Dalvik/Android stuff above WebKit. It’s funny listening to people talk about Native vs Web on Android, since in Android’s case, the Web Apps are FAR more low-level than things running in Dalvik Virtual Machine. This also may explain the extremely broken nature of WebKit on Android, and why there’s so much divergence of WebKit across these platforms.

I’m going to keep working on this. If you know anyone who’s actually patched WebKit on Android, or works on this at HTC, Motorola, Google, Samsung or another company, please let me know what your workflow looks like, since it’d be good to get ramped up on this to fix some of the extreme brokenness and quirks that are causing Android’s browser to quickly become one of the worst browsers for the web. Given how pro-Android I’ve been in the past, that should be a sign that things are really broken.

Recently, I watched some Mobile Web training, and I noticed that people overlook many things about Android. Namely these facts:

• WebKit is different on every phone
• The Browser and the WebKit engine are different things.
• WebKit can have a different engine

So, as usual I will talk about how the source to Android is OPEN, and how with the source code, I can look at the build scripts and find out what Android ACTUALLY supports instead of doing feature detection or the other things people in turtlenecks use to decide what Android can and can’t do very well. Below is the applicable Makefile for WebKit:


# Two ways to control which JS engine is used:
# 1. use JS_ENGINE environment variable, value can be either 'jsc' or 'v8'
# This is the preferred way.
# 2. if JS_ENGINE is not set, or is not 'jsc' or 'v8', this makefile picks
# up a default engine to build.
# To help setup buildbot, a new environment variable, USE_ALT_JS_ENGINE,
# can be set to true, so that two builds can be different but without
# specifying which JS engine to use.

ifneq ($(JAVASCRIPT_ENGINE),jsc)
ifneq ($(JAVASCRIPT_ENGINE),v8)
# No JS engine is specified, pickup the one we want as default.
ifeq ($(USE_ALT_JS_ENGINE),true)
JAVASCRIPT_ENGINE = $(ALT_ENGINE)
else
JAVASCRIPT_ENGINE = $(DEFAULT_ENGINE)
endif
endif
endif


So, if you’re running a low-end Android device, like a Motorola Quench, your Javascript Engine will be different from the Nexus S, HTC EVO with CM7, the Motorola Droid, the Nexus One, etc. Since nobody has seen the official Android 2.3 on anything other than the Nexus S, I can only really do experiments with Cyanogen on an HTC Dream, which isn’t really accurate.

Recently, there was an issue with the emulator image on Android 2.3 that directly affected PhoneGap. Because I have no idea what Javascript Engine is on the emulators, I suspect that this image may creep up again on low-end phones and cause even more fragmentation than before. This is a REALLY bad thing, since it makes web development on Android even more frustrating, and will chase developers away from that platform. Like everything else in Android, things are always changing, so if any phone manufacturers are reading this and can test PhoneGap on their low-end line and get back to me, that would be greatly appreciated.

A while ago, I broke my Nexus One, and because I switched from Rogers to Telus back in February, I was unable to use my other Nexus One and I had to go back to using the Motorola Milestone as my phone. Now, I’m trying to look at the bright side of this is that I was forced to live like most users of Android, which is that I’m stuck using Android 2.1 (and while I’ll happily root a Nexus One and void it’s warranty, I’m a bit more sketched out by the Milestone’s Signed Bootloader, and would like a backup phone before going through the rooting process.)

This isn’t a huge deal with PhoneGap, except that I found a bug that was only in Android Webkit. The issue is that Motorola’s Webkit behaves differently than the stock Emulator Webkit, and that each phone Motorola implements has a unique firmware. That’s where the Motorola Developer Add-Ons came to the rescue. I was able to get Emulator Images that were roughly similar to my phone’s build, and was able to confirm the bug. Motorola really stands out with this tool, and it seems that after doing some initial checking, it seems that Samsung only has an emulator for the Galaxy Tab, and not for their other Android devices, HTC doesn’t even have an emulator and just offers source so you can patch the Android Open Source Project and generate your own images. Sony Ericsson has an Emulator for their Android 1.6 devices so if you need to test on the Xperia line, you can test on that as well.

The downside is that the Emulator is painfully slow in comparison to the device. Furthermore, most Emulator Images are still very unfinished, and others have annoying side features. They may be a custom boot animation, or random resolutions that aren’t supported. The Android Emulator is definitely the weakest link, and it would be great if we could rely on Android Webkit to be the same across all devices, and behaviours (like proxies) to work across all devices as intended. Unfortunately, life is not that simple.

Last year at this time, I did a post about Ada Lovelace Day, and once again I’m back at it, doing another post, this one is going to be pretty short. This year, I wasn’t sure why I was going to write a blog post about, so I figure that this time, I’ll just write about my 2nd Year Computer Architecture Professor, Desa.

Desanka Polajnar was probably the first CS professor that actually made people worry about failing a Computer Science course when I was up at UNBC ten years ago. People for some reason couldn’t quite understand how boolean logic worked, and why K-Maps were a good idea. In fact, many people didn’t even realize why it was relevant to learn Assembly Programming at all, and many people consider knowing the internals of a processor a waste of time. I remember hating the first CPSC 230 class, because it was hard, and because it seemed to have absolutely no bearing, or connection to what I’d be doing now. I also remember thinking that MIPS and ARM assembly was useless because you know, everything runs on an x86.

Also, it was in her class that I got introduced to Hardware, which has turned into a pretty awesome hobby. It’s strange, since to this day, I haven’t met too many people that even know what a PDP-8 even is, let alone actually implement it with microcontrollers and TTL logic. Thanks to Desa, I can say that I actually have done that, and I think that’s pretty awesome. I definitely appreciate the time I spent in her classes, even if I didn’t seem like it back in the 90s, when I was actually in them.

Thanks

Recently, I started looking at Canvas on Android, and I came across the C5 Canvas Benchmarking test. I tested it on both the Motorola Milestone that I recently bought, and the Sony Ericsson Xperia X10, and it was pretty surprising. Now, it should be noted that the Motorola Milestone has a TI OMAP3430 processor, and is capable of speeds up to 600 MHz. However, the Xperia X10 has a Qualcomm Snapdragon processor, and has a clockspeed of 1.02 GHz. Now, people have been speculating whether this clock speed will make a difference, and I think that when you look at the results, it clearly does.

Now, I don’t think that performance is quite there yet, but you may see more advanced uses of Canvas on mobile web applications in the future.

I recently bought a new phone, the Motorola Milestone from Telus. The Milestone currently features Android 2.0.1, and is the first Canadian phone that actually features Android 2. So, while confirming fixes on the device, I noticed in LogCat that my WebCore wasn’t logging error messages. For those of you who don’t know, on Android 1.x, you could debug your javascript by using adb logcat and grepping for the WebCore errors. This frustrated me, since this wasn’t listed in the API changes. I then decided to sift through the WebView source code, and I found addMessageToConsole.

The method addMessageToConsole is a method that currently exists on the WebChromeClient that can be overriden with your own custom error message. Now, according to the Google Documentation, this is not a public method to be used until Android 2.1. However, I just tested it on my Motorola Milestone, and the method actually works. I’ve included this fix to PhoneGap and you should now be able to go back to debugging your Javascript directly on the device, as opposed to the other methods of using a different browser.

BTW: The code to add it is as follows:


public final class EclairClient extends WebChromeClient
{
private String TAG = "WebErrorLog";
// This is a test of console.log, because we don't have this in Android 2.01
public void addMessageToConsole(String message, int lineNumber, String sourceID)
{
Log.d(TAG, sourceID + ": Line " + Integer.toString(lineNumber) + " : " + message);
}
}


For backwards compatibility with Android 1.6, it’s a good idea to have this class extend a pre-existing WebChromeClient, and to dynamically chose the client you instantiate so you can actually get your application working.

After spending a day with Java reflection, I believe that we are now at the point where PhoneGap Android can now be supported on 1.5, 1.6 and 2.0 versions of Android. This wasn’t incredibly hard, except for the fact that I was unfamiliar with reflection, and the example on the Android site is less than helpful, since it deals with the Debug static methods, and not a real world object that can have things like instances. Also, reading Sun’s documentation made me realize how useless examples that use the word Class is as a classname.

The result of this is that we may have a JAR that can be used on Android 1.5, 1.6 and 2.0, which solves the divergence issue. However, this doesn’t mean that we have SQLite database storage that doesn’t exist on Android 1.5 or 1.6. It means that if you are using Android 2.0, you have access to the SQLite Database if you are running the Edge version.

After pairing down PhoneGap to a jar, it appears to be weighing in at under 200k. I’m thinking about distributing it as an installable library on Android so that if you have it installed, you can download PhoneGap compatible APKs. This would be similar to the Text-To-Speech API functionality. This may encourage Google to close the browser gap, or it may just get ignored if people just use their own stand-alone PhoneGap + app apps.

More info on this will come shortly.

So, after hearing the news that Nokia will be dropping Symbian from their High-End N-Series phones by 2012, and adopting Maemo, I decided that I would do a bit more research into the platform. I heard good things about it being open and it having an actual mature Linux stack (as opposed to what is running on Android and Palm), and I wanted to see how much effort it would take to get a PhoneGap prototype to run on it. At around 4 PM, I have this to show for it:

I used QtWebView to implement this, and it took more time to get the SDK started and working than it took to actually write the app and run it. I think getting the SDK for Maemo was the biggest hurdle for developing for the device so far. Of course, this is ONLY for Nokia phones, and not for many other phones, but the fact that the N900 was a device that I was considering until I found out it couldn’t support Canadian HSPA frequencies, I figure it was worth messing with. Also, QtWebKit is a part of Qt, and in theory can run on Windows Mobile with mostly the same codebase. With the uncertainty of whether Maemo would get Symbian WRT running on the device in some way, this may be an option.

At the end of the day, I’ll probably stick with my Android Phone, but this was rather interesting.

Google just released the latest version of the Android SDK, Android 2.0, which does most of what I need it to do. I noticed that so far the changes that Google has made to WebKit have not found their way back to the Android Open Source Project, which is disappointing, but overall it looks interesting.

Google has recently ripped out Gears and replaced it with proper HTML 5 support. This means that the Android 2.0 browser now supports HTML5 Video, Canvas, Geolocation and Database Storage. This is pretty huge for Android and for PhoneGap on this platform, since this allows for the write once run anywhere behaviour that we expect, without any weird gotchas.

Now, I’m going to talk about what is in WebView. For those of you who don’t know, WebView is the WebKit Java Component that PhoneGap, and all other WebKit-based Browsers use on the Android device to display web content. So far, I’ve tested Canvas, Geolocation, and HTML5 Database Support, and I’ve gotten some mixed results.

Canvas works right out of the box, while Geolocation seems to not work at all in the Emulator. Of course, I have to send it mock locations to get it to work, so I have no idea what this would be like on an actual phone. I can say the same thing with the video tag. There are issues with it not actually playing the video, BUT I think it’s the fact that the video is a higher resolution than what the Emulator can handle. We’ll know more once someone tries this on a Motorola Droid or other next-gen Android device.

The BIGGEST issue is with the Android Database Support. It’s there, but only on the Browser Application, where the settings can be properly setup. However, it’s not present on the WebView itself, where you have to set the Database Path and the the location for where the Database Files will be stored. Of course, we end up with this sort of awesomeness.

This is caused by the fact that for some reason we don’t have a way to set the size of the database locally, or the fact that we don’t have a way to handle when we do go over quota, or both, or neither. We don’t know because there’s no examples on the Internet at this point on how to fix this issue for Android. If you take the same code in WebKit Nightly, Safari or Midori or any other WebKit that supports HTML5 storage, it should work.

I’m hoping once the code drops for Eclair, I can look at the browser activity and find the place where it’s enabled. This should be rather trivial, but given the fact that this is bleeding edge stuff, I’m not suprised that WebView, something only used by developers, would be overlooked/not documented.

I hope to have more info on Android 2.0 soon, and we will be looking at dramatically changing PhoneGap for Android based on these change. In the meantime, feel free to download the Android 2.0 SDK and try it out for yourself.