Blogs

Hey

Recently, we’ve been making changes similar to the iPhone code so that we can make it even easier for people to write Android Applications. We’ve encapsulated the PhoneGap source code into a JAR file, so that your Android Application can literally be as short as this:


import android.os.Bundle;
import com.phonegap.*;
public class TestApp extends DroidGap {
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
super.loadUrl("file://android_asset/www/index.html");
}
}


Of course, you’d have to have the AndroidManifest setup correctly to accomodate the activities, but this is minor in comparison to how messy everything’s been before this change. We’ll have to do more restructuring of the repository to demonstrate how to use the new library in Android Applications, and this should allow for the Java code to be abstracted away and made even easier. Of course, we’d have to do more work to figure out how this will work on Android 2.0, but it’s some exciting stuff.

*EDIT*: Apparently Rogers didn’t push an upgrade to Android 1.6, and it’s very possible that they may never push up this update. Since I still use the phone I received from Google IO, I was not aware of this.

Recently, there’s been talk about Android splintering. This isn’t a rumour, it’s a fact. Here’s a short list of the Android Devices that are either in Canada right now or are slated to arrive and what they support:

• HTC Magic (Rogers): Android 1.5 (Upgradeable to 1.6)
• HTC Dream (Rogers): Android 1.5 (Upgradeable to 1.6)
• HTC Hero (Telus): Android 1.5
• LG Eve (Rogers): Android 1.5
• Motorola Milestone (Telus): Android 2.0 (Coming: Early 2010)

OK, this sucks! Android 1.6 introduced one MUST HAVE feature for Android, and now we’re excluding ALL the Canadian carriers? Now, it’s clear that Telus didn’t really have Android to begin with because of the fact that they were stuck with CDMA so long. However, that being said, this is not theoretical splintering, this is something that’s happened. Now, while I don’t care about the LG Eve, since it looks like a piece of crap (I can actually walk into stores and look at these devices!), I do care about the HTC Hero not being able to run PhoneGap.

Now, I could downgrade, but I lose support for the Sony Ericsson Xperia X10 and the Motorola Droid, but I can tell you right now that we’re not going to downgrade. This trend of shipping devices with a sub-standard version of Android needs to stop. I don’t expect people to ship with 2.0, because while it has nice, new features, it’s clearly got some issues that need to be resolved. However, shipping without the latest minor version (1.6), or at least pushing an OTA update is stupid. Hardware manufactuers and carriers need to keep in line with latest, which should be pushed to the Android Open Source Project repository. If that doesn’t happen early, and often, it’s going to keep frustrating developers.

So, after hearing the news that Nokia will be dropping Symbian from their High-End N-Series phones by 2012, and adopting Maemo, I decided that I would do a bit more research into the platform. I heard good things about it being open and it having an actual mature Linux stack (as opposed to what is running on Android and Palm), and I wanted to see how much effort it would take to get a PhoneGap prototype to run on it. At around 4 PM, I have this to show for it:

I used QtWebView to implement this, and it took more time to get the SDK started and working than it took to actually write the app and run it. I think getting the SDK for Maemo was the biggest hurdle for developing for the device so far. Of course, this is ONLY for Nokia phones, and not for many other phones, but the fact that the N900 was a device that I was considering until I found out it couldn’t support Canadian HSPA frequencies, I figure it was worth messing with. Also, QtWebKit is a part of Qt, and in theory can run on Windows Mobile with mostly the same codebase. With the uncertainty of whether Maemo would get Symbian WRT running on the device in some way, this may be an option.

At the end of the day, I’ll probably stick with my Android Phone, but this was rather interesting.

Recently, I’ve been looking at the old Audio Handler code that was contributed a while back and why it can only play on the SD Card. I assumed that there was some weird permission issue that didn’t allow for data to be written in other parts of the device, but I didn’t look into it too closely until recently.

In PhoneGap, the files that are being loaded are stored in the assets directory. The decision to do this was because they could be accessed through file://android_assets, and this seemed to be rather convenient for the time, since I wanted to prove that you could in theory take the HTML and Javascript from an iPhone PhoneGap app, and put it into an Android App and have it still work. However, the issue is that this is not really a file location.

All files in the Android solution are zipped up and compiled into an apk, which is signed and installed on a phone. Dalvik comes across this package, and runs this package much like a traditional Java VM runs a JAR. The big difference is that if you’re trying to do things like access a file natively, store data into a database, or run an executable, this doesn’t work. Now, in the past, we just used the SD card for when we needed file storage, BUT last week when figuring out the WebKit SQLite Support for Android 2.0, I discovered where the data actually was being stored on the device, in the /data/data/package_name/ directories.

I then decided to look at how other people are using this storage, and I checked out the Orbot project. A while back, I had a passing interest in porting Cryptographic tools such as OTR and Tor onto Android so that I could have this on my phone. However, other people who are more dedicated to making awesome things, actually made this happen with Orbot, and shared the source. I saw that they were grabbing an executable written in C, and storing it in the data directory. This allows for ARM code to be run on the device, and allows people to get around the Android SDK, and the problems that it has.

So, I’m faced with a dilemma. Do I keep with storing the data in the assets directory, or do I copy the www directory to the /data/data/app_name/www directory and allow the files to sit in the Application FS storage? Will there be an issue on initial install? Will Android PhoneGap require a loading screen where we do this sort of heracy? Putting layout and executable code in the /data/data directory herasy? What do you think?

So, after many days of using DDMS to poke around the Emulator subsystem I saw the way the new com.android.browser activity interacted with the data. I’ve took a TestCase code (which is like PhoneGap for Android, but stripped donw with only the WebView component), and hacked around with the old stickies example so that I can get it to work. So, here’s the steps to do it:

Step 1: Create a WebView, similar to the one in the PhoneGap source.
Step 2. Get the webSettings, and set the Database Path. The path should be the path to your databases, which will be /path/path/your_package_name/.


appView.setWebChromeClient(new WebClient(this));
WebSettings settings = appView.getSettings();
settings.setJavaScriptEnabled(true);
settings.setJavaScriptCanOpenWindowsAutomatically(true);
settings.setDatabaseEnabled(true);
settings.setDatabasePath("/data/data/org.infil00p.testcase/app_database");


Step 3: Create a WebChromeClient, and override the onExceededDatabaseQuota method. This is called when you first open the application because there’s initially no database setup. This will allow you to set the quota in Java. Since this was test code, I just threw an arbitrary value in here.


final class WebClient extends WebChromeClient {
Context mCtx;
WebClient(Context ctx)
{
mCtx = ctx;
}
public void onExceededDatabaseQuota(String url, String databaseIdentifier, long currentQuota, long estimatedSize,
long totalUsedQuota, WebStorage.QuotaUpdater quotaUpdater)
{
Log.d(LOG_TAG, "We exceeded the quota");
quotaUpdater.updateQuota(100000);
}
}


OK, we now have a database that we can store things in. When dealing with Android Databases, remember to do all your database debugging on the device or a rooted phone, since you can’t get access to the SQLite Databases from adb with the standard firmware on consumer devices, and this includes the latest images for the Android ADP1 and the Google Ion phones. I’ve just gotten this to work now, and I haven’t tested this method with the Geolocation Database, but I would assume that it would be similar, except that there would just be the Cached Location. What I found odd about this setup was the fact that I couldn’t write the database directly to the emulated SD Card on the device, which would make sense. I’m definitely going to play with this more, and refine it so that it’s less hacky and more polished. However, the stickies example works now, and we will try porting existing iPhone Applications that use this SQLite storage soon.

Google just released the latest version of the Android SDK, Android 2.0, which does most of what I need it to do. I noticed that so far the changes that Google has made to WebKit have not found their way back to the Android Open Source Project, which is disappointing, but overall it looks interesting.

Google has recently ripped out Gears and replaced it with proper HTML 5 support. This means that the Android 2.0 browser now supports HTML5 Video, Canvas, Geolocation and Database Storage. This is pretty huge for Android and for PhoneGap on this platform, since this allows for the write once run anywhere behaviour that we expect, without any weird gotchas.

Now, I’m going to talk about what is in WebView. For those of you who don’t know, WebView is the WebKit Java Component that PhoneGap, and all other WebKit-based Browsers use on the Android device to display web content. So far, I’ve tested Canvas, Geolocation, and HTML5 Database Support, and I’ve gotten some mixed results.

Canvas works right out of the box, while Geolocation seems to not work at all in the Emulator. Of course, I have to send it mock locations to get it to work, so I have no idea what this would be like on an actual phone. I can say the same thing with the video tag. There are issues with it not actually playing the video, BUT I think it’s the fact that the video is a higher resolution than what the Emulator can handle. We’ll know more once someone tries this on a Motorola Droid or other next-gen Android device.

The BIGGEST issue is with the Android Database Support. It’s there, but only on the Browser Application, where the settings can be properly setup. However, it’s not present on the WebView itself, where you have to set the Database Path and the the location for where the Database Files will be stored. Of course, we end up with this sort of awesomeness.

This is caused by the fact that for some reason we don’t have a way to set the size of the database locally, or the fact that we don’t have a way to handle when we do go over quota, or both, or neither. We don’t know because there’s no examples on the Internet at this point on how to fix this issue for Android. If you take the same code in WebKit Nightly, Safari or Midori or any other WebKit that supports HTML5 storage, it should work.

I’m hoping once the code drops for Eclair, I can look at the browser activity and find the place where it’s enabled. This should be rather trivial, but given the fact that this is bleeding edge stuff, I’m not suprised that WebView, something only used by developers, would be overlooked/not documented.

I hope to have more info on Android 2.0 soon, and we will be looking at dramatically changing PhoneGap for Android based on these change. In the meantime, feel free to download the Android 2.0 SDK and try it out for yourself.

So, in my last post, I wrote about trying to figure out the WebKit source. Now, unlike other open source projects that I played with or contributed to, WebKit isn’t exactly the most obvious one to hack on. The main things that make it difficult to grep and figure out was the the fact that there’s no obvious point of entry for new hackers.

Now, when it comes to other pieces of software with similar complexity, they approach this problem in numerous ways. For example, the Linux Kernel, (which I occasionally try to test and write modules for, none of which are any good to make it past my desktop) has a really good site called kernelnewbies.org, that I would recommend anyone who is new to Operating Systems read. It goes through how the Linux Kernel actually executes, the coding conventions, and how to code, test and patch a kernel. Sites like this are very important to overcome the whole “Einstein” complex that comes with Open Source software.

Now, with WebKit, there’s not as much for new developers to grab onto, so I went back to something that I was more familiar with, which was Mozilla. Now, I admit that in the past I’ve attended some MozCamps, and I’m friends with a few people there, but I’ve never touched the actual Firefox Source.

Until now.

Admittedly, this was a VERY easy bug to fix, of minor importance, and it was flagged as “Good First Bug”. I was looking for something that I could write quickly while working on other projects during the day. I’ve never used Mercurial until I fixed this bug, and I’m nowhere near being a XUL ninja, but I managed to get a bug fixed, which is awesome.

I feel that these social hooks are what’s needed in making a successful open source project, and that the people who put the time into flagging the bugs as [good first bug] in Bugzilla, or maintain things like Kernel Newbies are awesome. I hope that more people make sites that give people who have a passing interest in Open Source development the ability to contribute back, and it’s something that we try to keep in mind here when we make our stuff. For example, If you think that there’s something in PhoneGap that needs to be clarified from a contributor standpoint, please let us know and we’ll try to figure out how we can make it easier, (within reason, we’re sticking with git, because git and hg are awesome).

Recently, someone was talking to me about Apple, and was telling me about how it’s next to impossible to make sure that WebKit was secure and that it was a lost cause. Now, given that WebKit is basically the web experience of most mobile devices, this claim seemed like it required more looking into, and that it’d be a good idea to once again attempt to take a look into the WebKit Internals.

The thing is that there’s literally no documentation for the WebKit project beyond some mentions of events in the iPhone SDK docs. WebKit in general is a very ambiguous piece of software, and each version of webkit is different depending on what platform it’s compiled for and which features were built. The downside of this is that this is all based on conversations over beer, and a broken Bugzilla.

Given that I’m generally new to browser internals, I would have to say that looking at the WebKit source without any idea of how WebKit works is next to impossible. I’ve played with builds of Mozilla in the past, and I think that I’ll probably keep looking at Mozilla code simply because:

• The edge version compiles under Ubuntu
• Mozilla’s Bugzilla isn’t some secret thing where you have to be working for Apple, Google or some other company to read the bugs
• Mozilla seems to want people to look at the code

Open Source is great, but there seriously needs to be a “Hitchhikers Guide” to WebKit, and unfortunately WebKit is just not there. I admit that WebKit is a better browser when it comes to embedding it into various devices, but the clear lack of transparency when it comes to WebKit seems to make me think that the person who told me that it was a lost cause might actually be right. Developers need documentation, and the current lack of documentation is rather frustrating to say the least, that and the fact that you seem to have to have a Mac to develop for it, is another reason why I find myself going back to looking at Gecko.

Because documentation and a friendly community actually does matter.

Recently, there’s been concern over the state of WebKit on Android. If anyone has ever used the WebView, people realize that it doesn’t have gears. However, once you look at the source code of the Browser in Google Android (You can actually do this), you realize very quickly that it’s not an issue with the browser, it’s an issue with the Shared Library not being loaded.

Now, currently I don’t know if the current SDK would support the NDK-style JNI required for PhoneGap to do this, but it should be coming in Donut. However, I want to know when HTML 5 is coming to Android. If you take this page and try to run it on the Android browser, it will fail, but it should work on the iPhone, Safari and it works on my version of Midori that I’m running on Ubuntu.

I don’t have a copy of the source code to the Palm Pre, although I hope that one day in the future I can see their version of WebKit and see how they managed to get HTML 5 on the phone. I hope that Google follows suit and gets proper HTML 5 support on their browser instead of using gears in a shared library linked in the Browser APK. This is important since Gears is not HTML 5. I honestly enjoy going through the Android Source Code for the most part and seeing how the Java fits with the rest of the embedded system. It’d be great if I could do this with the Pre as well.

Open Web Vancouver was pretty awesome this year. The Pirate Party keynote by Rickard Falvange was awesome, and really was a solid follow up to Zak Greant’s talk on “The Age of Literate Machines”. Given the current climate with people fighting against more oppressive copyright, I really liked how relevant it was, and how it tied to the Civil Rights movement.

The second keyonote talk was also really well put together. I saw Angela Byron’s “Women in Open Source”, and it was interesting to see how many other people were shocked as to how few women were in Open Source. The talk was interesting, and depressing at times, but it was a good talk, and it showed a need for everyone in the Open Source community to wake up to this reality and do something about it.

The thing that irritated me the most about the conference was actually not the fault of the organizers of the conference, and is entirely the fault of Bell Canada, and the convention center. Jake Appelbaum of the Tor Project (and of Noisebridge) was giving a talk on Tor. Tor is an anonymizing router that allows for people in places such as Iran and China to access roughly the same internet as someone living in a country such as Canada or the United States, getting around government restrictions such as the Great Firewall of China. The problem is that unfortunately, the Vancouver Convention Center is more oppressive than the Chinese Government, because they blocked the following sites:

• Tor
• Kittens
• RapidAndroid.org (UNICEF)
• etc

Now, I can understand blocking something like 4chan (the Anime convention had a talk on it, BTW), but blocking Tor and UNICEF is crossing a line. The site in question had information on a project that I was looking to promote during my Android talk, which allowed for formatted SMS messages to indicate where things like Mosquito Nets to prevent Malaria should go, and where buckets for distributing clean drinking water should go. This project actually does a LOT of good. Too bad Bell Canada disagrees with me.

The other talks at the conference were pretty awesome. The big complaint that I had this year was
that the PhoneGap presentation by Brock and Rob was at the same time as Franklin Lopez’s presentation about Documenting Dissent. The Open Data talks by David Eaves were pretty interesting as well, since the City wanted to know what data it should open first.

In short, the conference was great, despite the fact that Bell censored the Wireless. I find that this will be the one thing that sticks in my mind this year, since the organizers did their best to position Open Source, Free Speech and Fair Copyright in the forefront.