Tuesday, June 3, 2008

Is Device Fragmentation Still Relevant?

Device fragmentation is probably one of the most hot headed topics in mobile programming, people everywhere keep repeating the mantra about how broken and buggy devices are that we all pretty much accept it as a fact and move along without re-evaluating each and every bug on modern devices. Recently I got into a debate about fragmentation with a colleague who doesn't agree with my views about device fragmentation and this reminded me of a presentation from Orange & Sun at J1 a few years back that was practically heckled because it tried to promote the concept of one jar for multiple devices...

My views on the device fragmentation status should be pretty clear if you are familiar with LWUIT generally here they are in an organized form:
CLDC 1.0 devices were often of low quality, however CLDC 1.1 devices from the past 3 years produced by the top manufacturers are pretty good. Even second tier manufacturers have improved their offerings substantially in recent years!
To target ALL phones especially old phones you would probably need some complex tricks. However with current technologies one JAR for all devices (assuming downloadable resources) is a very viable approach.
Targeting all phones would skyrocket your development costs up to 10 times or even more, its the 80/20 rule: 80 percent of the devices is 20% of the effort. Its more financially viable to add features to your application than it is to try and reach every broken phone out there.

I've talked allot with people over the device fragmentation issues and most of the points made are fixable and even incorrect, here in no particular order are some of my responses to these claims.

Claim: You will target the lowest common denominator!

The general idea is to adapt based on phone features. In LWUIT we detect 3D support on the device, if its available we offer cool features such as 3D transitions and if not we don't offer them. Its as simple as that.
E.g. for networking many devices have issues with more than one connection at a time, rather than limit all devices a simple networking hub that manages connections and handles failures can solve this problem in a portable way.

Claim: You will detect devices at runtime and increase code size! Worse you can't physically detect devices!
The idea is to avoid device detection when possible since this is not practical. The general idea is to detect bugs not devices, either detect the existence of a bug or detect the fact that it isn't there. Better yet just avoid bugs altogether... Our graphics implementation doesn't use translate at all, we do translate on our own which is just as efficient and far more portable. This is an example of avoiding a bug.
DrawRGB throws exceptions in some use cases, this is not a device bug but rather a spec limitation. We catch this exception once, turn on a flag and from this point on the workaround code for this issue is active instead of the standard drawRGB code. This is an example of detecting a bug (or feature in this case) rather than a device...
You can always place variables in the JAD file which is very easy to modify and doesn't require compilation or complex deployment, e.g. 3 softkey support can be activated in LWUIT using a simple jad property configuration.

Claim: Devices are really broken even today, we have to call System.gc() when load gets heavy.
I bought into this claim after seeing the problems it in action, but being an organized person I tried to reproduced the bugs such as memory fragmentation etc...
I couldn't find a single modern broken device! (although there are quite a few old broken devices from pre-cldc 1.1 days).
What I did find is this:
1. Devices are really bad about closing networking connections or streams, if you don't properly close all of them things will break.
System.gc() works around this by closing streams implicitly for you (in the native finalizers), so make sure you always have a finally block and cleanup properly...

2. Devices don't always handle multi-threaded access to the same resource well, some devices have a problem with many threads using the network and some have a problem with many threads accessing RMS. Just assign one thread per task which

3. Lots of restrictions exist on a device basis that are really well hidden in the spec, e.g. the M3G JSR 184 API has settings for a maximum texture size which must be taken into consideration when creating textures.

I can say with a pretty good sense of certainty that most modern VM GC's are pretty good, some of them are better than stuff we have in Java SE by default. Older VM's had bugs both in fragmentation and memory leaks but these seem to be an issue of the past.
Setting variables to null explicitly (unless you are in the middle of a method) is usually not necessary, invoking System.gc() is also no longer necessary even though some developer guidelines from some major companies still say so.

Claim: I need to target an older version of the device with broken firmware, the operator I'm working with is unreasonable.
Thats a management issue not a technical issue but I'll address that since its a very common complaint and I ran into it allot when working with startups. A startup has very little leverage against an operator and they get squeezed... Hard...
This is a business management issue which you need to pass on to your managers, the cost of running on a defective device include bad user experience and come at the expense of additional functionality since the time spent would increase considerably. The deployment costs which are often ignored by developers also rise considerably.

Claim: You can't guarantee all devices.
No one can.
Pre-processor calls can always be added if push comes to shove, in current devices though I think there are far better solutions. Even with pre-processor I would never commit to every device using every firmware for anything overly elaborate with full functionality. I saw quite a few companies making such commitments and they are just not realistically achievable, you cannot guarantee support for all devices no matter what you do!

Claim: OK, assuming LWUIT does solve UI portability... RMS, networking, JSR 75, WMA, MMAPI (your JSR of choice here) are still a portability problem.
To a lesser degree, UI is the biggest offender. RMS is painful but mostly because the spec is just completely cryptic and its limitations aren't very clear. There are several guidelines to using it efficiently and you can follow one of them. I intend to release an RMS persistence solution as part of the LWUIT-Ext project.
All JSR's have issues especially the new ones, as long as you don't try to do something "uncommon" there shouldn't be a serious problem. Most JSR implementations are based on the same RI code and so they even share the same bugs and limitations, every device has issues MMAPI was especially painful in earlier devices but it got better.
75 is relatively ancient by todays standards and its pretty stable, 226 still has some issues but its moving to stability very fast.
For the common use cases there shouldn't be a problem, for the more advanced uses workarounds like the ones mentioned above can solve most cases.

Claim: Java ME is too small and you can't program for ME in the same way as you do for SE.
That was true a few years ago, in todays devices this is no longer the case. The low end phones have 2mb of ram and allow jar sizes limited by storage only... This is pretty steep.
Sure operators make limitations but these limitations have moved up to the 300kb range well above what a reasonably generic application needs.

Claim: There are still pain points in 100% portability.
Icons are probably the worst offenders since they are such an easy fix for the manufacturers...
Certification is also a huge pain although it rarely requires code changes its still a porting hell.
Threading across devices is a pain, but its a pain even in Java SE and you will receive thread related problems even in the best phones (e.g. Sony Ericsson) so the solution is to design your application in a way that isn't affected by threading stalls.

None of these would change much with pre-processing, you would still need to test on all target platforms there is no real way of escaping that.

What is so bad about pre-processors?
No refactoring or proper compiler checks, issues on one device that are very hard to fix across all devices would make portability harder rather than easier in some regards. Programmers tend to overuse the pre-processor and often blame devices for their bugs rather than spend more time investigating the issues.
Size tends to grow because of all the copy & paste common in such architectures and complexity rises considerably with every device.

In some regards device fragmentation is only becoming worse: i-mode (DoJa), iPhone, RIMlets, Android etc... However, in these areas pre-processing is a worse solution than the problem itself. The C++ crowd solved the portability problem years ago by building abstractions despite the fact that they could use the pre-processor built into the language, in Java abstractions are even more powerful due to the dynamic nature of the language. With the problems associated with Java pre-processing I don't think it is the proper solution today.

No comments:

Post a Comment