LWUIT Porting And Device Issues

Devices are a pain, every device has its own oddities and trying to figure out what went wrong through guesswork is just not very productive.
I said before that supporting all devices isn't feasible in my opinion, but still we want to support as much as possible or at least know why we don't work on a given device. Sometimes getting a device to work isn't the easiest thing in the world, the application might crash right before we even started experimenting.
We might not have enough information about the device in general in order to determine if its even applicable.

So here are general preliminary guidelines of "what to do" when a device doesn't even start LWUIT and how to debug that particular scenario and maybe help us get onto yet another device...

First off start with measuring the device capabilities to see if LWUIT can feasibly run on the given device, we didn't have much luck so far with devices that have less than 2mb of memory so while these might work (especially if the screen size/bit depth is smaller than 320x240x24) there is no telling.

Try running a hello world on the device, if the hello world MIDlet works build upwards from that point. Make sure your hello world looks something like this:

public class LWUITMIDlet extends MIDlet {
 protected void startApp() {
     try {
         Display.init(this);
         Display.getInstance().callSerially(new Runnable() {
              public void run() {
                   try {
                       new Form("Hello").show();
                   } catch (Exception ex) {
                       ex.printStackTrace();
                       javax.microedition.lcdui.Form oops = new javax.microedition.lcdui.Form("Internal");
                       oops.append(ex.toString());
                       javax.microedition.lcdui.Display.getDisplay(this).setCurrent(oops);
                   }
              }
         });
     } catch (Exception ex) {
         ex.printStackTrace();
         javax.microedition.lcdui.Form oops = new javax.microedition.lcdui.Form("Exception");
         oops.append(ex.toString());
         javax.microedition.lcdui.Display.getDisplay(this).setCurrent(oops);
     }
 }

 protected void pauseApp() {
 }

 protected void destroyApp(boolean arg0) {
 }
}

Notice that I catch and display exceptions within the MIDP hi level API which should work on most devices and allow you to report the exception that occurred and maybe even track it down.
I also chose to execute everything possible on the LWUIT EDT thread to prevent potential race conditions or anything "funky", this is the simplest possible approach but we can obviously go deeper.
Try obfuscating code if the default doesn't work, some devices actually treat obfuscated code "better" (make sure to use progruard 4, other versions have some issues) mostly because of package scoping issues on some devices.

Ideally this should work or at least give some form of error which you can report, if it doesn't then you probably need to get into LWUIT's code which is a whole other blog post. From this point on just start adding application functionality until you can identify the source of your issue with the specific device.

Once you get running on a device but have issues within a specific device (that don't reproduce in emulator/simulator) logging is your best solution but this is an issue for another post.

Great LWUIT Article by Jim White On DevX

Jim White wrote an excellent LWUIT article for DevX and provided excellent/easy to follow examples.

Memory Leaks In LWUIT And Tracking Memory In Java ME

Tracking memory usage in Java ME is a pain and requires lots of understanding among devices and behaviors, the first instinct of many developers is to start littering the code with System.gc() which I wrote about in the past. I'll try to focus here on two distinct aspects: tracking memory in Java ME and memory pitfalls within LWUIT.

Lets start with tracking and observing memory usage, first don't rely on Runtime.free/totalMemory()... These methods lie. Don't rely on numbers from the WTK memory monitor, they too aren't correct. I found the Nokia S40 emulator to be the most accurate in terms of memory usage (although a bit of a pain to use). I still use the WTK memory monitor but not to track memory. I use it to track live instances of objects...
When I notice that a given application takes too much memory I look over the table of objects in the WTK memory monitor and look at my applications class list, within this list I look at the number of instances for every class. If a class has too many instances this is the point to investigate using a debugger.
E.g. recently memory was becoming an issue in an application after entering a given form, using the WTK memory monitor I noticed that with every visit to the form a new instance of the form was created (which is the right behavior) but the old instance wasn't cleared from memory.

What was keeping this form in memory?

LWUIT components are in a tree form similar to AWT/Swing and many other component toolkits. In LWUIT every component has a pointer to its parent accessible through the getParent() method. So essentially if a single component within the form is kept in memory the entire form will remain.

So I started with the obvious, I reviewed the form creation code looking all over to someone passing "this" pointers around. Then I looked for components being stored as class members (sure sign of a memory leak), no luck there either. This was getting harder than a typical session...
The solution to tricky problems like this is elimination, remove features until the problem is resolved. This can be misleading in memory leaks since leaked memory can be reduced if you remove images but the problem might not be solved... So the solution was to check against the WTK memory monitor which includes the instance count rather than memory, this displays the actual leak regardless of its size.

To make a long story short: it was the model coupled with a problematic LWUIT design decision.

The problematic form kept its ListModel instance throughout the application, which shouldn't be a problem. However, the List registers itself as a listener to the model so it can update based on model changes. Since the model was kept so was every list displaying the model and every parent form of every list!!!

Ugh.

We worked around this by adding a "flushListeners" method to that specific model implementation which removed all listeners.

Newer versions of LWUIT will be smarter about binding to list models, when initComponent() is invoked the List in the new LWUIT drop would bind the listener to the model. When deinitialize() is invoked the list removes itself as a listener. This will implicitly allow the list to be garbage collected while the user can keep holding the model which is a sensible use case.

We probably don't use initComponent() and deinitialize() enough across LWUIT, we initially wanted to avoid such complexities but due to all the issues of constructor order and initialization complexity we had to add them. They are invoked before showing the form (or adding to a visible form) and after removing the form implicitly. These methods are very powerful tools in the arsenal of MIDP developers.

So here are some rules of thumb for debugging a memory leak in ME:

1. Identify the problem in object terms not in heap terms - try to find out the class/object that is leaking, not the amount of memory that is leaking.

2. Narrow down the problem - remove features until you find the exact features responsible.

3. Some memory leaks are only felt on an emulator (e.g. Nokia S40 emulator) but can still be tracked using the WTK memory monitor which is a simulator. This doesn't imply a bug in one or the other they are just very different in their implementation.

4. Increase the amount of memory used by some elements to accentuate the problem - add dummy arrays as members of classes so the problems will be more visible.

5. Emulators don't support triggering GC, just navigate a bit in the UI to cause GC. The amount of memory taken is meaningless if there was no GC beforehand.

6. If triggering GC manually helps then check your code. This generally helps in cases where a connection wasn't closed or a resource was not released. Some phones might not be as forgiving for these errors and GC calls just hide the problems.

7. Don't keep members for classes if possible, pass data on the stack. This is easier in threading terms and solves lots of small memory gotchas, the cost of passing on the stack is very low in todays phones. Threading mistakes are just impossible to debug on any device...

8. Speed and memory can be traded off and this can be decided on a per device basis. E.g. LWUIT's Indexed Image. Furthermore, you can store state in RMS and access it dynamically on a low memory device while keeping it accessible for a high end device.

9. Runtime.freeMemory/totalMemory has nothing to do with reality in modern devices don't treat it as a fact.

10. Device limitations are often accessible via the API, read the documentation properly e.g. JSR 184 has properties that define everything such as the maximum texture size thats supported on the device.

For LWUIT specific tips on memory and performance check our developers guide.

What Happened to My Widgets???

Your first reaction when looking at the new drop of LWUIT might be: what happened to my checkbox/radio button/combo box decoration and who do I blame?



That would be me. I changed the look of those 3 components so I'm to blame, but I have a good excuse. Previously the combo/radio button/checkbox elements were implemented as images drawn next to the particular entry. This looked very good in the looks and feels which we support by default, but this has several drawbacks:

1. Doesn't fit well with all look and feels and doesn't look equally good even in our own look and feels.

2. The obfuscator couldn't remove the image we used to draw these elements so you would pay the cost for all 3 elements even if you didn't use any of them in your application.

3. They wouldn't scale to match larger/smaller widgets.


The solution is to draw them using graphics primitives which solves all of the problems above and most specifically the look and feel problem. I know some of you will miss the old look and would want to keep it (or better yet use your own images...). For you guys we opened up a new set of API's in DefaultLookAndFeel to set the images that would be used in those situations specifically: setCheckBoxImages(Image, Image), setRadioButtonImages(Image, Image) & setComboBoxImage(Image).

So if you want your old checkboxes back just create the image objects and invoke the methods above, notice that changing the theme in the look and feel will not change these images...

Implementing A Selected Item Ticker In A List

A discussion came up in the LWUIT forum/list regarding animating list items e.g. ticker functionality where the selected element in the list is animated. This is a confusing subject because the renderer paradigm inspired by Swing doesn't allow a state and thus doesn't allow animations. So the List component itself must be animated. However if you want to animate a specific item you must implement the animation logic in the renderer code thus decoupling the functionality.
This is a bit counter intuitive but there are complexities when creating an efficient/scalable list thats infinitely customizable... Once you accept the basic premise of renderers this approach makes allot of sense.

Notice that the renderer in the code bellow is not really animated but rather allows the list (which is) to paint its state. Also notice that the list animate() method takes care to invoke its super class otherwise list traversal animations would break.

Form f = new Form("Ticker List");
class TickerRenderer extends DefaultListCellRenderer {
   private int position;
   public void resetPosition() {
       position = 0;
   }
   public void incrementPosition() {
       position += 5;
   }

   public void paint(Graphics g) {
       if(hasFocus()) {
           String text = getText();
           Style s = getStyle();
           int width = s.getFont().stringWidth(text);
           if(getWidth() >= width) {
               super.paint(g);
               return;
           }
           UIManager.getInstance().getLookAndFeel().setFG(g, this);
           int actualPosition = position % width;
           g.translate(-actualPosition, 0);
           g.drawString(text, getX() + s.getPadding(LEFT), getY());
           g.translate(width + 20, 0);
           g.drawString(text, getX(), getY());
           g.translate(actualPosition - 20 - width , 0);
       } else {
           super.paint(g);
       }
   }

   public Component getListCellRendererComponent(List list, Object value, int index, boolean isSelected) {
       super.getListCellRendererComponent(list, value, index, isSelected);
       return this;
   }
}
final TickerRenderer renderer = new TickerRenderer();
List myList = new List(new String[] {"First", "Second", "Really long string that goes on and on and on and on and on and on",
   "and on and on and on and on and on and on and on and on and on and on and on and on",
   "Well enough already with the damn long strings this is really getting old...",
   "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 0 1 2 3 4 5 6 7 8 9 !!!!....."}) {
   private long tickTime = System.currentTimeMillis();
   private int lastSelection = -1;
   public boolean animate() {
       boolean val = super.animate();
       if(hasFocus()) {
           long currentTime = System.currentTimeMillis();
           if(currentTime - tickTime > 300) {
               if(lastSelection == getSelectedIndex()) {
                   renderer.incrementPosition();
               } else {
                   lastSelection = getSelectedIndex();
                   renderer.resetPosition();
               }
               return true;
           }
       }
       return val;
   }
};
myList.setListCellRenderer(renderer);
f.setLayout(new BorderLayout());
f.addComponent(BorderLayout.CENTER, myList);
f.show();

The Many Roads to Round Buttons (Advanced LWUIT UI Customization)

One of the more important UI elements that we didn't demonstrate as part of LWUIT were irregularly shaped components, currently everything in LWUIT is a square. However, we built everything in place so it doesn't have to look that way to the casual user since components can be translucent its remarkably easy to create a rounded button.
In this post I will present several distinct approaches for creating round buttons and in them also highlight some of the more interesting and less obvious ways in which we can customize LWUIT.

Assuming we want to make all our buttons round the simplest way requires hardly any code... We can just open the resource editor and set the background image/padding to be appropriate values in order to customize the look of buttons considerably. As you can tell from the screenshots of the resource editor this is only a part of the job, we also need to disable the buttons border painting which (at the moment) can only be accomplished in code:

button.setBorderPainted(false);

This approach has some really cool benefits and some interesting drawbacks:
1. Simple - there is very little work to do in order to implement such a button.
2. Animation support - just set an animation to the background of the button with no code changes.

The drawbacks are a bit annoying:
1. Every single button must have its border disabled (wide application change).
2. Limited way to indicate that a button has focus - you can't change the image on focus, there is only one sure fire way of indicating focus using the fgSelectionColor attribute
3. Padding is hardcoded which can be an issue in some extreme cases.
4. Image scaling can look odd in some layouts.


One of the simpler approaches to rounded buttons includes setting the icon to a rounded icon, a button also provides icons for rollover and pressed allowing us to indicate the main 3 states of a button using images. I won't dwell much on this approach since its self explanatory its benefits are:

1. Perfect UI fidelity - icons aren't scaled and will look exactly as you build them.
2. State transition for the button is handled by LWUIT


The drawbacks make this appropriate only for very specific needs:

1. Code changes required globally.
2. Hardcoded text and UI size.


This is a great solution if thats what you need but its hardly global...

The next solution is simple: Derive button

When deriving a button we are essentially free to paint anything we want by overriding paint/paintBorder. This allows us to use several different "tricks" for drawing round or rounded buttons such as just "drawing" the button using the graphics primitives or drawing images as we did before. Once we derive button we can also avoid the "dirty" tricks of using padding to keep out of the edges of the button.

The first approach of drawing using graphics primitives is trivial:

Form test = new Form("Rounded Button");
Button b = new Button("Rounded") {
public void paintBorder(Graphics g) {
    g.drawRoundRect(getX(), getY(), getWidth() - 1, getHeight() - 1, 8, 8);
}

public void paintBackground(Graphics g) {
    if(getStyle().getBgTransparency() != 0) {
        if(hasFocus()) {
            g.setColor(getStyle().getBgSelectionColor());
        } else {
            g.setColor(getStyle().getBgColor());
        }
        g.fillRoundRect(getX(), getY(), getWidth() - 1, getHeight() - 1, 8, 8);
    } else {
        super.paintBackground(g);
    }
}

public void paint(Graphics g) {
    UIManager.getInstance().getLookAndFeel().drawButton(g, this);
}
};
test.addComponent(b);
test.show();

Notice that we need to override painting the background and we essentially disable translucency of buttons, the main reason for that is that LWUIT currently doesn't support translucent round rectangle drawing. If we were to paint a regular rectangle (as the default drawing does) it would "peek" from the edges of the round rectangle.

The second approach for drawing a rounded button is very similar but based on image drawing (2 images to be exact here to your right):

Form test = new Form("Rounded Button");
Button b1 = new RoundButton("Round 1");
Button b2 = new RoundButton("Round 2");
test.addComponent(b1);
test.addComponent(b2);
test.show();

class RoundButton extends Button {
private static final Image UNSELECTED;
private static final Image SELECTED;
static {
    Image s = null, u = null;
    try {
        u = Image.createImage("/RoundButton.png");
         s = Image.createImage("/RoundButtonSelected.png");
    } catch(IOException err) {
        err.printStackTrace();
    }
    UNSELECTED = u;
    SELECTED = s;
}

private Image selected;
private Image unselected;

public RoundButton(String text) {
    super(text);
    setBorderPainted(false);

}

public void paintBackground(Graphics g) {
    if(hasFocus()) {
        if(selected == null || selected.getWidth() != getWidth() || selected.getHeight() != getHeight()) {
            selected = SELECTED.scaled(getWidth(), getHeight());
        }
        g.drawImage(selected, getX(), getY());
    } else {
        if(unselected == null || unselected.getWidth() != getWidth() || unselected.getHeight() != getHeight()) {
            unselected = UNSELECTED.scaled(getWidth(), getHeight());
        }
        g.drawImage(unselected, getX(), getY());
    }
}

public void paint(Graphics g) {
    UIManager.getInstance().getLookAndFeel().drawButton(g, this);
}
}

Notice that we kept the original selected/unselected images to scale from, this prevents repeated scaling from degrading the quality of the image to a point of being unrecognized...

Both of these approaches have benefits and drawbacks some common to both and some unique for each of them. Common benefits for deriving a button:

1. Single point of extension.
2. Ability to selectively make a individual buttons rounded.
3. Full behavior flexibility and full control over look and drawing.

The drawbacks include:

1. Global code changes to make all buttons rounded.

The primitive drawing option includes the following drawbacks too:

1. No anti-aliasing (rather ugly).
2. No translucency (this is partially fixable but is a big pain to fix).


The image based solution has the following problems:

1. Doesn't scale well.
2. Occupies allot of memory.


Normally I would pick the image based round button but there are more than one ways to accomplish this, one of the cool features of LWUIT is painters which provide the ability to skin arbitrary components. Lets say I want "some" of my buttons to have rounded edges and also "some" of my labels to have them too, if I derive button I would need to derive label too thus creating multiple subclasses and polluting my code. Painters decouple the painting of a components background from the component itself thus allowing us to plugin any drawing code we may desire, normally they are implicitly installed by the theme/look and feel to draw the background image or color. You can create your own painter which is a remarkably powerful tool since it can be installed selectively onto any component to provide your own rendering logic:

class RoundButtonPainter implements Painter {
private static final Image UNSELECTED;
private static final Image SELECTED;
static {
    Image s = null, u = null;
    try {
        u = Image.createImage("/RoundButton.png");
        s = Image.createImage("/RoundButtonSelected.png");
    } catch(IOException err) {
        err.printStackTrace();
    }
    UNSELECTED = u;
    SELECTED = s;
}

private Image selected;
private Image unselected;
private Component cmp;
public RoundButtonPainter(Component cmp) {
    this.cmp = cmp;
    cmp.getStyle().setBgTransparency(0);
    ((Button)cmp).setBorderPainted(false);
}

public void paint(Graphics g, Rectangle rect) {
    if(cmp.hasFocus()) {
        if(selected == null || selected.getWidth() != cmp.getWidth() || selected.getHeight() != cmp.getHeight()) {
            selected = SELECTED.scaled(cmp.getWidth(), cmp.getHeight());
        }
        g.drawImage(selected, cmp.getX(), cmp.getY());
    } else {
        if(unselected == null || unselected.getWidth() != cmp.getWidth() || unselected.getHeight() != cmp.getHeight()) {
            unselected = UNSELECTED.scaled(cmp.getWidth(), cmp.getHeight());
        }
        g.drawImage(unselected, cmp.getX(), cmp.getY());
    }
}
}

Form test = new Form("Rounded Button");
Button b1 = new Button("Round 1");
Button b2 = new Button("Round 2");
b1.getStyle().setBgPainter(new RoundButtonPainter(b1));
b2.getStyle().setBgPainter(new RoundButtonPainter(b2));
test.addComponent(b1);
test.addComponent(b2);
test.show();

You can do the same for labels etc...

What if I wanted the round button to apply to every single button out there without code changes... Sure I could use the bgImage feature mentioned before but then I won't have the flexibility of rendering as I see fit. The solution for this problem is to derive the look and feel in order to override the rendering of all buttons!

UIManager.getInstance().setLookAndFeel(new RoundButtonLookAndFeel());
Resources r1 = Resources.open("/javaTheme.res");
UIManager.getInstance().setThemeProps(r1.getTheme(r1.getThemeResourceNames()[0]));

Form test = new Form("Rounded Button");
Button b1 = new Button("Round 1");
Button b2 = new Button("Round 2");
test.addComponent(b1);
test.addComponent(b2);
test.show();

class RoundButtonLookAndFeel extends DefaultLookAndFeel {
private static final Image UNSELECTED;
private static final Image SELECTED;
static {
    Image s = null, u = null;
    try {
        u = Image.createImage("/RoundButton.png");
        s = Image.createImage("/RoundButtonSelected.png");
    } catch(IOException err) {
        err.printStackTrace();
    }
    UNSELECTED = u;
    SELECTED = s;
}

private Image selected;
private Image unselected;
public void bind(Component cmp) {
    if(cmp instanceof Button) {
        cmp.getStyle().setBgTransparency(0);
        ((Button)cmp).setBorderPainted(false);
    }
}

public void drawButton(Graphics g, Button cmp) {
    if(cmp.hasFocus()) {
        if(selected == null || selected.getWidth() != cmp.getWidth() || selected.getHeight() != cmp.getHeight()) {
            selected = SELECTED.scaled(cmp.getWidth(), cmp.getHeight());
        }
        g.drawImage(selected, cmp.getX(), cmp.getY());
    } else {
        if(unselected == null || unselected.getWidth() != cmp.getWidth() || unselected.getHeight() != cmp.getHeight()) {
            unselected = UNSELECTED.scaled(cmp.getWidth(), cmp.getHeight());
        }
        g.drawImage(unselected, cmp.getX(), cmp.getY());
    }
    setFG(g, cmp);
    super.drawButton(g, cmp);
}
}

Notice how similar this code is to the rounded button code, yet this one applies globally to all buttons using the look and feel. We need to disable some button defaults in the bind method and we then draw the button as we would when deriving the button manually.
The result is identical to the result of the previous example made by deriving the button itself only this result applies globally to every single button we create, the look and feel is limited mostly by your imagination and the cool thing about it is that it is a single PLUGGABLE extension layer that can be replaced on the fly!

Last but not least, we can combine the look and feel with painters to provide even more extensibility and pluggability as such:

class RoundButtonPainterLookAndFeel extends DefaultLookAndFeel {
public void bind(Component cmp) {
    if(cmp instanceof Button) {
        cmp.getStyle().setBgPainter(new RoundButtonPainter(cmp));
    }
}
}

How cool is that!