Tuesday, May 25, 2010

No Longer LOST

LOST has come to a rather disappointing end so this is probably my last LWUIT themed lost demo... Fortunately due to the size of the lost cast I was able to get a decently large set of names for this particular demo.



I really don't like scrollbars on touch devices, they don't "feel" right especially once you have used a proper touch device (please enough with the resistive displays...). Every now and again we get a "scrollbar type scrolling" in LWUIT request and we always say the same thing "get over it". Scrollbars can't work on small screens, only gestures can...

Then I tried the Android address book... It was illuminating in the sense that it kept the kinetic scroll gestures I love but provided this unintrusive thumb next to the scrollbar that would "appear" when touching the screen and gently fold when you let go of the screen. The cool thing about it is how it reacts to dragging. Unlike the rest of the screen, when you drag the thumb it acts similarly to a scrollbar thumb by dragging in the opposite direction...

This on its own would not seem like a big deal but the cool part is that when you use this method a letter indicating the area of the address book where you are is displayed in the center of the screen!

This allows users to find what they are looking for much faster on Android devices when dragging their thumb, the best part is that it isn't limited just to English and can work for every language! (E.g. the iPhone's right side index of letters doesn't localize well).



I decided I want something like this in LWUIT and implemented it in the code bellow (you can check the LWUIT incubator for the full code), since its purely in LWUIT it will work for all touch devices including J2ME devices such as the Nokia 5230 in the video (a 200 USD unlocked phone!).

As a bonus my version does some things the native Android version doesn't e.g. LWUIT supports screen rotation with this feature and the Android contacts application is always in portrait mode.



There are some requirements such as the list has to be the top level component in a none-scrollable form since it needs to do its own scrolling. I overrode the scrolling behavior when detecting the thumb which is why I had to derive the list. It was also useful for me when writing the letters for the entries.



Other than that the code is relatively simple and shows how you can manipulate LWUIT's scrolling behavior completely without changing a single line of code in LWUIT itself...



/**

* This class must be the top level scrollable to work propely, all of its parent containers

* must be scrollable false!

*

* @author Shai Almog

*/


public class ThumbList extends List {

/**

* Delay for the thumb to start "returning" from the moment the user released the touch screen.

*/


private static final int THUMB_SLIDEBACK_DELAY = 2200;



/**

* Duration for the slide animation

*/


private static final int THUMB_SLIDE_DURATION = 300;



/**

* Indicates whether the thumb image is is showing

*/


private boolean thumbShowing = false;



/**

* Flags for thumb slide timeout

*/


private long thumbTimerStartTime;

private int thumbTimer = -1;



/**

* Animation motion returning the thumb to its "place"

*/


private Motion thumbSlidebackMotion;



/**

* Thumb coordinates on the screen, the X isn't the real X since the width should be added

*/


private int thumbPositionX;

private int thumbPositionY;



/**

* Flag indicating that we are now dragging via the thumb and not the gesture

*/


private boolean thumbDragMode;



/**

* Background image for the letter displayed on the screen

*/


private Image transparentRoundRect;



/**

* Font used for the letter on the screen during thumb drag mode

*/


private Font largeFont = Font.createSystemFont(Font.FACE_PROPORTIONAL, Font.STYLE_BOLD, Font.SIZE_LARGE);



/**

* Image of the thumb

*/


private Image thumb;



public ThumbList(ListModel model) {

super(model);

try {

thumb = Image.createImage("/thumb.png");



// the thumbnail image is too small for really high DPI devices, double it

if(Display.getInstance().getDisplayWidth() > 600 || Display.getInstance().getDisplayHeight() > 600) {

thumb = thumb.scaledHeight(thumb.getHeight() * 2);

}

} catch (IOException ex) {

ex.printStackTrace();

}

int size = largeFont.charWidth('W') * 3;

transparentRoundRect = Image.createImage(size, size);

Image mask = Image.createImage(size, size);

Graphics g = mask.getGraphics();

g.setColor(0);

g.fillRect(0, 0, size, size);

g.setColor(0x999999);

g.fillRoundRect(0, 0, size, size, 12, 12);

g = transparentRoundRect.getGraphics();

g.setColor(0xffffff);

g.fillRoundRect(0, 0, size, size, 12, 12);

g.setColor(0);

g.fillRoundRect(2, 2, size - 4, size - 4, 12, 12);

transparentRoundRect = transparentRoundRect.applyMask(mask.createMask());

}



/**

* We must register as an animated otherwise the thumb won't get callbacks to slide back into place

*/


protected void initComponent() {

getComponentForm().registerAnimated(this);

}



protected void deinitialize() {

getComponentForm().deregisterAnimated(this);

}



/**

* Overriding the press to detect thumb presses and to start showing the thumb

*/


public void pointerPressed(int x, int y) {

thumbShowing = true;

thumbSlidebackMotion = null;

thumbTimer = -1;

thumbPositionX = thumb.getWidth();

int myY = y - getAbsoluteY() - getScrollY();

int myX = x - getAbsoluteX() - getScrollX();

if(myX >= getWidth() - thumbPositionX && myY >= thumbPositionY && myY <= thumbPositionY + thumb.getHeight()) {

thumbDragMode = true;

return;

}



super.pointerPressed(x, y);

}



/**

* We block pointer events from the list when in thumb drag mode and move the list

* ourselves in this method

*/


public void pointerDragged(int x, int y) {

if(thumbDragMode) {

float scrollH = getScrollDimension().getHeight() + thumb.getHeight();

float ratio = ((float)y - getAbsoluteY() - getScrollY()) / ((float)getHeight());

setScrollY((int)(scrollH * ratio));

repaint();

} else {

super.pointerDragged(x, y);

}

}



/**

* We block pointer events from the list when in thumb drag mode, we activate the animation

* to hide the thumb

*/


public void pointerReleased(int x, int y) {

if(thumbDragMode) {

thumbDragMode = false;

repaint();

} else {

super.pointerReleased(x, y);

}

thumbTimerStartTime = System.currentTimeMillis();

thumbTimer = THUMB_SLIDEBACK_DELAY;

}



/**

* We don't need to override the actual paint method since we must draw our own scrollbar

*/


protected void paintScrollbarY(Graphics g) {

super.paintScrollbarY(g);

if(thumbShowing) {

float scrollH = getScrollDimension().getHeight() + thumb.getHeight();

float offset = (((float) getScrollY()) / (scrollH - getHeight()));

thumbPositionY = (int) (offset * (getHeight() - thumb.getHeight()));

g.drawImage(thumb, getX() + getWidth() - thumbPositionX, getY() + thumbPositionY);

if(thumbDragMode) {

int tx = g.getTranslateX();

int ty = g.getTranslateY();

g.translate(-tx, -ty);

int x = getWidth() / 2 - transparentRoundRect.getWidth() / 2;

int y = getHeight() / 2 - transparentRoundRect.getHeight();

g.drawImage(transparentRoundRect, x, y);

g.setFont(largeFont);

char c = getCurrentChar();

g.setColor(0xffffff);

g.drawChar(c, getWidth() / 2 - largeFont.stringWidth("" + c) / 2,

getHeight() / 2 - transparentRoundRect.getHeight() / 2 - largeFont.getHeight() / 2);

g.translate(tx, ty);

}

}

}



/**

* Gets the character matching the current list element (assumed) that the user sees on the screen

*/


private char getCurrentChar() {

float scrollH = getScrollDimension().getHeight() + thumb.getHeight();

float offset = (((float) getScrollY()) / (scrollH - getHeight()));

int item = (int)(getModel().getSize() * offset);

if(item < getModel().getSize() && item > 0) {

return ("" + getModel().getItemAt(item)).charAt(0);

}

return ' ';

}



/**

* Update thumb animation state

*/


public boolean animate() {

boolean v = super.animate();

if(thumbTimer > 0) {

long t = System.currentTimeMillis();

thumbTimer = THUMB_SLIDEBACK_DELAY - ((int)(t - thumbTimerStartTime));

if(thumbTimer < 0) {

thumbSlidebackMotion = Motion.createLinearMotion(thumb.getWidth(), 0, THUMB_SLIDE_DURATION);

thumbSlidebackMotion.start();

}

v = true;

} else {

if(thumbSlidebackMotion != null) {

thumbPositionX = thumbSlidebackMotion.getValue();

if(thumbSlidebackMotion.isFinished()) {

thumbSlidebackMotion = null;

thumbShowing = false;

}



// we still want to return true to render the last frame for a finished motion

v = true;

}

}

return v;

}

}



Saturday, May 15, 2010

Animated Gifs Everywhere

Its not that LWUIT doesn't support animated GIF's as much as the underlying implementation doesn't support them. MIDP doesn't provide any API's to extract the frames of the animated GIF's and get any indication of when to repaint them.
The solution would seem to parse animated GIF's ourselves and paint them ourselves, this isn't trivial since the animated GIF compression and painting logic is rather complex. However, Ugo Chirico seems to have done just that quite a while back!
His image class allows loading and displaying an animated GIF within LWUIT and works for many animated GIF's. It probably won't become a part of LWUIT in part due to copyright issues, but I have made some changes to the way he chose to integrate into LWUIT so you don't actually need to change LWUIT in order to integrate with his code.
Furthermore, when using my modified LWUIT implementation animated GIF's will be automatically detected and "just work" even within the HTML component and other elaborate use cases in LWUIT.
I achieved this by overriding the LWUIT implementation (the VKB version) and all that's necessary to integrate it is to invoke AnimatedGifFactory.install(); before the Display.init() call.

You can get my code from the vprise directory in the LWUIT incubator project, the project includes a small test application within as well.

This can serve as a template to anyone wishing to add additional image support to LWUIT, there are several tricks I used to achieve this:

  • Image loading requires detecting the image type from the stream, however once I started reading the stream I will need to handle all image types.... To solve this I added BufferedInputStream which guarantees that mark()/reset() would work on the stream and thus I'm able to restore the input stream to its original state in case the image is not a GIF!
  • I return an internal "native" image, but I don't use MIDP at any point. While I haven't tested it I bet my code works well on Android/RIM since I only use LWUIT logic internally.
    One would obviously need to modify the classes from which I derive to the appropriate Android/RIM implementations for this to work...
  • LWUIT animation calls update the state of the GIF, Ugo did this perfectly and I just used the implementation to map directly to his code.

Monday, May 10, 2010

Wednesday, May 5, 2010

Timeline Animations

When I was working on the original resource editor we really wanted to add an easy to use animation tool. The main issue was getting it to work on a Series 40 device with 2mb of heap space and a relatively high resolution.

I came up with the notions of IndexedImage and StaticAnimation to solve these problems by limiting the bits per pixel in the image/animation. The animations provided were very static and limited, they still suffered from memory issues and performance issues so the problems weren't completely solved.

Sometime along the way I came up with the concept of EncodedImage that allows us to leverage the JPEG/PNG compression in runtime to trade RAM overhead for performance. This allowed us far more power than IndexedImage and to a great degree we shifted all our current code to use EncodedImage rather than IndexedImage. For StaticAnimation we had no substitute, until now... Recently I committed the Timeline animation which is supposed to be a more elaborate animation model leveraging encoded image, its highly experimental and probably buggy but it already shows off the general direction.

Its not supposed to rival the elaborate use cases of Java FX/Flash/SVG, its a simple animation engine designed mostly for Spalsh screens and small effects. In the video to your right you can see the resource editor editing a simple timeline composed of 3 images a background, a logo and a flying saucer. You can generally add an image and determine an effect and a duration for said effect. Not much in general but in the right hands it can be plenty.

I hope to enhance this significantly and would like reasonable feature requests/suggestions to decide on direction. You can try the current resource editor in the webstart link at the lwuit site (click the orange "Launch" button, I can't place it here since the JavaScript doesn't work in Blogger).

The video demonstrates the creation of a simple timeline from 3 images, AnimationObjects are then added to position and move these images appropriately. One of these images is a Sprite which is automatically broken down into frames. Try the resource editor and the new classes within the animation package to get a better grasp of what I'm talking about and comment on the post.

On a different subject matter we are looking for some feedback on the HTML component from people who are taking it to production. If you represent such a company please drop us a line to lwuit at Sun.com.