Skip to content

Notes from a talk about DiamondTouch

I went to another University of Colorado computer science colloquium last week, covering Selected HCI Research at MERL Technology Laboratory. I’ve blogged about some of the talks I’ve attended in the past.

This talk was largely about the DiamondTouch, but an overview of Mitsubishi Electronic Research Laboratories was also given. The DiamondTouch is essentially a tablet PC writ large–you interact through a touch screen. The biggest twist is that the touch screen can actually differentiate users, based on electrical impulses (you sit on special pads which, I believe, generate the needed electrical signatures). To see the DiamondTouch in action, check out this YouTube movie showing a user playing World Of Warcraft on a DiamondTouch. (For more on YouTube licensing, check out the latest Cringely column.)

What follows are my loosely edited notes from the talk by Kent Wittenburg and Kathy Ryall.

[notes]

First, from Kent Wittenburg, one of the directors of the lab:

MERL is a research lab. They don’t do pure research–each year they have a numeric goal of business impacts. Such impacts can be a standards contribution, a product, or a feature in a product. They are associated with Mitsubishi Electric (not the car company).

Five areas of focus:

  • Computer vision–2D/3D face detection, object tracking
  • Sensor and data–indoor networks, audio classification
  • Digital Communication–UWB, mesh networking, ZigBee
  • Digital Video–MPEG encoding, highlights detection, H.264. Interesting anecdote–realtime video processing is hard, but audio processing can be easier, so they used audio processing to find highlights (GOAL!!!!!!!!!!!!) in sporting videos. This technology is currently in a product distributed in Japan.
  • Off the Desktop technologies–touch, speech, multiple display calibration, font technologies (some included in Flash 8 ), spoken queries

The lab tends to have a range of time lines–37% long term, 47% medium and 16% short term. I think that “long term” is greater than 5 years, and “short term” is less than 2 years, but I’m not positive.

Next, from Kathy Ryall, who declared she was a software person, and was focusing on the DiamondTouch technology.

The DiamondTouch is multiple user, multi touch, and can distinguish users. You can touch with different fingers. The screen is debris tolerant–you can set things on it, or spill stuff on it and it continues to work. The DiamondTouch has legacy support, where hand gestures and pokes are interpreted as mouse clicks. The folks at MERL (and other places) are still working on interaction standards for the screen. The DiamondTouch has a richer interaction than the mouse, because you can use multi finger gestures and pen and finger (multi device) interaction. It’s a whole new user interface, especially when you consider that there are multiple users touching it at one time–it can be used as a shared communal space; you can pass documents around with hand gestures, etc.

It is a USB device that should just plug in and work. There are commercial developer kits available. These are available in C++, C, Java, Active X. There’s also a Flash library for creating rapid prototype applications. DiamondSpin is an open source java interface to some of the DiamondTouch capabilities. The folks at MERL are also involved right now in wrapping other APIs for the DiamondTouch.

There are two sizes of DiamondTouch–81 and 107 (I think those are the diagonal measurements). One of these tables costs around $10,000, so it seems limited to large companies and universities for a while. MERL is also working on DiamondSpace, which extends the DiamondTouch technology to walls, laptops, etc.

[end of notes]

It’s a fascinating technology–I’m not quite sure how I’d use it as a PC replacement, but I could see it (once the cost is more reasonable) but I could see it as a bulletin board replacement. Applications that might benefit from multiple user interaction and a larger screen (larger in size, but not in resolution, I believe), like drafting and gaming, would be natural for this technology too.

CalendarTag: A simple JSP Calendar Component

Update, 8/10/2006: There’s at least one other open source calendar generation taglib that looks like it’s worth a download (even though it’s older). That author hoped that code would make it into the Jakarta Taglibs but I didn’t find any evidence of that.

I was looking around at jsp calendar components for a client. I’d previously had a hard time finding a decent one, so I didn’t hold out much hope. Luckily, this search was slightly different–I don’t need any fancy features, just a simple calendar display rendered in html, with links on the day numbers. In addition, the calendar needs to be indexable, so FlatCalendarXP, which I’ve used before, is not an option.

I looked at the HtmlCalendarBean and the Calendar Taglib, both from ServletSuite. I expected the price to be reasonable, but didn’t expect source, which was important for the client.

A bit of digging around on SourceForge found the perfect project: CalendarTag. This project looks abandoned, but the author got things into good shape before ceasing development. I’d say the project is mature, rather than abandoned. In addition, he responded to my questions (including “is this project alive?”) in less than 24 hours.

I found the documentation to be very useful. In addition, you can customize the
calendar’s output to a very large degree. I especially like the decorator, which lets you control exactly how each day is displayed by implementing a relatively simple interface or extending a a class.

Make no mistake. This is a simple component which just displays a calendar in html. There’s no support for users or events (Update 3:24. I should have read the docs more closely and said, there’s no built in support for events or users. From within the calendar, you have access to the request and so can code up event handling and/or user specific calendars.). If you want those features, I’d look at a more complex calendaring system. Calendartag does something simple and does it well.

One tip–you’ll need to have the standard-1.0.4.jar file available to your web application, as well as the calendartag-1.0-rc4.jar file, otherwise you see this rather fearsome exception:

org.apache.jasper.JasperException: org/apache/taglibs/standard/tag/el/core/ExpressionUtil
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:254)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2422)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:163)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:199)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:828)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:700)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:584)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)

root cause

javax.servlet.ServletException: org/apache/taglibs/standard/tag/el/core/ExpressionUtil
at org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:536)
at org.apache.jsp.index_jsp._jspService(index_jsp.java:59)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:210)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:295)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:241)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2422)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:163)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
at
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:199)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:828)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:700)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:584)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)

Using wget to verify on the fly page compression

I’ve written about wget before, but I just found a very cool use for it. I’m looking at ways to speed up a site, by stripping out whitespace. I found one servlet filter by googling around. However, that code has a license restriction for commercial use (you just have to pay for it).

A bit more looking and I found this fantastic article: Two Servlet Filters Every Web Application Should Have. Give it a read if you do java web development. The article provides a free set of filters, one of which compresses a servlet response if the browser can handle it. (I’ve touched on using gzip for such purposes before as well.)

I was seeing some fantastic decreases in page size. (The article states that you can’t tell, but FireFox’s Page Info command [ Tools / Page Info ] seemed to reflect the differences.) Basically, a 300% decrease in size: 50K to 5K, 130K to 20K. Pretty cool. Note that if your application is CPU bound, adding this filter will not help performance. But if you’re bandwidth bound, decreasing your average page size will help.

I couldn’t believe those numbers. I also wanted to make sure that clients who didn’t understand gzip could still see the pages. Out comes trusty wget. wget url pulls down the standard sized file. wget --header="accept-encoding: gzip" url pulls down a gzipped version that I can even ungzip and verify that nothing has changed.

The only issue I saw was that ‘ c ‘ is apparently rendered as the copyright symbol in uncompressed pages. For compressed pages, you need to use the standard: ©. Other than that, the compression was transparent. Way cool.

GWT Mortgage Calculator Conclusion

This past week has been a whirlwind, including a whole lot of learning on my part and three releases of the Colorado HomeFinder site. Just to rehash, I’ve built three versions of the software:

In general, from a java developer perspective, this is my take on the strengths and weaknesses of GWT.

Strengths:

  • Can use normal java dev/debug environment–in particular, being able to set breakpoints in what will be javascript was useful
  • Hides javascript cross browser messiness
  • Allows rich uis to be built for web using same paradigms as other windowing toolkits (event listeners, layouts, panels, etc). As some folks have said, it is Swing for the web.
  • Ability to create widget libraries–using module ‘inheritance’ you can easily leverage other folk’s work. See this list for a collection of GWT widgets.

Weaknesses:

  • Build integration–I really don’t understand why they haven’t wrapped in an ant task, though others have done it for them, and there is an -ant switch on projectCreator.cmd which generates a stub build.xml
  • Documentation–again, I’ve found the Google Groups to be tremendous, while the Google provided documentation could use some brushing up.
  • Some cross browser weirdnesses–the only serious one I saw was IE not responding to click events on one page I built. This was due to a table of width 100% next to the div where the GWT widgets were dynamically creating DOM elements. But it only showed up on IE.
  • Any ui built by GWT is not indexable by search engines. This necessarily limits what you can do with it–you can build web applications but not web sites.

How to receive craigslist searches via email

Updated 1/7/2008: Note that Squeet appears to be non functional at this time.  Craig2Mail.com still appears to work.  Please let me know (via the comments) if there are other sites that provide this valuable service.

craigslist is an online classified ad service, with everything from personals to real estate to bartering offered online. I’ve bought a table from Denver’s craigslist and I know a number of folks who have found roommates via craigslist.

If you have a need that isn’t available right now, you can subscribe to a search of section of craigslist. Suppose you’re looking for a used cruiser bike in Denver, you can search for cruisers and check out the current selection. If you don’t like what you see but don’t want to keep coming back, you can use the RSS feed link for the search, which is at the lower right corner. Put this link into your favorite RSS reader (this is a simple application that manages RSS feeds, which are essentially lists of links. I’d recommend Bloglines but there are many others out there) and you can be automatically apprised of any new cruisers which are posted.

(You find tons of stuff via RSS–stock quotes, job listings, paparazzi photos… The list is endless.)

If you don’t want to deal with yet another application, or you’re not always in your RSS reader (like me), you can set up an RSS to email gateway. That way, if your cruiser bike search is so urgent you don’t want to let a good deal get away, you receive notification of a new posting relatively quickly. If you want, you can even email it to your mobile phone.

The basic steps:

  1. Go to the Squeet signup page. Sign up for a free account. Don’t forget to verify it–they’ll send an email to the address you give them.
  2. Open up a new browser window and go to craigslist, choose the city/section you are interested in, and do a search. The example up above was ‘cruiser’, in the bike section of the Denver CL.
  3. Scroll down to the bottom of the search results and right click on the RSS link. Choose either ‘Copy Shortcut’ or ‘Copy Link Location’, depending on your browser.
  4. Switch back to the Squeet window, and click in the ‘FEED URL’ box. Paste in the link you just copied. Choose your notification time period–I’d recommend a frequency of ‘live’, since cruiser bikes in the Denver area tend to move pretty quick. Then click the subscribe button.

That’s it. Just wait for the emails to roll in and soon enough you’ll find the cruiser bike of your dreams. Just be aware that it’s not real time–I’ve seen lags from 30 minutes to 2 hours from post to email. Still, it’s a lot easier that clicking ‘Refresh’ on your browser all day and night.

Step #3: Add a tabbed interface and other enhancements

Previously, I described how to integrate a network call into the GWT mortgage calculator. (See Step #2, Step #2a and Step #2b.) Now I’m going to add an another, more complex mortgage calculator panel. This will add in more real world costs. Both the simple and advanced calculators will be available at the same time, using a tabbed interface. Additionally, I’ve added a servlet which will serve up correct JSON in hosted mode, which means that anyone who wants to download and play with this code will be able to do so without any software beyond the source, the GWT, and a modern version of Eclipse. See the new tabbed interface in action and download the source. (Please note that the interface online is not exactly the same as the interface you can build via the download–my client had me make a couple of changes.) I’ll also talk about how I integrated this calculator onto a property specific page.

Adding a tabbed interface was actually quite easy. I used a TabPanel. The only hiccup was that you cannot reuse widgets. I thought it’d be nice if the loan amount, for example, stayed the same between the tabs. However, if you place a widget in two different containers, the last one wins–the widget simply isn’t in the first container. I tested this with the VerticalPanel and the FlexTable. Instead, I had to create new widgets. I’m sure that I could have achieved synchronicity between widgets with a KeyboardListener, but the payoff didn’t seem worth the hassle. Other than that, this is a minor revision. I did some refactoring and pushed validation up to the button widgets.

The JSONServlet shows how to keep server and client source in the same tree, and also provides a simple way to toy with JSON client-server communication. I tested this and you should be able to simply untar the source, import the project into Eclipse, do a bit of GWT judo, and run the project. Steps (only tested for windows, sorry) to reproduce the final working product:

  • You will have to download the GWT since I can’t legally redistribute it.
  • Unzip it into, for example, c:\gwt
  • Download and untar the mortgage calculator source, say into c:\mtgcalc
  • Copy gwt-user.jar and gwt-dev-windows.jar into the lib directory, for example c:\mtgcalc\lib. (This lets you run the MortgageCalc-compile.cmd batch file to compile the app from the command line. If you just want to see the javascript in action, you can stop there and copy the resulting .html/.xml/.gif files to a host someplace and view in a browser.)
  • To set up the debugging, start Eclipse.
  • Import the project vai the File \ Import menu item.
  • When you’re ready to run the browser in hosted mode, edit the launcher in Eclipse. Choose the Run menu, then the Run… option.
  • Select the MortgageCalc application (under Java Application)
  • Choose the Classpath tab. Remove the gwt-user.jar and gwt-dev-windows.jar files.
  • Click under ‘User Entries’
  • Click the ‘Add External Jars’ button.
  • Navigate to where you unzipped GWT (for example, c:\gwt and select both gwt-user.jar and gwt-dev-windows.jar
  • Click ‘Open’
  • You should see the two jars with this icon next to them: . If you see this one instead: , that means you added them as jars, not as external jars, and you won’t be able to run in hosted mode. (Updated 6/21: You’ll get the error described here: [ERROR] The browser widget class could not be instantiated java.lang.UnsatisfiedLinkError: Unable to load required native library 'gwt-ll')
  • Click ‘Apply’ and then ‘Run’.

The next step is to integrate the calculator into an existing JSP and pull some information from that JSP page. Since this is tightly integrated into the Colorado HomeFinder site, code would probably not be useful, so it’s not provided. However, if you’d like to see the results, visit Colorado HomeFinder and search for a home, click through to a property and then click the ‘Mortgage Payment’ link. (Sorry this process is not streamlined, but any link I put to a specific property page on that site will expire in a few months.)

The ideal calculator gets listing price and other information from the server, and prepopulates the appropriate fields. However, the calculator needs to know what property it is being shown for; in other words, it needs to pick property specific data out of the page. There are a couple of possible methods: I could write a javascript variable to the page and use JSNI to turn it into a data structure that my compiled gwt code could read. Or I could put text into a hidden div and then use the Google DOM classes to get the data.

Since what I needed to retrieve was a few simple numeric values, I went with the DOM option. If I had a more complex data structure to communicate, using JSNI might have been a better option. I used getElementById to find the appropriate div and getInnerText to get the contents of that division. With those pieces, I could query back to the server and get the price, taxes, etc. which I would use to fill in the appropriate text boxes on the mortgage calculator.

And I believe that’s all for now. There are, of course, many places where Icould expand this calculator, but this is functional enough and we now have an understanding of the strengths and weaknesses of GWT. And that’s the subject of my next post.