Skip to content

All posts by moore - 78. page

Browser Caching

It depends on how your website/web application is used, but browser caching can give you fantastic performance increases for very little effort. Especially with a database driven site that is primarily read-only (many ecommerce sites), proper browser caching can decrease the number of pages you serve per user, which in turn increases the number of users supportable by a given set of hardware.

I found this caching tutorial to be very helpful in understanding just how to cache pages, as well as RFC 2616, which states in section 13.2.4 that the Cache-control: max-age header takes precedence over the Expires header (for the browsers who speak HTTP 1.1). This examination of the support of various browsers is also excellent reading. There’s also a cool tool called the Cacheability Engine which examines caching behavior of web pages, if you don’t want to look at the headers yourself (using Fiddler or LiveHTTPHeaders). I encountered occasional errors with the engine, but it was pretty neat to use.

[tags]browser caching[/tags]

HTTP Technology | moore |

GWT Internet Explorer quirks

I am building another GWT component and ran across two IE quirks that I would like to share.

1. Multiple modules on one page: I could not find a way to have two GWT modules on the same page–that is, having two gwt:module meta tags on one page. (Having two on one page works fine in FireFox, as long as gwt.js is only included once.) Instead, I followed the advice in this post, and created a hybrid module that inherited from both modules I wanted to display. In the two modules I was inheriting from, I simply check to see if a span is available to put the module’s UI in. (Don’t forget that inheritance is defined differently for GWT than for a typical programming language.)

2. Cookie size limits: Cookies set by GWT have limit in size in IE. Again, I didn’t see the same behavior with FF1.5, but IE 6.00.29 has a limit of 802 characters in the value portion of a cookie. (The name I was using was 5 characters long.) Longer values can be set, and are sent to the server, but I could not find a way to read cookies with a length longer than 802. I didn’t drop down into JSNI because I looked at the Cookies source code and couldn’t see what I’d do different.

Update Dec 5 2006: I was trying to nail this down to submit a bug report to Google, but saw some really weird behavior with IE’s cookies. It seemed to handle a cookie longer than 802, then it seemed to not handle such a cookie, and now it is handling a cookie with value length of 1202. See this page for an example. The first two cookies have a length of 802, the second two have a length of 1202. Weird!

[tags]cookies,GWT, Google Web Toolkit, Internet Explorer, multiple modules[/tags]

GWT | moore |

Uploading a file in php using an iframe

I recently wrote a PHP file upload application. Yawn, right? There’s a manual entry on this task, which tells you just how often folks need to do it.

Well, I threw in a few new tricks–they were new to me at least. Basically, instead of posting to a page in the typical way, I post to a 1px by 1px invisible iframe, which then generates some javacript:

<form action="/upload.php" enctype="multipart/form-data"
method="post" target="target_upload">
<iframe id="target_upload" name="target_upload"
style="border: 0pt none ; width: 1px; height: 1px">

———–

<script type="text/javascript">
if (parent && parent.uploadFinished) {
parent.uploadFinished(<?php echo $result;?>);
} </script>

The next question is how to communicate from the iframe, which is handling the processing, to the parent window, which is where the user interface is. The upload process parses the file and puts it in a database. If there are any errors, the parsing code ouptuts “0”, otherwise, it returns “1”. This value is passed to a javascript function which is written to the iframe. It looks something like the above, where the uploadFinished function basicly shows divs containing an appropriate message.

How is this superior from a regular page post? Not by very much. It’s not truly asynchronous–you still see the browser wheel spin. The only browser actions you can take while an upload is proceeding are those that open in a new window. That’s not much, but it’s more than a normal post allows you to do. In addition, this is relatively easy to do and you don’t have to deal with creating a new ‘successful upload page’.

[tags]PHP, file upload[/tags]

PHP Programming | moore |

Destroying robot generated Tomcat sessions

A large effort goes into creating sites that are crawlable by robots, such as Google, Yahoo! and other search engines. However, these programs can create a large number of sessions, if the site is based on servlet technology. Per the servlet spec (the 2.3 specification, page 50), if a client never joins a session, new sessions will be created for each request.

A session is considered new when it is only a prospective session and has not been established. Because HTTP is a request-response based protocol, an HTTP session is considered to be new until a client joins it. A client joins a session when session tracking information has been returned to the server indicating that a session has been established. Until the client joins a session, it cannot be assumed that the next request from the client will be recognized as part of a session.The session is considered to be new if either of the following is true:

  • The client does not yet know about the session
  • The client chooses not to join a session.

These conditions define the situation where the servlet container has no mechanism by which to associate a request with a previous request.

Since all these extra sessions take up memory, and are long lived, a client asked me to look into a way to invalidate them. (I’m not the first person to run into this problem.) The easiest way to do that was to build a filter that examined the User-Agent HTTP header; here’s a nice list of User-Agent values. If the client was any of the robots, we could safely invalidate the session. For some reason, in with Tomcat 4.1, I needed to run session.isNew(); before running session.invalidate();, otherwise the session wasn’t destroyed. The filter was placed at the end of the request chain, as outlined in this article, by calling chain.doFilter(request, response); before the invalidation filter looked at the request or response.

I haven’t seen any performance problems with creating a session and then throwing it away, probably because java is so good at garbage collecting short lived objects. If I did, conditionally disabling session participation in a JSP might be an option to pursue.

Java Tomcat | moore |

Installing the median user defined function on MySQL

I just re-read “How To Lie With Statistics”, which is so good I think it should be required reading in every middle school. In it, the author makes the point that there are three kinds of ‘averages’: arithmetic mean, median and mode (here I am, contributing to Wikipedia’s dominance, due to my laziness in looking up alternative definitions of statistical concepts). In general, the median is the most informative average, because it’s not skewed by a small number of outliers.But mysql (and other databases I’ve worked on) don’t natively supprt the medan, whereas I believe most support average (by which they mean ‘arithmetic mean’). Sure, you can use a stored procedure (as suggested here for PostgreSQL. However, I’m working with MySQL 4, which does not support stored procs. However, there is another solution: user defined functions. These seem like stored procedures, except you have to write them in C (or C++).

Now, I’m not a C programmer. Luckily, someone has written and released a set of mysql user defined functions that include median (as well as many other statistical manipulations). The bad news is that it hasn’t been updated for years. The good news is that with a bit of luck and many downloads, I was able to get the median function working on mysql, both on windows as a dll, and on linux as a shared library. To repeat, I am not a C programmer, so if you see any head thumping errors below, please let me know and I’ll update this document.

First off, I was working with these versions of mysql: c:\Program Files\MySQL\MySQL Server 4.1\bin\mysql.exe Ver 14.7 Distrib 4.1.10a, for Win95/Win98 (i32) and mysql Ver 14.7 Distrib 4.1.7, for pc-linux (i686)

To get median working on windows, you need to:

  1. Download the mysql-udf tarball.
  2. patch the files if you’re running a version of mysql greater than 4.1.1. patch available here, or the patched tarball is here.
  3. Download and install Visual C++ Express. (If you have a C compiler on Windows, you can skip this step and the next. Oh, and the ones following that will probably be different. (Here’s a blog post about creating a UDF using Visual Studio C++ 2003.)
  4. Download and install the platform SDK; I only followed through step 3. If you don’t, you’ll get ‘windows.h’ errors when you try to compile the UDF.
  5. Untar the myslq-udf tarball. Patch if needed.
  6. Install the mysql header files. I was able to do this via the Windows Installer, which let me modify my existing mysql installation; I had to add the ‘C Include Files / Lib Files’ feature.
  7. Create a new directory. Copy udf_median.cc from the untarred directory to this new directory.
  8. Create a new file in that directory called udf_median.def. This file contains all the methods the UDF is exporting. Or you can just download the file I used here.
  9. Open Visual C++ Express
  10. Choose File / New / Project From Existing Code. Hit Next. Browse to the directory you just created. Create a name for the project. Hit Finish
  11. Edit the udf_median.cc file and comment out the #ifdef HAVE_DLOPEN line as well as the corresponding #endif. If I didn’t do this, I kept getting link errors, as I guess everything between those preprocessor directives was not being compiled.
  12. Add the mysql include files: right click on the project and choose properties. Expand ‘Configuration Properties’ then ‘C/C++’ and click ‘General’. On the right, add an include directory. Navigate to the Mysql include directory and add that.
  13. Add the module definition file: right click on the project and choose properties. Expand ‘Configuration Properties’ then ‘Linker’ and click ‘Input’. Add ‘udf_median.def’ to the key ‘Module Definition File’.
  14. Make sure VC knows this is a DLL: right click on the project and choose properties. Expand ‘Configuration Properties’ and click ‘General’. Choose ‘Dynamic Library (.dll)’ for Configuration Type. If you don’t do this, you’ll get errors like: error LNK2019: unresolved external symbol _WinMain because the compiler thinks you’re trying to build an application.
  15. Right click on the project and choose ‘Build’. This gives you a DLL in the Debug directory.
  16. Copy the DLL to the bin directory of your mysql installation.
  17. Create the function by logging in to mysql and running this command: CREATE AGGREGATE FUNCTION median RETURNS REAL SONAME 'udf_median.dll';. (The user you log in as will need to have the ability to insert rows into the mysql tables.)
  18. Test and enjoy.

Deploying the UDF to linux is much simpler, mostly because you don’t have to install a compiler, linker, etc. I used ‘gcc (GCC) 3.3.4’.

  1. Download the mysql-udf tarball.
  2. patch the files if you’re running a version of mysql greater than 4.1.1. patch available here, or the patched tarball is here.
  3. Untar the myslq-udf tarball. Patch if needed.
  4. Edit the udf_median.cc file and comment out the #ifdef HAVE_DLOPEN line as well as the corresponding #endif.
  5. Compile and link the code. Do not use the instructions on the mysql-udf homepage. If you compile with those flags, you’ll get this error when you try to add the function: mysql> CREATE AGGREGATE FUNCTION median RETURNS REAL SONAME 'udf_median.so';
    ERROR 1126 (HY000): Can't open shared library 'udf_median.so' (errno: 22 /usr/lib/udf_median.so: undefined symbol: _Znwj)    . Rather, use the instructions in this bug report: gcc -shared -lstdc++ -I /usr/include -I /usr/local/include -I /usr/local/mysql/include/ -o udf_median.so udf_median.cc'
  6. Copy the shared library to a directory where mysql will see it. I put it in /usr/lib.
  7. Create the function by logging in to mysql and running this command: CREATE AGGREGATE FUNCTION median RETURNS REAL SONAME 'udf_median.so';. (The user you log in as will need to have the ability to insert rows into the mysql tables.)
  8. Test and enjoy.

We have pushed this UDF to production with replicated servers and haven’t seen any issues with it yet.

I want to extend my thanks to:

[tags]median, mysql, user defined functions[/tags]

Databases MySQL Programming | moore |

Comments on upgrading to version two of Google Maps

I recently upgraded a simple simple Google Map that I built last spring to display some of the cross country skiing around Boulder. You can see the original version here. I built this based on this XML.com article, using XMLHttpRequest to retrieve the data from the server and Gmarker.openInfoWindoXSLT() with this XSL stylesheet to present the data.

I decided to upgrade this map last week to version two. Since openInfoWindowXSLT is no longer supported on every browser, I feared that the upgrade would take significant effort, even though the map very simple. However, the upgrade ended up being easier than I thought it would be. To get started, I read the Google Upgrade Guide–this document explains just what changes were made in the API. The changes that affected my map included:

  • A few method name changes–centerAndZoom becomes setCenter
  • GPoint is no longer used to indicate a latitude and longitude location on a map, and its replacement, GLatLng, reverses the order of the constructor’s arguments.
  • Zoom levels are flipped around, with larger numbers now signifying higher resolutions
  • The biggest effort was modifing the code not to use the XSLT process for generating infoWindows. However, this was easier than I thought it would be. I simply wrote a javascript method that mimicked what the XSL had previously done. Sure, accessing the DOM elements was a bit of a hassle that required some debugging (that’s the win of XSL–declarative DOM access), but the alternatives were either ignore browsers that don’t have built-in XSLT support (Safari) or integrate AJAXSLT, a Google sponsored project to provide cross browser XSLT support. If this were a larger project that depended on more XSLT, I probably would have done the latter.

Upgrading my (admittedly very simple map) took about 1.5 hours. Visit the new map and take a look at the code.

[tags]google maps upgrade[/tags]

Google Maps Programming | moore |