Skip to content

The Power of the Internet to Create Low Friction Marketplaces


apples photo
Photo by Royalty-free image collection

RipeNearMe is an online marketplace for locally grown food.

Ever had a peach, apple or lemon tree go bananas with a bumper crop and not know what to do with it? Or maybe you’ve seen your neighbours’ trees overloaded, left to the birds, falling to the floor and going to waste? We have too, and that’s why we started RipeNearMe: a web app that connects people through the food we each grow ourselves.

As we near the end of harvest season here in Colorado, it is great to see this kind of marketplace for free or low cost food.  Maybe the bags of zucchini at the office are on their way out?

Regardless, it would be impossible to have this kind of market if the internet (and specifically the HTTP protocol) wasn’t powering it.  The transaction costs are simply too high and the value of the goods too low.

Any other examples of low friction marketplaces that the internet enables?

Firesheep, or, beware the passwordless wireless network

I’m late to the party in writing about firesheep.  For those not into web security, it basically means that it has become easy peasy to grab someone’s credentials when they are surfing the web via a non encrypted wireless network–the kind that used to be at a few coffee shops but now are at laudromats and car repair shops.

The upshot: think long and hard about surfing any sites that you sign in to that doesn’t encrypt all traffic thereafter (I’m looking at you, Yahoo! Mail).  If you must, consider running blacksheep, change your password regularly, don’t have the website ‘remember you’, and make sure you sign out (which typically invalidates your cookies)–don’t just close the browser.

Jeff over at Coding Horror does a great job of explaining what the fundamental issues are as well as possible solutions, and I had a friend point out that you can extend firesheep with a bit of javascript.

Setting headers in a PHP include directive

I am currently working on a project that uses a PHP CMS for the bulk of the site. However, it pulls some content from a page generated by JSTL on a tomcat. I was using the fmt tag to do some pretty printing of numbers, and it looked fine when I viewed the content as served by tomcat. However, none of the formatting was carrying through when the PHP page included the jsp. Why? It appears that the fmt tag needs a locale. I confirmed that with wget.

Now, how to set the header with the PHP include. The include man page was no help, and mod_rewrite won’t let you modify headers, as far as I can tell. Once the request arrived to tomcat, that was too late.

Some digging around in the PHP documentation on fopen turned up this gem:

Custom headers may be sent with an HTTP request prior to version 5 by taking advantage of a side-effect in the handling of the user_agent INI setting. Set user_agent to any valid string (such as the default PHP/version setting) followed by a carriage-return/line-feed pair and any additional headers. This method works in PHP 4 and all later versions.

Click through for a code example. The aforementioned documentation talks about custom headers, but it worked for sending the Accept-Language header that the fmt tags needed.

Squid Notes: A fine web accelerator

I recently placed squid in front of an Apache/Tomcat based web application to serve as a web accelerator. We could have used Apache’s mod_proxy, but squid has the ability to federate and that was considered valuable for future growth. (Plus, Wikipedia uses squid, and it has worked out pretty good for them so far.) I didn’t find a whole lot of other options–Varnish looks good, but wasn’t quite documentation and feature rich enough.

However, when the application generates a page for a user who is logged in, the content can be different than if the exact same URL is visited by a robot or a user who is not signed in. It’s easy to tell if a user is signed in, because they send cookies. What was not intuitive was how to tell Squid that pages for logged in users (matching a certain header, or a certain URL pattern) should always be referred to Tomcat. In fact, I asked about this on the mailing list, and it doesn’t seem as if it is possible at all. Squid caches objects at the page level, and can’t cache just pieces of a page (like I believe, among others, OSCache can).

I compromised by deleting the cached object (a page, for example) whenever a logged in user visits it. This forces squid to go back to the origin server, guaranteeing that the logged in user gets the correct version. Then, if a non logged in user requests the page, squid again goes back to the origin server (since it doesn’t have anything in its cache). If a second logged in user requests the same page, squid serves it out of cache. It’s not the best solution, but it works. And non logged in users are such a high proportion of the traffic that squid is able to serve a fair number of pages without touching the application.

Overall I found Squid to be pretty good–even with the above workaround it was able to take a substantial amount of traffic off the main application. Note that Squid was designed to be a forward proxy (for example, a proxy at an ISP caching commonly requested pages for that ISPs users), so if you want to use it as a web accelerator (in front of your website, to increase the speed of pages you create), you have to ignore a lot of the documentation. The FAQ is a must read, especially the section on reverse proxying and the logs section.

[tags]proxy, squid,increasing webapplication performance[/tags]

Browser Caching

It depends on how your website/web application is used, but browser caching can give you fantastic performance increases for very little effort. Especially with a database driven site that is primarily read-only (many ecommerce sites), proper browser caching can decrease the number of pages you serve per user, which in turn increases the number of users supportable by a given set of hardware.

I found this caching tutorial to be very helpful in understanding just how to cache pages, as well as RFC 2616, which states in section 13.2.4 that the Cache-control: max-age header takes precedence over the Expires header (for the browsers who speak HTTP 1.1). This examination of the support of various browsers is also excellent reading. There’s also a cool tool called the Cacheability Engine which examines caching behavior of web pages, if you don’t want to look at the headers yourself (using Fiddler or LiveHTTPHeaders). I encountered occasional errors with the engine, but it was pretty neat to use.

[tags]browser caching[/tags]