Skip to content

Firesheep, or, beware the passwordless wireless network

I’m late to the party in writing about firesheep.  For those not into web security, it basically means that it has become easy peasy to grab someone’s credentials when they are surfing the web via a non encrypted wireless network–the kind that used to be at a few coffee shops but now are at laudromats and car repair shops.

The upshot: think long and hard about surfing any sites that you sign in to that doesn’t encrypt all traffic thereafter (I’m looking at you, Yahoo! Mail).  If you must, consider running blacksheep, change your password regularly, don’t have the website ‘remember you’, and make sure you sign out (which typically invalidates your cookies)–don’t just close the browser.

Jeff over at Coding Horror does a great job of explaining what the fundamental issues are as well as possible solutions, and I had a friend point out that you can extend firesheep with a bit of javascript.

Writing a munin plugin

Recently, we’ve run into some stability issues with our main web application.  It’s a small company, so even though I’m definitely not the ops guy, everyone is pitching in with ideas and suggestions.  One thing that the ops guy did install that has been super useful is munin.  This graphing software lets you monitor, over time, many different aspects of your web application and/or servers.  It has been invaluable in letting us know what the effects of the various changes we’ve made have been.

Questions that munin helps you answer can pretty impressive.  For example, does doubling the amount of memory available to your webapp container help stability?  How can you know unless you’re measuring stability?  What happens if you prohibit certain bots from visiting your website?  When during the day or week is your server running hottest?

We have a watchdog that monitors our main application server, and if it is not responsive, restarts it.  The watchdog also records when the restart occurred.  I decided, as a fun project, to write a plugin for munin that would graph the number of restarts per day, as a high level ‘are we more stable yet’ graph.

Writing a plugin was trivial–it’s a shell script that follows certain output formatting. All I really needed was this HOWTO and this explanation of the types of data sources, though the FAQ is, as per usual, worth a scan.

Munin is by no means perfect (my dream feature would be the ability to annotate graphs at a certain moment in time; ‘this is when we released version 2.1’), but it is a huge hammer in the IT toolbox for understanding current and historic behavior of your application.

Gratitude List

Every so often, I just need to stop and say thanks.

Thanks for my family and parents, providing all kinds of education when they raised me.

Thanks to my wife, for being patient and loving, even when I’m not my best.

Thanks to my past and current clients, colleagues and employers, for giving me amazing opportunities, challenging me and trusting me.

Thanks to the vast number of people who contribute to making the internet a fantastic place (this is a software blog, after all).  Whether you create content or build plumbing, the internet is richer for your contribution (even the guy behind lolcats, but especially stuff like this and this).

Every so often, I get wrapped up in some small drama, but nothing knocks me back on my heels and makes me say ‘Damn, I’m lucky’ like making a gratitude list, even one like this which is incomplete.

Say thanks–it feels good!

Q and A: Unexpected technical issues when contracting

I got a question from a friend who is doing some freelancing.

Perhaps an odd question, but when you do web work and run into issues that are only showing up in IE browsers, do you bill the client for the extra time it takes to try to figure out how to make the site work on that crappy browser? I know the web developer(s) we used for the farm calculator [a project for which he was the client] bill for everything, even if they are redoing something they screwed up… but I’m curious as to your way of handling things like this. I want to be fair to my client, and myself!

This is a great question, and goes beyond just “IE browser” issues.  Here was my answer:

When I run into an IE problem, I will usually stop and ask the client if making it work perfectly on IE is really important to them.  It would be useful to have stats for IE on their website (or the broader internet: http://en.wikipedia.org/wiki/Usage_share_of_web_browsers ), so they can know if 10% of their users are on IE6 or 0.01%.  It also would be useful to have an estimate for how long it will take (as long as you’re clear that it is an estimate).

If I’m billing time and materials, and I’ve had this conversation, I absolutely bill the client, but try to keep them informed as to how long this is taking me.

If it is a fixed bid, then I might go to the client and say ‘I’ve run into this issue, for this browser, which is x% of your website traffic.  There’s solution A and solution B, but both of them are things I didn’t expect.  Can we talk about this additional work’.  If they say no, I grit my teeth and deal.

So, to make it more broadly applicable, if you run into issues that you didn’t expect, here’s my advice:

  • Stop work and identify the issue.  Don’t keep spinning your wheels.
  • Gather useful facts to help the client make an informed decision (IE browser % in the example above).  Include a rough estimate if you can, but make sure the client knows it is an estimate.
  • Talk to the client about the issue and find some kind of resolution.
  • If the resolution is you doing the work, then, if you are on a fixed bid, explain how you didn’t consider this particular issue and see if the client is flexible about paying for it.
  • If the resolution is you doing the work, and you are on a time and materials contract, then bill for the extra work.
  • In either case make sure you keep the client in the loop about time spent and schedule changes due to the issue.

Surprises come up all the time.  What is important is that you come to a fair accommodation with your client.

GWT Code Splitting Process Tips

I have written before about GWT code splitting, a cool feature available in GWT.  While I am definitely no expert, it seems to be different that options offered by other, comparable web toolkits.  Just doing a quick survey (thanks, Ajaxian!), there seems to be a lot of code compilation happening.  Some of it is at deployment, pruning unused library code (like here, and here).  Some of it is aimed at making whatever javascript you write, better (like this and this).  There’s even tools aiming to document (in json) all javascript features supported by browsers, so that you can intelligently target user agents.

GWT code splitting is different from all of that.  It is developer determined split points in javascript code.  A developer says, essentially, all code after ‘this call’ should be downloaded and made accessible to the runtime when ‘this call’ is made.  More at the canonical reference; you should read this multiple times if you want to really understand code splitting.

As the document above mentions, “effective code splitting requires iteration”.  I’ve spent a fair bit of time in the last week doing some code splitting, and wanted to share my punch list (and build on my previous code splitting post).

First, understand the three major types of chunks your code will be split into.

  • Initial download
  • Leftover code
  • Exclusive fragments

The document above has a great diagram, which also includes other types of code which I have not run into.   The important thing to realize is that minimizing the initial download is important, but if any of the exclusive fragments are download, the leftover fragment will be downloaded.  Therefore, a better goal is to maximize the exclusive fragment size.

Here are places places to put split points, in order of preference

  • subsystems easily isolated (encryption library, etc).  Consider using the AsyncProxy, which is a great way to hide a subsystem
  • user actions (onclick events, etc).  Be aware of latency when toe code is downloaded and executed
  • when enabling spans are present

Use GWT.runAsync directly.  I tried to get all fancy with commands and abstracting it away, and didn’t have much success.  You can surround it with a timer, though, and that
may make sense when you are using an enabling span.

if (span is present) {
Timer t = new Timer() {

@Override
public void run() {
GWT.runAsync(new RunAsyncCallback() {
public void onFailure(Throwable caught) {
// handle error
}

public void onSuccess() {
foo();
}
});
}
};
t.schedule(150); // wait 150 ms before trying to download this code
}

You will spend a lot of time compiling, so only target a few browsers for compilation.  When I added the -compileReport option, the compile time for the main module went from 2 minutes to 8 minutes.  However, once you have the url for the compile report (something like /extras/modulename/soycReport/compile-report/SoycDashboard-0-index.html) you can actually see the results about a minute in.  At that point, you can stop the compile process.

Method call hunting will make you love your IDE java method search (control-H in eclipse–here’s a useful cheat sheet)

Watch out for constants and static methods.  In fact, I end up pulling out my constants to a class shared between GWT and my java server so that I can name spans like this
<div id="<%=Constants.SCROLL_ANCHOR_ID%>"></div> and be assured that GWT components will always be able to attach to those spans.

If module A is called by module B, C and D, avoid putting the GWT.runAsync calls in modules B, C and D.  This would force module A into the leftover code chunk, increases the number of split points, compilation time, and memory needs.  Instead, put the GWT.runAsync calls into module A, which pushes all that code into an exclusive chunk.  If you have a bunch of small splits points in your compile report, look at it carefully, especially if they are in the same class and see if you can push the split points back into the common code.

Code splitting failures (onFailure) should be dealt with the same way any remote access failure is dealt with in your application.  I haven’t found a more intelligent method than informing the user that the operation failed, however.

Set goals for splitting and try to avoid getting sucked in.  Because it is an iterative process, you can end up spending a lot of time with this.  Sometimes changes can have counter intuitive side effects, so make sure you track both total, initial and leftover code chunk sizes.

This post was based on a system with total obfuscated code size of approx 750kb, compiled using GWT 2.1.1.

Code splitting in cross domain GWT components

I just upgraded to GWT 2.1.  Lots of new features, though they are mostly aimed at building webapps (as opposed to enhancing websites).

The upgrade process was pretty painless.  A couple of notable changes (for my app):

  • <code>DeferredCommand</code> has been deprecated, and replaced with <code>Scheduler.get().scheduleDeferred</code>.  However, the API didn’t really change all that much, so this update is a simple search and replace (looks like the documentation has not yet been updated).
  • <code>HTTPRequest.getAsync</code>, which has been deprecated for years, finally disappeared.

The big win for me was code splitting when using the cross site (xs) linker.  This means that code served from one host can run on other domains and can include code splitting in the same manner as code that runs on the original host.

Why is this a big deal?  Two of our key widgets run on both our main host and subsidiary hosts.  These widgets are relatively complex, and it would be great for performance to split out that code so it wouldn’t be loaded on pages where it wasn’t shown.  This wasn’t really possible before GWT 2.1.  I’m still in the process of putting in the code split calls, but will let you know what the savings are when I’m done.

Update Jan 8, 2011: I was able to cut about 18% off the initial and leftover code download size.

What did I ship in 2010?

In the spirit of this post, I wanted to outline what I shipped this year.  (Shipped in the sense of software–building something and putting it out to the world.  Personal achievements in the past year, not least of which was getting married to my lovely wife, aren’t in the scope of this list.)

  • 41 blog posts on this blog, and 95 posts on two others I run.
  • Many, many releases (over 20) of COhomefinder.com.
  • Released version 2.0 of a dating site for the mentally ill.
  • Rewrote my Colorado CSAs directory and vastly increased the number of CSAs contained.
  • With a team, researched and presented a plan for a local Boulder currency.
  • Scoped, built (with a team), and launched an ecommerce website.
  • Set up a scholarship for a permaculture course.

As Seth said about his list, there were other projects I didn’t have the guts or follow through to launch, but I’m pretty happy with this list.  Here’s to 2011!

Shuttering a blog

Not this one!  But recently, as part of an effort to simplify my life, I’ve shuttered a couple of blogs I had been running for a while.  Here’s my list of best practices to do so:

  • Forgive yourself.  Running a blog is a lot of work, so if you don’t have time to do it adequately, it’s better to admit it than to do a poor job.
  • Try to find someone else who is interested.  If you’ve built up a following (you know how many visits you’re getting a day, right) and want to continue to see whatever topic you are blogging about more fully explored, you may be able to find someone else.  If you do, make sure to add them as a contributor, and discuss where they see things going from there–do they want to fully take over the blog, just write the occasional post, or what?
  • Announce the hibernation on the blog.  Feel free to phrase is as a long pause, or a hibernation, or whatever, but make sure you tell your readers.
  • Leave the content up for as long as you can.  Consider moving it to a different server if you need to consolidate, but if you’ve put up useful posts, people will continue to visit via search engines.
  • Leave up your contact info as well.  I’ve had several emails about my shuttered blog, and they’ve actually spiked my interest to start writing again.
  • Shut down your CTAs; for instance, I had a newsletter signup on one blog.  To me, it’s not fair to ask people to give their email address if I’m never going to send them useful content.  Along with the previous two suggestions, this is really about being respectful of your readers.

Do you have any tips for shutting down a blog in a graceful manner?  Here’s an interesting post on the topic, but written from the perspective of a more professional blogger.

Stats mix is out of beta

Statsmix, a cool dashboard builder which I previously reviewed,  is out of beta, according to an email I just received, and an announcement on their blog.

They have have added info about pricing on their site.

I took a brief look at the new release and it seems to address the pricing issue, but not the integration issue.  Still, if I were running a business and needed some custom reporting measured over time, StatsMix would merit a serious evaluation.

“Thinking in Systems” by Donella H. Meadows

I recently finished “Thinking in Systems” by Donella H. Meadows.  A primer on systems theory, this book is a very accessible introduction to these structures that influence so much of our ecology, economy and lives.

The book is only seven chapters and around 200 pages long, but covers a lot of ground.  First, she covers the basics of systems, including stocks (the thing that changes over time, like water in a bathtub or money in a bank account), flows (the movement of stocks, like a drain in a bathtub or wages deposited into a bank account) and various kinds of feedback loops (the things that act on a stock, like a thermostat monitoring bathtub temperature–a balancing loop, or interest in a bank account–a reinforcing loop).  Then she spends some time examining the some members of the systems ‘zoo’, by varying the type and number of stocks and feedback loops, and observing behaviors that arise.

She then dives a bit deeper into this behavior, and examines why systems are stable, and at the same time what kind of surprises arise from them.  She also looks at some of the archetypes of problematic systems, like the tragedy of the commons, escalation and addiction, and proposes some ways to avoid these archetypes or how to withdraw from them.  She starts off each archetype with an example drawn from the newspaper headlines when she was writing (in the 1990s).

The next chapter enumerates change points–where can you or I intervene to most effectively use our time to modify systesm we find abhorrent or unfair or wrong.  The author discusses twelve different areas to apply energy, ranging in effectiveness from changing numbers or parameters in the system, like the tax rates and the level of the minimum wage (minimal effectiveness), to transcending paradigms (maximal effectiveness).  She calls the list order “slithery” and acknowledges that this is a first draft and that points of leverage can increase or decrease in effectiveness depending on many factors.  But even having such a list is extremely useful as a starting point to think about change.

The last chapter is all about lessons from systems, of which my favorite is “Stay Humble–Stay a Learner” where she quotes a psychologist taking about the difficulty of becoming an error-embracer; we don’t just have to accept that we make mistakes, we have to admit them.

After a slow start, I really enjoyed this book.  The examples were accessible and grounded in the real world.  And she explained complicated ideas in ways that made me wonder why I hadn’t thought of it like that before–always the hallmark of a challenging and worthwhile book.  The author wrote “Limits To Growth” in the 1970s, so there are definitely progressive undertones throughout the book, but nothing over the top.

This book is a great way to get a gentle introduction into systems thinking, which is a fundamental way of understanding the world.  It’s not going to give you any easy answers, but will give you one more tool in your mental toolbox to understand the problems of the world.  I wish she had had a small section of exercises at the end, because they would have offered a deeper understanding of systems with some hands on work, but other than that, I have no quibbles.