Skip to content

All posts by moore - 56. page

Own your social media–install Storytlr

I guess I’m just not very trusting, because I like to have copies of my data.  I host my own blog, rather than use blogger or wordpress.com.  I host my own email (or at least one of my two main accounts).  I prefer to document interesting things on my blog, rather than a site like Quora or Stack Overflow (though I do have an account on the latter).  Heck, even though I use an open ID provider, my own domain is the master, and I just delegate to myopenid.com.

So, since I recently have been putting a bit more effort into my social media presence (you can find me on twitter here), I looked around to find a backup solution.  I did find one–Storytlr–via this article on backing up your twitter feed.  It apparently used to be a hosted service, but now is open source–code here, install instructions here.  (There’s at least one for pay service too, but then, you don’t really own your data, plus I’m cheap.)

It was pretty trivial to install.  I ran into this issue with Storytlr not recognizing that PDO was installed, but the fix (hacking the install script) worked, and I didn’t run into the Zend error also in that bug post.

I also ran into an issue where I chose an admin password of less than six characters on install.  Storytlr was happy to let me do that, but then wouldn’t let me enter the exact same password when I was logging in for the first time.  To fix this, I had to update the password column in the users table with a new MD5 string, created using this tool.

So, what does Storytlr actually give me?

  • Access to my data: I set up feeds to be polled regularly (requires access to cron) and can export them to CSV whenever I want.  And I keep them as long as I want to.
  • One single point of view of all my social content.
  • Really easy way to add more feeds if I join a new social network.  Here are the sites/networks Storytlr supports right now.

The issues I ran into are:

  • Technical issues, resolved as documented above.
  • No support for facebook.  (Well, there’s this experimental support, announced here, but nothing that is part of the project.)  This is big, given how bad Facebook is with respect to privacy.  I am not sure what my next steps are here.
  • Not wanting others to have access to my lifestream.  This was easily fixed with a Auth directive.

If you are depending on social media sites, have some technical chops, a server to host it on, and want to ensure a historical archive, you should look at Storytlr.

How to connect a Jabra VBT2050 earpiece to a Palm Centro

Everyone should use an earpiece.  My SO is reading “Disconnect: the truth about cell phone radiation, what the industry has done to hide it, and how to protect your family” and it’s some scary stuff.

I’ve struggled with setting up my earpiece enough times that I want to document what I did just now.  There are a lot of instructions on the internet, but they all seem incomplete, or aimed at a different phone.  Here’s the Jabra manual (PDF), even though it isn’t much help.

So, here are my step by step instructions on how to connect my AT&T Palm Centro with my Jabra VBT2050 earpiece.

  • Turn off the earpiece by holding down the side button until you see 4 fast blinks.  Disconnect it from the charger.
  • Then, turn off bluetooth on the phone.  Disconnect it from the charger.
  • Turn on the bluetooth on your phone
  • Make sure your phone’s bluetooth setting is ‘visible’
  • On the phone, choose ‘setup devices’
  • Choose ‘trusted devices’
  • Choose ‘add device’
  • Turn on the Jabra earpiece by holding down the button you used to turn it off
  • Continue to hold the on/off button down until you see it blinking three times slowly.
  • Press the center button of the earpiece (the one you use to connect/disconnect calls) for about 10-20 seconds, until the light on the earpiece is steady.
  • Choose ‘find more’ on your phone.
  • Select the Jabra earpiece.
  • Choose ‘OK’
  • Enter the passcode: 0000 (I’ve not found a way to change this, which seems rather idiotic).
  • Choose ‘OK’  You should now be at the ‘Trusted devices’ screen.
  • Select the Jabra earpiece from the list
  • Press the menu key on your Centro.  It’s next to the ‘Alt’ key.
  • Select ‘Connect’.  The earpiece should be blinking one time slowly.
  • Click ‘OK’ on out.  You should see a blue set of headphones next to your bars when you get to the main screen.
  • You can change your phone’s bluetooth visibility to ‘hidden’

Hope this helps!

Firesheep, or, beware the passwordless wireless network

I’m late to the party in writing about firesheep.  For those not into web security, it basically means that it has become easy peasy to grab someone’s credentials when they are surfing the web via a non encrypted wireless network–the kind that used to be at a few coffee shops but now are at laudromats and car repair shops.

The upshot: think long and hard about surfing any sites that you sign in to that doesn’t encrypt all traffic thereafter (I’m looking at you, Yahoo! Mail).  If you must, consider running blacksheep, change your password regularly, don’t have the website ‘remember you’, and make sure you sign out (which typically invalidates your cookies)–don’t just close the browser.

Jeff over at Coding Horror does a great job of explaining what the fundamental issues are as well as possible solutions, and I had a friend point out that you can extend firesheep with a bit of javascript.

Writing a munin plugin

Recently, we’ve run into some stability issues with our main web application.  It’s a small company, so even though I’m definitely not the ops guy, everyone is pitching in with ideas and suggestions.  One thing that the ops guy did install that has been super useful is munin.  This graphing software lets you monitor, over time, many different aspects of your web application and/or servers.  It has been invaluable in letting us know what the effects of the various changes we’ve made have been.

Questions that munin helps you answer can pretty impressive.  For example, does doubling the amount of memory available to your webapp container help stability?  How can you know unless you’re measuring stability?  What happens if you prohibit certain bots from visiting your website?  When during the day or week is your server running hottest?

We have a watchdog that monitors our main application server, and if it is not responsive, restarts it.  The watchdog also records when the restart occurred.  I decided, as a fun project, to write a plugin for munin that would graph the number of restarts per day, as a high level ‘are we more stable yet’ graph.

Writing a plugin was trivial–it’s a shell script that follows certain output formatting. All I really needed was this HOWTO and this explanation of the types of data sources, though the FAQ is, as per usual, worth a scan.

Munin is by no means perfect (my dream feature would be the ability to annotate graphs at a certain moment in time; ‘this is when we released version 2.1’), but it is a huge hammer in the IT toolbox for understanding current and historic behavior of your application.

Gratitude List

Every so often, I just need to stop and say thanks.

Thanks for my family and parents, providing all kinds of education when they raised me.

Thanks to my wife, for being patient and loving, even when I’m not my best.

Thanks to my past and current clients, colleagues and employers, for giving me amazing opportunities, challenging me and trusting me.

Thanks to the vast number of people who contribute to making the internet a fantastic place (this is a software blog, after all).  Whether you create content or build plumbing, the internet is richer for your contribution (even the guy behind lolcats, but especially stuff like this and this).

Every so often, I get wrapped up in some small drama, but nothing knocks me back on my heels and makes me say ‘Damn, I’m lucky’ like making a gratitude list, even one like this which is incomplete.

Say thanks–it feels good!

Q and A: Unexpected technical issues when contracting

I got a question from a friend who is doing some freelancing.

Perhaps an odd question, but when you do web work and run into issues that are only showing up in IE browsers, do you bill the client for the extra time it takes to try to figure out how to make the site work on that crappy browser? I know the web developer(s) we used for the farm calculator [a project for which he was the client] bill for everything, even if they are redoing something they screwed up… but I’m curious as to your way of handling things like this. I want to be fair to my client, and myself!

This is a great question, and goes beyond just “IE browser” issues.  Here was my answer:

When I run into an IE problem, I will usually stop and ask the client if making it work perfectly on IE is really important to them.  It would be useful to have stats for IE on their website (or the broader internet: http://en.wikipedia.org/wiki/Usage_share_of_web_browsers ), so they can know if 10% of their users are on IE6 or 0.01%.  It also would be useful to have an estimate for how long it will take (as long as you’re clear that it is an estimate).

If I’m billing time and materials, and I’ve had this conversation, I absolutely bill the client, but try to keep them informed as to how long this is taking me.

If it is a fixed bid, then I might go to the client and say ‘I’ve run into this issue, for this browser, which is x% of your website traffic.  There’s solution A and solution B, but both of them are things I didn’t expect.  Can we talk about this additional work’.  If they say no, I grit my teeth and deal.

So, to make it more broadly applicable, if you run into issues that you didn’t expect, here’s my advice:

  • Stop work and identify the issue.  Don’t keep spinning your wheels.
  • Gather useful facts to help the client make an informed decision (IE browser % in the example above).  Include a rough estimate if you can, but make sure the client knows it is an estimate.
  • Talk to the client about the issue and find some kind of resolution.
  • If the resolution is you doing the work, then, if you are on a fixed bid, explain how you didn’t consider this particular issue and see if the client is flexible about paying for it.
  • If the resolution is you doing the work, and you are on a time and materials contract, then bill for the extra work.
  • In either case make sure you keep the client in the loop about time spent and schedule changes due to the issue.

Surprises come up all the time.  What is important is that you come to a fair accommodation with your client.

GWT Code Splitting Process Tips

I have written before about GWT code splitting, a cool feature available in GWT.  While I am definitely no expert, it seems to be different that options offered by other, comparable web toolkits.  Just doing a quick survey (thanks, Ajaxian!), there seems to be a lot of code compilation happening.  Some of it is at deployment, pruning unused library code (like here, and here).  Some of it is aimed at making whatever javascript you write, better (like this and this).  There’s even tools aiming to document (in json) all javascript features supported by browsers, so that you can intelligently target user agents.

GWT code splitting is different from all of that.  It is developer determined split points in javascript code.  A developer says, essentially, all code after ‘this call’ should be downloaded and made accessible to the runtime when ‘this call’ is made.  More at the canonical reference; you should read this multiple times if you want to really understand code splitting.

As the document above mentions, “effective code splitting requires iteration”.  I’ve spent a fair bit of time in the last week doing some code splitting, and wanted to share my punch list (and build on my previous code splitting post).

First, understand the three major types of chunks your code will be split into.

  • Initial download
  • Leftover code
  • Exclusive fragments

The document above has a great diagram, which also includes other types of code which I have not run into.   The important thing to realize is that minimizing the initial download is important, but if any of the exclusive fragments are download, the leftover fragment will be downloaded.  Therefore, a better goal is to maximize the exclusive fragment size.

Here are places places to put split points, in order of preference

  • subsystems easily isolated (encryption library, etc).  Consider using the AsyncProxy, which is a great way to hide a subsystem
  • user actions (onclick events, etc).  Be aware of latency when toe code is downloaded and executed
  • when enabling spans are present

Use GWT.runAsync directly.  I tried to get all fancy with commands and abstracting it away, and didn’t have much success.  You can surround it with a timer, though, and that
may make sense when you are using an enabling span.

if (span is present) {
Timer t = new Timer() {

@Override
public void run() {
GWT.runAsync(new RunAsyncCallback() {
public void onFailure(Throwable caught) {
// handle error
}

public void onSuccess() {
foo();
}
});
}
};
t.schedule(150); // wait 150 ms before trying to download this code
}

You will spend a lot of time compiling, so only target a few browsers for compilation.  When I added the -compileReport option, the compile time for the main module went from 2 minutes to 8 minutes.  However, once you have the url for the compile report (something like /extras/modulename/soycReport/compile-report/SoycDashboard-0-index.html) you can actually see the results about a minute in.  At that point, you can stop the compile process.

Method call hunting will make you love your IDE java method search (control-H in eclipse–here’s a useful cheat sheet)

Watch out for constants and static methods.  In fact, I end up pulling out my constants to a class shared between GWT and my java server so that I can name spans like this
<div id="<%=Constants.SCROLL_ANCHOR_ID%>"></div> and be assured that GWT components will always be able to attach to those spans.

If module A is called by module B, C and D, avoid putting the GWT.runAsync calls in modules B, C and D.  This would force module A into the leftover code chunk, increases the number of split points, compilation time, and memory needs.  Instead, put the GWT.runAsync calls into module A, which pushes all that code into an exclusive chunk.  If you have a bunch of small splits points in your compile report, look at it carefully, especially if they are in the same class and see if you can push the split points back into the common code.

Code splitting failures (onFailure) should be dealt with the same way any remote access failure is dealt with in your application.  I haven’t found a more intelligent method than informing the user that the operation failed, however.

Set goals for splitting and try to avoid getting sucked in.  Because it is an iterative process, you can end up spending a lot of time with this.  Sometimes changes can have counter intuitive side effects, so make sure you track both total, initial and leftover code chunk sizes.

This post was based on a system with total obfuscated code size of approx 750kb, compiled using GWT 2.1.1.

Code splitting in cross domain GWT components

I just upgraded to GWT 2.1.  Lots of new features, though they are mostly aimed at building webapps (as opposed to enhancing websites).

The upgrade process was pretty painless.  A couple of notable changes (for my app):

  • <code>DeferredCommand</code> has been deprecated, and replaced with <code>Scheduler.get().scheduleDeferred</code>.  However, the API didn’t really change all that much, so this update is a simple search and replace (looks like the documentation has not yet been updated).
  • <code>HTTPRequest.getAsync</code>, which has been deprecated for years, finally disappeared.

The big win for me was code splitting when using the cross site (xs) linker.  This means that code served from one host can run on other domains and can include code splitting in the same manner as code that runs on the original host.

Why is this a big deal?  Two of our key widgets run on both our main host and subsidiary hosts.  These widgets are relatively complex, and it would be great for performance to split out that code so it wouldn’t be loaded on pages where it wasn’t shown.  This wasn’t really possible before GWT 2.1.  I’m still in the process of putting in the code split calls, but will let you know what the savings are when I’m done.

Update Jan 8, 2011: I was able to cut about 18% off the initial and leftover code download size.

What did I ship in 2010?

In the spirit of this post, I wanted to outline what I shipped this year.  (Shipped in the sense of software–building something and putting it out to the world.  Personal achievements in the past year, not least of which was getting married to my lovely wife, aren’t in the scope of this list.)

  • 41 blog posts on this blog, and 95 posts on two others I run.
  • Many, many releases (over 20) of COhomefinder.com.
  • Released version 2.0 of a dating site for the mentally ill.
  • Rewrote my Colorado CSAs directory and vastly increased the number of CSAs contained.
  • With a team, researched and presented a plan for a local Boulder currency.
  • Scoped, built (with a team), and launched an ecommerce website.
  • Set up a scholarship for a permaculture course.

As Seth said about his list, there were other projects I didn’t have the guts or follow through to launch, but I’m pretty happy with this list.  Here’s to 2011!

Shuttering a blog

Not this one!  But recently, as part of an effort to simplify my life, I’ve shuttered a couple of blogs I had been running for a while.  Here’s my list of best practices to do so:

  • Forgive yourself.  Running a blog is a lot of work, so if you don’t have time to do it adequately, it’s better to admit it than to do a poor job.
  • Try to find someone else who is interested.  If you’ve built up a following (you know how many visits you’re getting a day, right) and want to continue to see whatever topic you are blogging about more fully explored, you may be able to find someone else.  If you do, make sure to add them as a contributor, and discuss where they see things going from there–do they want to fully take over the blog, just write the occasional post, or what?
  • Announce the hibernation on the blog.  Feel free to phrase is as a long pause, or a hibernation, or whatever, but make sure you tell your readers.
  • Leave the content up for as long as you can.  Consider moving it to a different server if you need to consolidate, but if you’ve put up useful posts, people will continue to visit via search engines.
  • Leave up your contact info as well.  I’ve had several emails about my shuttered blog, and they’ve actually spiked my interest to start writing again.
  • Shut down your CTAs; for instance, I had a newsletter signup on one blog.  To me, it’s not fair to ask people to give their email address if I’m never going to send them useful content.  Along with the previous two suggestions, this is really about being respectful of your readers.

Do you have any tips for shutting down a blog in a graceful manner?  Here’s an interesting post on the topic, but written from the perspective of a more professional blogger.