Skip to content

All posts by moore - 59. page

Using JSP as a templating language for GWT

I wanted to share a solution to a problem a client of mine had–needing the same (or very similar) content in two places–in a lightbox, managed by GWT, and in an HTML page.  The content itself had GWT widgets inside it (that would need to be functional in both places).

The solution was to expose the HTML content as a JSONP file for GWT consumption (and just have the HTML displayed for the HTML page).  To do this,

  • we build the HTML page (in this case, using JSTL and JSP, but any other HTML view technology would work).
  • then we build code that retrieves the page (I used HttpClient)
  • and encodes the entire HTML content as an attribute of a json object (I used json-simple)
  • and exposes it to the world via a URL

Now that we have the content as JSON encoded HTML, we need to consume it:

  • first, create a JSONP consumer, as outlined here
  • then, when you get the content, pull it into an HTMLPanel, not just a HTML widget
  • create new instances of your GWT widgets and attach them to your known span ids using the HTMLPanel.add() method.
  • if your content depends on external css files, you’ll need to add and remove them via JSNI when the lightbox is shown (if you can’t integrate the CSS into the existing site)
  • if your lightbox and html content differ slightly, you can hide or show spans via your server side presentation layer to configure your in lightbox widgets

Benefits

  • One location for display–one place to make changes
  • No layout in GWT–everything happens in HTML
  • Cross domain–since the content is exposed via JSONP, it can be consumed by javascript on any domain.

Detriments

  • Security–you’re exposing yourself to some possible security issues, though since you control both the endpoints, they shouldn’t be too bad.  Read this excellent document carefully to understand the implications.
  • Complexity–lots of moving pieces
  • Caching–JSONP calls are cached by the browser, and a control refresh doesn’t seem to refresh them.  I end up clearing my cache a lot when debugging.
  • Two calls to the server for every lightbox shown.
  • Not sure if it would work in developer mode–possibly with an external server.  Haven’t tried this.

I haven’t played around with any of the GWT templating solutions, but I don’t believe they would work in this case, because of the need for a plain, crawlable HTML page.

GWT GWT Mini Patterns | moore |

Two books of interest to freelancers

I recently read two finance books that might be of interest to freelancers.

The first is The Money Book For Freelancers, Part-Timers, and the Self-Employed by Joseph D’Agnese and Denise Kiernan.  I was pleasantly surprised to see a finance book aimed directly at people with sporadic income.  This book is written in an easy going style, with examples drawn from the author’s lives, as well as those of their friends.

The major takeaway is that you should pay yourself first–when a check comes in, allocate a percentage for taxes, a percentage for retirement, and a percentage for an emergency fund.  Do that first, so you aren’t even tempted to spend it.  I do this because of my corporate structure and SIMPLE IRA, but if you’re a new freelancer, or someone who is a schedule C worker, then it’s great advice.

Then, after a while, when you’re comfortable with the system, you can start to save for other goals.  One of the strongest parts of the book was when they encouraged you to sit down and write out your particular financial goals.  It’s tough to do, because when you do so, then you’re confronted with the hard work of trying to achieve them, but it’s great advice.

While this book was full of tips for dealing with the income side, I felt like the outgo side was covered as well.  They had advice about breaking up your expenses into required (taxes, retirement, emergency), overhead (rent/mortgage, food, insurance), and variable (eating out, classes).  Once you know your overhead and required expenses, if you get a windfall (that big client finally paid!) you can siphon that off into a separate account and use that for future months of expenses.  The authors also did a good job talking nuts and bolts, going down to the details of what type of bank accounts you need.

They didn’t really cover some things that are on the periphery of financial concerns–corporate structure, handling employees and contractors and types of insurance, for instance.  But doing so would probably have distracted from the main point of the book, so I understand.

If you’re a freelancer, and this strategy doesn’t work for you, you probably want to check this book out.

I also read Are you a Stock or a Bond by Moshe A. Milevsky.  This book is a fascinating look at the best way for modern savers with 401(k)s and IRAs to replicate what the WWII generation had–a pension.

The hook of the book is that savers should consider their lifetime earning potential and job prospects when planning a retirement asset allocation strategy.  Those who are in a more stable job (the author uses himself as an example–a tenured professor) should consider their earnings more bond-like and thus invest more in stocks (to the point of borrowing to buy stocks).  Those in a more tenuous job (farmer? I don’t remember his example) should do the opposite.

The hook, while thought provoking, is not really the focus of the book.  In fact, I was a bit disappointed that he didn’t spend more time telling you how to determine if your income stream was more stock like or more bond like.  Instead, he charges off into what you should do with your accumulated savings when you retire.

The short answer–buy an annuity.  The reason for this is that the guaranteed income from an annuity is impossible to replicate from the same amount of capital invested individually.  Why?  Because an annuity spreads payments across a population in which some will die before others.  Annuities essentially transfer wealth from those who die early to those who die later.  This asset class reduces your greatest financial risk at retirement–running out of money.  Just like a pension.

Definitely a thought provoking book, and one that would be especially useful to people approaching retirement.

Books Business | moore |

Boulder Startup Week

Come find a job in Boulder May 4 – 8, or at least enjoy the startup scene.

The startups in town are rolling out the red carpet (not really, I don’t think there is one in town) but figuratively, for you, talented developer, designer, UI, manager or startup enthusiast, to come for a visit.  There are a ton of jobs for developers, marketers, designers and managers (and we need more talent in town!).

Could be the best event highlighting the tech + fun sides of Boulder since Boco.  Positive press here.
[tags]startups, boulder, events[/tags]

Jobs | moore |

Malware Removal Guide

I had an issue last week with my Windows PC, which is currently running Windows XPSP3.  Mac users, you can stop reading right now.

Midafternoon one day, I started receiving errors when I would try to open certain executables by clicking on their icons (on the desktop, the start menu, in explorer).  The error message was “The specified path does not exist” followed by the pathname of the executable.  When I opened the file manager and browsed to that path, sure enough, the file was there.  In addition, when I shut my computer down, I was seeing the 0xc000012 error message.
There were some really weird things about this behavior.  First of all, it affected some executables (like notepad and all my browsers) but not others (like yahoo messenger).  Second, starting up the executables from the command line worked, as did starting them up by another program (as in, one of the programs which started up fine had a ‘check for updates’ option, which started FireFox just fine).  Also, I could start any program by right clicking on it and ‘running as’ a different user.  Finally, in safe mode, everything worked fine, which indicated, thankfully, that I didn’t have a hardware problem, and also gave me a chance to do a backup of everything, Just In Case.
I won’t walk you through all the links I found via google, or explain my frustration, or talk about the tools I used to try to fix the issue.  I’m just going to tell you what worked for me.  I finally stumbled on this fantastic malware and spyware removal guide from Geeks To Go.  I printed it out, downloaded all the tools, and followed the directions faithfully.  And now I’m up and running.

This was my first spyware infection in the 7 years I’ve been running Windows XP.  I hope to go another 7 years, but if I don’t, I now know where to turn to first.

[tags]malware removal,spyware removal[/tags]

Useful Tools | moore |

One customer’s experience with offshoring

I’ve been thinking about offshoring for some time, but haven’t had many interactions with someone who really used an offshore developer. Recently, I ran into a client who did try to develop some software using an offshore developer, and was able to ask him some questions about the experience.  (Disclaimer: this client owns the dating site I blogged about recently).  Plus, outsourcing/freelancing is on my mind, due to the pick.im launch.
——————-

Dan: What did you ask your offshore developer to do? Did they succeed in that?

Michael:  We asked that a dating and friendship site be created for adults with mental illness and that it mimic sites similar to Match.com. We were assured that this would be no problem, but that was far from how things transpired. We did end up with a finished product, but I wouldn’t say it was a success. Nightmare would define things more accurately.

D:  How exactly was the finished product deficient? Did it not perform well? Did it not meet your expectations? Did it not do what the developer said it would do? Were there unfixed bugs? Did it look bad?

M: There were certain features that never worked, and the developer kept maintaining that everything worked just fine.  No, it did not meet the expectation that we would have a fully operational site with the features we had requested along with our bid that was accepted.  He also promised support for his work, but when we started experiencing bugs he would only fix for additional fees.  We didn’t know anybody else and really feel trapped.  … The overall aesthetics were fine, but the internal workings had issues.

D: What were your motivations for using them? What was the cost savings?

M: Money was the only motivation. I have to say that the cost savings were significant up front. However, we were constantly investing money to fix issues that plagued the site from day one. So, we probably ended up spending more, because we eventually scrapped the website and started over.

D: Did the offshore developer build the ASP website we recently scrapped (for skadate), or was there a previous iteration?

M: No, that was the site that the offshore developer originally built.  However, we found a local guy who made some modifications and was able to fix many of the issues.

D: How did you manage them? How were the requirements documented?

M: This was very loose and managed through email. I have to say, we were very ignorant and really should have sought out the advice of an expert before proceeding with the project. We knew what we wanted the site to do, but we really had no clue what we were doing…shame on us. What  can I say though, using an offshore freelancer sounded pretty good for a social worker trying to make things happen on a shoestring budget.

D: So, did you have a master checklist of requirements, or did you just kind of email the developer feature by feature?

M: There was a list of basic features that was provided when we posted the job, but I wouldn’t call it a master list.  That would make it sound like we were prepared!  We didn’t even know what language the site should be written in.  We did not know that asp was kind of a dinosaur language…

D: How did you find them?

M: getafreelancer.com

D: How were they lacking, and/or not lacking?

M: Let me try to recap a bit. Things started out nicely. There was a lot of email communication and even some discussion over skype (although the language barrier made this difficult, so we primarily stuck with email). The great communication didn’t last though. The email responses slowed to a trickle and then all together stopped.

Fortunately, the payment funds had been deposited into an escrow account, and we refused earlier requests to release part of the funds until the work had been completed. This was a recommendation by the freelance site, and were we glad we at least had this leverage. When we finally resumed communication with the developer he threatened to stop work on the site until we released the funds. We were so exhausted that we called his bluff, and told him to go ahead and keep the site. To make a long story short, he completed the project, and we went our separate ways. Oh, and the project took twice as long as he had agreed.

Retelling this is making my blood pressure rise, so I think I’ll leave it there…

D: What would you do differently next time to increase your chances of success?

M: I would not go offshore, and to be honest, I probably wouldn’t choose someone who I couldn’t drive over to their house. I’d also make sure I sucked it up and paid for someone with development knowledge who could  help manage the project.

D: Do you think that your project failed primarily because you went offshore, or primarily because you were a relative novice at managing a technical project, or some combination of the two?

M: I’m sure a bit of both, but it definitely would have helped if we would have had someone with development expertise that could have managed the project for us.

——————-

Wow.  What a nightmare!  (For a contrasting experience, read this excellent article about using RentACoder.)

So, as I alluded to in my last couple of questions, I think that this project would have been hard for any developer, anywhere.  Obstacles to overcome include:

  • shoestring budget
  • lack of technical knowledge
  • missing/unknown requirements
  • lack of project management.  It sounds like the client wasn’t able to do this, and the developer couldn’t or didn’t.

However, I think the issues above were exacerbated by offshoring because of

  • language and timezone barriers
  • missing trust building opportunities (face to face meetings, local get togethers)
  • no real reputation risk for the developer if his work was not up to snuff (sure, his rating on a site might go down, but that’s different than running into someone you screwed at a networking event)

Anyway, a fascinating look at how offshore development can go awry.  Try to think about the total costs, rather than just the hourly wage.

Business Programming | moore |

Review of SkaDate dating software

I recently helped a client move an existing dating site from a custom ASP/MS-SQL system to an off the shelf PHP/mysql platform.

The off the shelf software we ended up choosing was SkaDate.  I haven’t really found a good review of SkaDate out there, so I asked Max Chadwick to collaborate with me on a review.  (Max provided some design, system configuration and project management, and I focused on back end system setup and data migration.)  Updated 4/2: We used SkaDate 7.5, versions 1485 and 1550.

Note that we had a challenge not present for a typical SkaDate installation: migrating ~1100 user accounts (and mail messages) from one unknown system to another.  I had to learn two very different data models and map from one to the other.  On a fixed bid project.  Whoops.

Oh well, live and learn.

Skadate pluses:

  • Price: this is a big one.  You get a lot of features for only $350.
  • Technology: it is built on the LAMP stack, so there are a lot of developers out there who can help you extend the platform.
  • Support: they have a client site with some useful PDF documentation.
  • Had a defined and documented upgrade procedure (even though it was a hassle).
  • Changing look and feel was relatively easy; we went with one of the many predefined templates and only had to hack a little bit of CSS and a couple of images.
  • Caching: SkaDate caches of php files and css.  Performance was reasonably snappy on a shared hosting account.
  • You get the source code.
  • Version control support: it wasn’t hard to find out which files/directories to pull into version control.
  • Support for 5-6 languages out of the box.  We didn’t use this, though.
  • Geographic features: SkaDate knows a lot about cities and where they are located, around the world.

SkaDate Minuses:

  • Support: they charged $90/hour $95/month for support; I didn’t end up asking them for much help, but the times I did, they immediately wanted ssh access to the server (which tweaked me out).  I’m guessing that SkaDate might be a loss leader for ‘support services’.
  • In general the administrative interface was unintuitive and could use some work.
  • Intricate object and data model: lots of indirection, and because it is PHP, no IDE to help you unravel it.  I discovered this when asked to turn off a particular feature (that didn’t have an admin setting)–I’d have to hunt through 3-4 files to find out where a UI element was set up.
  • Secretive nature: they don’t really give you any documentation until you pay for it; however, they do provide a demo and when I emailed them and explained the situation (“I’m a developer and want enough information to do a bid for a project”) they responded with some of the documentation.
  • They had a new release midway through the project, and then a user found several of the php files had been hacked.  Not exactly confidence inspiring.
  • Initial configuration of the site was complicated: all the site features were turned on and the site was pre-configured with specific payment/membership options. The tricky part was not just turning off features, but figuring out that not only did the feature need to be turned off, but that the navigation needed to be disabled for that same feature on another section of the admin area.
  • Setting up custom dating fields was cumbersome partly because of the poor interface but also because there were a number of dating fields already set up that needed to be removed.
  • The software uses dollar signs ($) in some of the automated directories.  This caused an issue with mod_security on one of the hosts we tried to use; go with one of their suggested hosting providers.

I think SkaDate is a good choice for a basic dating site; if you need just the features in the demo and you’re willing to spend some time unraveling the administration UI and you are on a tight budget. Expect some bugs and frustrations, but hey, you only paid $350!
I would hesitate to use it as a platform for a more fully featured dating site until I’d reviewed the alternatives.

Final grade: B-

[tags]skadate review,dating site software[/tags]

Dynamic Languages Web Applications | moore |

BrowserMob: Load test your applications using the cloud

Via this tweet from Matt Raible, I learned of BrowserMob.  This service allows you to easily load test your web application.

I set it up in about 2 minutes to do a simple load test of a client’s site (though 5 pages).  They make it free to ‘test drive’ their service (though the free not enough to actually stress your site).  It is extremely easy to test a path through a publicly facing system.

The report was good enough; you get screen captures of pages that have failures, and they do a good job of making some of the performance data pretty and intelligible.  Again, I didn’t really load test anything, so I didn’t examine the report as closely as I would have in a real world scenario.  The service is built using Selenium, and I believe they allow you to upload full featured selenium tests (if you have already invested in this technology, but don’t want to build out a cloud network).

This service is of particular interest to me because last year I was part of a project that built a selenium grid on Amazon EC2, using these instructions.

If we’d known about BrowserMob, I’m not sure we would have used them, as I don’t know what our budget was, but it would have been nice to have that in the evaluation mix.

[tags]browsermob, cloud services,load testing[/tags]

Cloud Computing Useful Tools Web Applications | moore |

Moving data from MS-SQL to mysql

I recently worked on a project where I needed to port data from a MS-SQL database to a mysql database.  There are programs, both payware and freeware that will help with this process, but I didn’t have ODBC access to the MS-SQL database, which these programs require.  All I had were a bunch of insert statements that looked like this:

INSERT INTO
[sample].[dbo].[users_info]([ID],[registration_day],[registration_month],
[registration_year],[registration_time],[first_name],[last_name],
[username],[userpwd],[repeat_pwd],[birth_date],[birth_day],[birth_year],
[street],[state],[city],[zip_code],[email_address],[membership_startdate],
[online],[gender]) VALUES(463,N'15',N'7',N'2009',N'9:13:52 AM',N'Homer',
N'Simpson',N'homerrocks',N'beerbeer',N'beerbeer',N'January',N'1',N'1999',
N'234 Main',N'Alaska',N'Springfield',NULL,N'homer@thesimpsons.com'),
CAST(0x9AE90000 AS SmallDateTime),
CAST(0x00009CD700000000 AS DateTime),N'Man');

I wrote a perl script to turn that dialect of SQL into a mysql friendly dialect (feel free to download it).

The most interesting parts were those CAST statements.  This forum post and this blog post helped me turn those casts into real dates.  (After loading the inserts into mysql, I did some post processing, using the helpful case statement and str_to_date function to rationalize some of the data.)

[tags]mssql, mysql, data migration[/tags]

Databases MySQL | moore |

How to get IE to accept third party cookies

Third party cookies are most often used by ad serving companies, because they are set by an image or other resource that can be pulled from a different domain, they are one way of tracking behavior across websites.

A short example: If I visit site A (www.foo.com) and site B (www.bar.com) and they both pull an image from site C (www.baz.com), it can set cookies for the site C domain (.baz.com), with a value of siteA or siteB.  Then when I visit site C, it ‘knows’ that I’ve been to sites A and B.

This can be sinister (see the wikipedia link above for privacy concerns).  However, if you have multiple web properties, then you may want to tie user behavior together across properties.  Third party tools like Google Analytics offer one way to do this, but if you want custom application behavior, then third party cookies are probably the way to go.

Firefox (at least my version of firefox, without any add ons or options changes) treats third party cookies much the same as first party cookies; that is, they just work.  However, IE requires a bit more hoop jumping, as they check the p3p compact policy.  A compact policy is basically a header you set which looks like this: P3P: CP="CAO PSA CONi OTR OUR DEM ONL".  I looked around for an easy explanation of what the various values are and how to set them in a coherent manner, but didn’t run into anything very useful.  However, I finally stumbled on this tutorial, which points you to this IBM software, which you can use to create a compact policy.  The tutorial also walks you how to use that software, which is not entirely intuitive.

As far as I can tell, IE doesn’t actually check for the existence of the corresponding policy file, nor does it care if the site does what the p3p header says it does, but it does require a valid compact policy.

After you’ve added that header, IE (versions 6-8) will accept your third party cookies.

[tags]third party cookies, compact policy, howto[/tags]

Tips Web Applications | moore |