Tue, 11 Aug 09
Posted in Programming, Dynamic Languages, Web Applications, Business at 8:05 pm by moore
I recently was working for a client who has their own time tracking system–they use Harvest. They want me to enter time that I work for them into that system–they want more insight into my time use than monthly invoice. However, I still use my own invoicing system, FreshBooks (more on that choice here) and will need to invoice them as well. Before the days when APIs were common, or if either of these sites did not have an API, I would have had three, equally unsavory, choices:
- Convince the client to use my system or at least access it for whatever data they needed
- Send reports (spreadsheets) to the client from my system and let them process it
- Enter my time in both places. This option would have won, as I don’t like to inconvenience people who write me checks.
Luckily, both Harvest and FreshBooks provide APIs for time tracking (Harvest doco here, FreshBooks doco here). I was surprised at how similar the time tracking data formats were. With the combination of curl, gnu date, sed, Perl and bash, I was able to write a small script (~80 lines) that
- pulled down my time data for this client, for this week, from FreshBooks (note you have to enable API access to your account for this to work)
- mapped it it from the FreshBooks format to the Harvest format
- then posted it to Harvest.
A couple of caveats:
- I still log in to Harvest to submit my time (I didn’t see a way to submit my time in the API documentation), but it’s a heck a lot easier to press one button and submit a weeks worth of time than to do double entry.
- I used similar project and task codes in both systems (or, more accurately, I set up the FreshBooks tasks and projects to map to the Harvest ones, since FreshBooks is what I had control over). That mapping was probably the most tedious part of writing the script.
You can view my script here, or at least a sanitized version thereof. it took about an hour and a half to do this. Double entry might have been quicker in the short term, but now I’m not worried about entry mistakes, and submitting my time every week is easy! I could also have used XSLT to transform from one data format to the other, but they were so similar it was easier just parse text.
Technorati Tags: getharvest, freshbooks, time tracking, process automation
Permalink
Wed, 29 Jul 09
Posted in Dynamic Languages, Web Applications, Useful Tools at 9:23 am by moore
I recently installed a open source web analytics tool called piwik. (You can demo it at that site.) I found out about it via the sourceforge.net mailing list. It was the featured project for July 2009. It bills itself as an alternative to Google Analytics (GA) (actually, right now, the home page states “Piwik aims to be an open source alternative to Google Analytics.”) and I can see why it does so. The architecture is similar, JavaScript executing on every page and sending data to a server; the interface is similar as well, with lots of whizzy Web 2.0, JavaScript heavy features and detailed data.
I had to see been using the Wusage installation that came with my web hosting service. piwik was quite a step up from that, with richer graphs, results and UI. Plus, because it was JavaScript executing and I was assured that every visit was actual visit by an actual person. Since it’s hosted on my server, I control all the data, which was a sticking point for me considering using Google Analytics.
I recently upgraded to 0.4.2, which broke the dashboard, but I’ve been assured a fix is in SVN (Update Aug 4: They no longer plan to fix the bug, but there is a workaround in that thread.). If you want to get the latest code, go here. You can download 0.4.1, the last working version I know of, here. I’ll update this to point to the piwik website when they have a release up that works. For some reason they don’t have a release archive that I could find.
So what’s good about piwik? Well compared to what, Google analytics, or other website analytics tools? This is a fundamental question, because if you are using GA just for the web stats piece, or are using some other static logfile analysis tool, piwik is well worth reviewing.
In comparison to Google Analytics
The downside is
- you have to maintain another server/database, etc. I imagine that someone will offer piwik via SAAS sometime soon, though I couldn’t find anyone doing that right now.
- it’s a beta product and is not as mature as Google Analytics, as evidenced by the 0.4.2 issue above
- some key GA features are missing (goals, funnels, etc).
In comparison to the other website analytics tools I’ve used, AWstats (which I’ve written about before and is open source) and wusage (not open source, but free with my hosting contract), piwik has
- a slick user interface
- JavaScript execution, so you know you’re getting a real browser instead of a bot (the javascript browser guarantee)
- click outs easier to track
- easier configuration
- javascript widgets available
The downside is:
This is obviously not intended to be a full, detailed analysis of all the differences between these tools, but I think that piwik has a lot of promise. They have a roadmap full of planned features but they definitely aren’t yet an alternative to Google Analytics for anyone who uses some of the more advanced features of that product. Funnels, the click overlay or goals, are all unsupported in piwik as of this version. In the forums, I saw several requests for such richer analysis tools, and in the roadmap I saw a goal tracking plugin as a blocker for version 1.0, so the team is aware of the lack.
When browsing around doing research for this post, I saw a post (sorry, couldn’t find it again) about how piwik features would be developed for smaller websites because it’s an open-source alternative, but I believe that the support of openX (an ad server company that I wrote about in the past), who is funding at least one of the developers, will prevent such feature capture. In addition, I find that open source projects that have an existing project to model themselves on (like GA), tend to try to reach feature parity. If piwik continues on its current valid path of replicating Google Analytics features, then I think it will live up to its aim.
If you’re simply using Google Analytics to see who referred traffic to your sites, or for which keywords search engines are showing your site, and you want something more open or more control of your data, piwik is a good fit. If you use any other web stats tool, and want a slicker admin interface or the javascript browser guarantee, piwik is also worth a look.
Update, 7/31: A friend pointed out this broad survey of the current state of free (as in beer) web analytics options
Technorati Tags: piwik, the javascript browser guarantee, google analytics, piwik vs google analytics, web stats
Permalink
Thu, 26 Feb 09
Posted in Dynamic Languages, Web Applications at 11:51 am by moore
Dion has a long, interesting, painful article about supporting cut, paste, and copy operations for Bespin, a web based code editor.
In our world with the editor [in webkit], we use the before events to set things up, and we have to do something pretty hacky to make it happen. The copy event itself only actually goes through if you are on an element that supports it. There are hacks around this too. For example, if you want to be able to get a copy event on a div, you need to turn on contentEdible and set the tab index to -1. Strange huh?
All I can say is ‘Ow!’
Permalink
Wed, 10 Oct 07
Posted in Java, Dynamic Languages, GWT, YUI at 4:52 pm by moore
After about a year of working with GWT, it seems to me like there are two places where it is really useful as a technology:
- Quickly building relatively sophisticated user interfaces for entire web based applications. An intranet ‘client-server’ type application, like a timesheet, would be a perfect fit. If you use Java on the server side, domain objects can even be shared.
- Building small widgets that have anything beyond the simplest logic. This is the best way to integrate GWT into an existing application–add small bits of functionality that improve the user experience. You can use GWT to manage, reuse and package this logic.
However, what GWT is best for is not the limiting factor for GWT; rather, if you aren’t a Java developer, GWT just doesn’t make sense. (I’m ignoring the fact that if a user doesn’t have JavaScript enabled, GWT doesn’t make sense, since this is a failing of almost all the Web 2.0 rich user interface toolkits).
For me, being a Java developer and a fan of Eclipse, GWT is a natural fit for a number of reasons. The Java object serialization support, the use of an IDE to code Javascript, the JRE emulation, and the event driven user interface model all make it extremely comfortable to develop in the language. If you’re already coding the server side in Java, GWT is one less language to learn (until you need to do something that isn’t provided for in the emulation libraries, or you need to use a Java 1.5 feature, or a bug leaks up through the abstraction; of course, these problems will never happen).
While I don’t have deep knowledge with other toolkits (I’ve worked slightly with the Yahoo! User Inteface Library and have toyed with Dojo), it seems to me that many many folks can get by using them; there’s no tie to Java.
If someone was going to ask me whether or not they should use GWT, I’d boil it down to the following questions:
- Are your developers familiar with Java? (If ‘no’, don’t use GWT.)
- Are your developers familiar with JavaScript? (If ‘yes’, consider not using GWT.)
- Are you integrating with an existing app? (If ‘yes’, GWT might be a good fit.)
- If so, are you planning to ‘web 2.0′-ify the existing application, or add widgets to enhance existing functionality? (If planning to ‘web 2.0′-ify existing functionality, don’t use GWT.)
On a final note, I don’t want to bag on GWT too much. GWT has improved tremendously over the past year or so, and I’m very glad to have used it. I think it’s quite cool tech, and I think it has really improved the user experience on my client’s site.
Thank you, Google, for releasing GWT and making it available for me to use.
Permalink
Thu, 13 Sep 07
Posted in Dynamic Languages, Web Applications at 9:51 am by moore
I recently was debugging the dreated “Operation Aborted” error, which was appearing in both IE6 and IE7. (Incidentally, Joel has written a great delineation of the three parts of software: design, development and debugging.) I usually do development in FF and then test in IE. But this bug is specific to IE, which made debugging harder. I was using IE6, looking at the error message and trying to find the line numbers it referenced. Pretty miserable. The fact that I was dealing with external code (Google Analytics, Google Website Optimizer) made it all the more fun!
I was testing at work, and feeling a bit frustrated. I found a reference to the Microsoft Script Debugger, but wasn’t able to find it separate from Microsoft Office.
I went home and tested on IE7, and saw the same error. I thought I’d try to find the debugger for IE7. It was easier than I thought–all you have to do is go to ‘Tools’, then ‘Internet Options…’, click the Advanced tab and uncheck ‘Disable Script Debugging (Internet Explorer)’. Then, when you visit a page with a javascript error, it asks you if you want to debug it. Fantastic!
As far as I know, that was a vanilla install of IE7. I’m thrilled they’re shipping a debugger, as it will make developing for them just a little bit easier.
Update: I was incorrect, I think that you need to install the script debugger separately. Link below.
Technorati Tags: ie7, javascript debugger
Permalink
Tue, 15 May 07
Posted in Java, Dynamic Languages, PHP at 11:10 pm by moore
Have you ever had a project written primarily in PHP, but there were some java systems that you wanted to connect to? Perhaps there was a third party library or a java API that you really want (or are required) to use, but PHP had been chosen for other reasons.
The php java bridge is your solution. The documentation is good, and the performance is pretty darn good (if you’re looking for benchmarks, search for ‘How fast is it?’). You can use the bridge over sockets, a xml based protocol via servlets, or over a mono application. The mailing list is active, and it looks like the primary developer actually answers questions on said list (as of May 2007).
I’ve always been a bit hesitant regarding cross language programming: “Great, now I get to debug in two languages that communicate via sockets! Yahoo!”. But for a certain set of circumstances, using this bridge can be the right answer.
Technorati Tags: php, java, cross-language programming, php java bridge
Permalink
Tue, 12 Sep 06
Posted in Books, Programming, Dynamic Languages, Google Maps at 8:04 pm by moore
Seven months ago, I wrote about Google Maps Gotchas. I mentioned Scott Davis’ Google Maps API Pragmatic Friday article, published by the Pragmatic Programmer folks. Well, a few things have happened since then. In April, Google released version two of their maps API (though they still haven’t set a date when version one will no longer be supported), Scott revised his article and I spent a tax deductible $8.50 to give it a read. What you’ll find below is my take on his article.
The good: first, the ordering was easy, and I received my custom PDF (complete with “Prepared Exclusively for Daniel Scott Moore” as a footer on every page) in less than 20 minutes. Scott explains in a very easy to understand fashion how to create a map. He also covers each of the API’s javascript objects and how to use them. In particular, I thought the list of events and objects that fire them (in the ‘Events’ chapter) was a good reference. Now, Google provides a class reference, but Scott’s are a bit easier to understand here’s a comparison, for the Gmarker class:
Google API:
A GMarker marks a position on the map. It implements the GOverlay interface andthus is added to the map using the GMap2.addOverlay() method.A marker object has a point, which is the geographical position where the marker is anchored on the map, and an icon. If the icon is not set in the constructor, the default icon G_DEFAULT_ICON is used.
After it is added to a map, the info window of that map can be opened through the marker. The marker object will fire mouse events and infowindow events.
Davis’ Book:
In the Core Objects section, we introduced the GLatLng. A GLatLng stores a Latitude / Longitude coordinate, but it doesn’t offer you a way to visualize it on a map. A GMarker is the way to add GLatLngs GMarker to the map for display purposes. The GMarker constructor takes a GLatLng as the only required argument.Once we have the marker, we need to tell the map to display it; map.addOverlay(myMarker) should do the trick. (Objects that you superimpose over the map are called Overlays.) You can remove the Overlays marker using map.removeOverlay(myMarker). To remove all overlays, use map.clearOverlays( ).
var myPoint = new GLatLng(38.898748, -77.037684);
var myMarker = new GMarker(myPoint);
map.addOverlay(myMarker);
Theoretically a map can support an unlimited number of markers, but anecdotal evidence suggests that performance starts to slow down significantly after a hundred or so markers. (File under, “Doc, it hurts when I do this.”)
I liked the real world examples–the fact that you could click through and see the code Scott was writing about in action on his website is a real plus. In addition, he builds a decently complex example in Chapter 7 where the user can add and delete cities. He also gives a good warning about examples that use Gmap, rather than Gmap2.
However, there were some issues. Scott’s coverage of the upgrade to version two of the API is, unfortunately, rather spotty. In his blog, the June release of that feature, and the April revision of the book). He also doesn’t cover GDownloadURL, a convenience method for XMLHttpRequest processing, or the GUnload methods. I’ll freely admit that the maps API is a moving target, and some of the omissions above may be due to that.
However, there are other problems. Though billed as a beginner book, he omits what I consider to be one of the fundamental challenges of Google Maps development–the performance obstacles large numbers of database driven markers (other than the comment mentioned above in the GMarker reference). In addition, he doesn’t cover design options, nor cross browser issues (like the transparent PNG in IE issue).
In the last chapter, he mentions good examples of mapping websites, but Scott omits references to useful websites–something that even dead tree books do. In particular, he doesn’t mention mapki.com (a wiki full of useful user provided data) nor the Google Maps group (which some users consider a primary differentiator between Google and Yahoo Maps).
One final gripe is that the 75 pages of content that I expected were really only 45–text only filled about 60% of the column width. I expect that in articles I read for free on the web, but in books that I pay for, I like a bit higher content to page ratio.
In short, this ebook is a good choice for the first time Google Maps builder. This is due to the tutorial nature of much of the book, the examples, and the explanation of typical good javascript code, such as using anonymous functions for the event handlers. It is not entirely adequate in covering version 2 of the API, possibly due to API changes, and it ignored some of the more complex aspects of the API.
If you’re looking for a folksy introduction to Google Maps api, it’s worth the $8.50 to have a coherent guide. If you’ve muddled through one google maps project, piecing together things from the API docs and various blogs, it becomes less worthwhile. But if you want some kind of discussion about complex Google Maps issues this document is not the right place to look.
Technorati Tags: Scott Davis, Google Maps, Pragmatic Fridays
Permalink
Fri, 08 Sep 06
Posted in Programming, Dynamic Languages, Databases at 10:18 am by moore
Here’s my new theory about scripting languages: everybody loves SQL (or, more likely, the data SQL makes accessible), so every scripting language evolves until it can access an RDBMS, whether it’s JavaScript, or Movable Type’s template language or server side includes.
(See also the history of SQL.)
Technorati Tags: SQL, javascript, scripting languages
Permalink