RSS Pick: Clay Shirky

Clay Shirky only posts once every few months, but when he does, he posts excellent long form articles about the intersection of the internet, journalism and society. Like Dion, he has also posted to Medium.

One of my favorites:

The most important fight in journalism today isn’t between short vs. long-form publications, or fast vs. thorough newsrooms, or even incumbents vs. start-ups.

If you are into big thinking about how the internet is going to change how information is controlled, distributed and charged for, Clay is your guy.

Follow people on Hacker News, get new posts in your inbox

Ever since Jeff Beard introduced me to Hacker News, it’s been one of my favorite sites.  You can check out my profile here.

Justin Jackson pointed me to a useful tool which lets you follow activity on HackerNews: HNWatcher. This lets you set up searches for various terms, users or other interesting topics on Hacker News, and emails you the results regularly.

Right now, I simply follow a few users that say interesting things, but I could see it being as useful as google alerts for staying on top of certain topics.

If you’re an avid Hacker Newser, or are interested in what this demographic (startups, hackers, overindexed on Silicon Valley, overwhelmingly young and male) has to say, sign up for HNWatcher today.

Helping a friend gather data and reach prospects with gentle intros

coffee photo

Photo by My Aching Head

I had coffee with a friend the other day, and he shared a business idea. I thought it was an awesome idea–I certainly saw the need in the marketplace and believed he had the skillset and resources to execute on the idea.

He’s still in the exploratory phase, so I offered to send gentle intros to people in my network who I thought would benefit from his idea. (The target market is anyone with a custom web application that makes money, or anyone who builds custom web applications and is looking for a way to provide ongoing support–if that is you, contact me if you would like to learn more.)  I asked him to write a small spiel that he’d feel comfortable with me sharing.  If you are thinking of doing this, make your friend write a spiel for you.  If they can’t write a spiel, chances are they won’t be good at follow up and your intros will be wasted.

Then, I went through my LinkedIn network and put contacts into categories:

  1. this person (or the company for which they work) might want to partner with my friend
  2. this person (or the company for which they work) is a possible client for my friend’s offering
  3. this person might know people who are in categories 1 or 2.
  4. this person (or the company for which they work) is not a good fit for what my friend is working on
  5. who is this person?

And then I sent soft pitch emails to almost everyone in categories 1, 2 and 3.  The content varied based on which category someone was in, but for category 1, the email was something like:

I have a friend who owns a hosting company who is looking to talk to consulting companies about a possible new product he is thinking about offering.  Here is his spiel:


[…spiel from friend …]


I wasn’t sure if this kind of software maintenance was something that your company wanted to keep inhouse, or if you would be interested in discussing this with him.  I wanted to check before I did intros.    Is this something you think is worth learning more about?

This way, my friends and contacts on LinkedIn don’t get spammed from someone they don’t know.  Instead, they get an informative email from me, asking if they want to learn more.  If they do (and about 10% did), I do mutual introductions, and then the ball is in their court.  (Side note: here’s a great intro email etiquette guide.)

Why did I do this?  Well, there were a couple of reasons.

First and foremost, because I thought it would be a win win for both sides.  My friend gets more data about his offering and how the market will react to it.  My contacts/friends on LinkedIn learn about a new product from a trusted source.

Second, I was able to do some social network housecleaning.  I was able to ‘unlink’ with all people in category #5–it’s always nice to clean up your social graph.

Third, I reached out to people and had some interesting conversations.  Some folks I hadn’t talked to in years.  It’s good to reach out to people, and always better to do so with something of use to them, rather than a plea for work.

This was a fair bit of effort (a couple of hours).  I can’t imagine doing this monthly, but once a quarter seems reasonable, especially if I’m reaching out to a different segment of my network each time.  And I don’t have to do the whole process every time–spiel, linkedin, soft pitch, intro.  I actually like scanning news sites and simply sending interesting articles to old contacts: “Thought you might be interested in this <link> because of XXX and YYY”.  Those are super simple to send, and again, provide value and raise your profile.

Next time you talk to a friend who has a great idea, who can execute on it, and who will follow up with anybody you introduce them to, consider reviewing your social graph for prospects.  Gentle intros can benefit all three of you.

Dropwizard vs Spring Boot

wizard photo

Photo by seanmcgrath

I just rolled off a project where I chose to use Spring Boot to create a number of microservices.  I have also written a number of Dropwizard services, and wanted to compare the two while they were fresh in my mind.

They have a number of similarities, of course.  Both Spring Boot and Dropwizard create standalone jarfiles that can be deployed without needing a container.  Both favor convention over configuration and want to help you do the right thing out of the box.  Both are java based.  Both have monitoring, health checks, logging and other production nicitiies buit-in.  Both are opinionated–making a lot of choices for the developer, rather than forcing the developer to choose.  Both make it easy to leverage existing libraries.  Both have a focus on performance.

However, there were a number of reasons I choose Spring Boot over Dropwizard for the recent project, and these highlight the differences.  The first is that dependency injection is built into Spring Boot in a way that it simply isn’t with Dropwizard.  Of course, there are third party solutions for bolting DI onto Dropwizard, but we also needed a java DI framework that would handle lifecycle events, which pretty much means Spring.  Finally, this project wasn’t all about REST services, and while Dropwizard has some support for other types of services, it really is designed as a performant HTTP/REST layer, and certainly almost all the questions about Dropwizard online are about REST and APIs.  Spring Boot, on the other hand, aims to provide support for a plethora of different types of services.  Plus, you can leverage the rest of the large Spring codebase.

There are some other differences as well.  Dropwizard uses shading to build fat jars while Spring Boot uses nested jars.  As far as support, Spring, as usual, wins on the documentation front, with loads of accurate docs.  But Dropwizard definitely has a larger community around it (compare the activity of the DW google group to the Spring Boot forums).

If you are writing a REST API in Java, Dropwizard is a great choice (here’s a review of other options I did a few months ago).  If you want to build microservices that integrate with other types of components (queues, nosql, etc), Spring Boot is what I’d recommend.

Update 12/8: Per this tweet, the spring forums aren’t used because of spam, but you can find plenty of support on StackOverflow with questions tagged ‘spring-boot’.

Thoughts on Amazon CloudFormation

cloud formation photo

Photo by eschipul

I recently set up Amazon CloudFormation for a fairly complicated application in AWS.  For those unfamiliar with this service, it allows you specify a number of AWS resources in a declarative way in a JSON document, create them all at once (it’s called a ‘stack’), manage them as one entity, and destroy them.  You are billed just as you would be if you created the resources by hand.  But it’s a versionable, replicable way to create resources.

The distributed application for which I was creating the stack had the following components:

  • queues (SQS)
  • databases (dynamodb, including secondary indices)
  • compute (EC2)
  • alarms (Cloudwatch)
  • storage (S3)
  • a VPC and Subnets
  • event logging (kinesis)
  • hadoop (Elastic Map Reduce)

The last four items were not configured by the CloudFormation template I wrote.  S3, VPC and subnets because I leveraged existing resources, and Kinesis and EMR because they are not supported by CloudFormation.  (Kinesis has some support, but CloudFormation doesn’t allow you to specify a name of a stream, which makes it pretty useless when you want to post or read from a specific stream.)  However, while it would be preferable to have everything specified in CloudFormation, partial stack creation was useful–I just documented the other requirements in the CloudFormation template–because:

  • resource configuration like queue timeouts, names, read throughput, etc can be applied uniformly–consistency is enforced.
  • the infrastructure is defined and documented in one place, allowing a new developer to get up to speed quickly.
  • tags can be applied uniformly.
  • CloudFormation supports parameters, so that you can preface every resource with a deployment environment specific variable (‘stage’, ‘dan-dev’, etc), or have different DynamoDB throughput for different deployments.
  • if different configuration needs to be tested, you can stand up a new stack in minutes and test it.
  • the template can be stored in your version control system, allowing someone to see how things changed over time.  Yay, commit logs!

There were some other possible benefits I just didn’t have time to explore fully before the project wound down.

  • autoscaling groups seemed like they’d be extremely useful.  These aren’t a CloudFormation only tool, but CloudFormation seems an ideal way to define and use them.
  • the ability to create and delete stacks opened up the possibility of creating developer specific environments for debugging issues.

If you are going to start with CloudFormation, I highly recommend setting up an initial environment by hand, and then running CloudFormer, a small application written by Amazon which reads from your existing AWS infrastructure and generates a CloudFormation template.  I used CloudFormer to create a template for everything in our AWS account, and then picked and chose what was pulled over to the new template.  There were a few issues with this though:

  • There was a bug in the CloudFormation documentation for DynamoDB schemas.  You want to use this syntax: "KeySchema": { "HashKeyElement": { "AttributeName": "attrname", "AttributeType": "S" }, ... }.  CloudFormer generated them correctly, however.
  • CloudFormer coerces names of some resources resources including VPCs and subnets to strings, and I had to back those out when I wanted to use existing resources.

Other than not being able to fully define an application (because of dependencies on unsupported AWS tools like Kinesis and EMR), what other downsides does CloudFormation have?

  • it locks you into AWS.  Openstack Heat is an alternative that works across clouds, or so I read.  And, really, once you decide on AWS, is a infrastructure creation script going to be the one thing that keeps you from moving?
  • it is tied to infrastructure creation (though there is resource by resource support for in place updates).  If you want to modify one queue setting, you have to tear down and create anew the entire stack.  I found this to be relatively quick (15 min or so).
  • you are still writing scripts in the UserData section of the EC2 definition to set up your server environment.

After this experience, and reviewing my thoughts above, I believe the sweet spot of CloudFormation is setting up dev and QA environments quickly, and documenting infrastructure choices when you are committed to AWS.

My Experiences with a Digital Sabbath

I’ve tried a digital sabbath a few times in the past year.  If you aren’t familiar with the concept, it means taking one day a week and putting away all your digital devices.  No smartphones, tablets, laptops or desktop computers (do people still use those?  I do!).  For one day, I even skipped making phone calls.  Focus on the here and now.  Read a book.  Play with your kids.  Go outside.  Do that home improvement project you’ve been meaning to get to.  Look, there’s even a website about the digital sabbath!

I’ve done this a few times and it is tough.  Why?  If I have any questions about anything, I reach for my phone or tablet–when does Home Depot open?  How do I cook sunchokes?  That is relatively easy to counter–just prepare ahead of time, or accept not knowing.  I’ve even been known to pull out a copy of the white pages (yes, they still distribute that).

I also feel I am ‘maximizing’ my time–when I can read about Clojure or respond to tweets while brushing my teeth, I feel like I’m doubling my time.  It’s the same feeling I have when I run the washing machine and the dishwasher–I can sit on the couch and read because I’m ‘doing’ two jobs already!  So, a sabbath removes a major source of attention fragmentation.

The harder part of a digital sabbath is the non informational uses of my phone.  Frankly, I use my phone to escape boredom and frustration.  Of course, it is still entirely possible to ‘check out’ with a book or even daydreaming, but using a phone makes it so dang easy.  I think it is because it feels like you are accomplishing something worthwhile easily–gaining new knowledge, interacting with someone across the world.  Maybe because those use to be hard hard tasks–you had to check a book out of the library, or write someone a letter or make an expensive phone call.  Now the effort/reward has a radically decreased numerator, but my brain is still in the 1980s and doesn’t recognize it.


While I can learn plenty and make plenty of friends through your phone/tablet/internet connected whatzit, a digital sabbath forces you to ive in the now and the here.  Escapism is fine in small doses, but a digital sabbath forced me to confront how often I use my phone for that purpose.

© Moore Consulting, 2003-2014 +