Using Amazon Mechanical Turk

chess-1215079_640So, after over a decade, I finally found a use case where I had the clout and the need to use mechanical turk. I wanted to write about my experiences.

What I used it for: We were looking for some data on businesses.  We had business name, city and state, and wanted full contact information.  We paid a dime for each listing, and asked for email address and physical address.  We asked about each listing twice so that we’d have some kind of double check.

How effective was it? This varied.  If you were using the master workers, it was very effective, but slower.  If you open it up to all workers, you have to review their work more closely.  The few times I rejected someone’s task, they wrote back and asked why and tried to make it right, which was a testament to the power of the system (it records rejections).  Make sure you break the work into a couple of smaller groups so you can iterate on your instruction set (when workers asked questions on the first set, the answers went into the instructions for the second set).  We still had to review all the listings and double check any that didn’t match between both task answers, but that was a lot quicker than googling for each business and doing the research ourselves.

How much did it cost? On the order of a couple hundred bucks to process around fifteen hundred listings.

What kind of time savings did we see? Assume we had 1500 business names, and it took us 90 seconds to google the business name and find the information.  That is 1500 listings * 1.5 minutes == 37.5 hours, and this is on the low end.  Instead, it took about 2-3 hours of setup, and then 36 hours of calendar time (when I was able to do other things like sleep and work on other problems), and we were done.  Then I would say it was about 7-10 hours of review. So you are trading a couple hundred bucks for at least 20 hours of saved time.

Would I do it again? I think mturk is perfect if your problem has the following three attributes: more money than time, a task that is extremely simple, and time to review the finished product.

Other tips? You have to build it some kind of sampling for correctness. I have no idea what the quality is if you pay more than a dime per task.  Make sure you think about edge cases.  Provide tips to your workers (“check whois records as well as google”).

The Deployment Age

If you haven’t read The Deployment Age (and its follow on post), you should go read it right now.

The premise is that we’re entering a technology super cycle with the Internet and PC where the technology will become far more integrated and invisible and the chief means of financing will be internal company resources.  The focus will be on existing markets, not creating new ones, and refinement rather than innovation.

If you work in technology and are interested in the big picture, it is worth a read:

Some things we’ve learned over the past 30 years–that novelty is more important than quality; that if you’re not disrupting yourself someone else will disrupt you; that entering new markets is more important than expanding existing markets; that technology has to be evangelized, not asked for by your customers–may no longer be true. Almost every company will continue to be managed as if these things were true, probably right up until they manage themselves out of business. There’s an old saying that generals are always fighting the last war, it’s not just generals, it’s everyone’s natural inclination.

Go read it: The Deployment Age.

Good time had by all at the HN Meetup

Where can you talk about super-capacitors vs batteries, whether you should rewrite your app (hint, you shouldn’t), penetration testing of well known organizations, cost of living, and native vs cross platform mobile apps?  All while enjoying a cold drink and the best fried food the Dark Horse can offer?

At the Boulder Hacker News Meetup, that’s where.  We had our inaugural meetup today and had a good showing.  Developers, startup owners, FTEs, contractors, backend folks, front end devs and penetration testers all showed up, and, as the Meetup page suggests, ate, drank and chatted.

Hope to see you at the next one.

Twitter as conversation

tweet photo

Photo by MDGovpics

I confess, I’ve been guilty of using twitter as a broadcast only mechanism.  I have two main accounts and one of them is purely broadcast (I use tumblr to post links to Twitter and a Facebook business page–more on that experience).  But, inspired by this analysis of Marc Andreesen’s tweets during 2014, I was inspired to start using Twitter as a way to have conversation.  As of late December,  I’m committed to responding to at least one tweet when I open up Twitter.  Not a retweet, not a favorite, but an honest to god reply.

So far, I’ve enjoyed this.  If someone is tweeting out an article, it forces me to read the article critically.  If someone says something provocative, I can respond with a question.  I have started to unfollow people who just post links (like I did) because I’m looking for conversation.  Seeking conversation gets addictive pretty quickly.

The downside is that this takes more time.  Not much more (it’s only 140 characters after all) but more time.  Frankly, it can also be a bit scary to yell your opinion to the whole world (or at least that subsection of the world that is on Twitter and is reading your tweets, which is much smaller).  I do that on this blog all the time, but responses on Twitter are far less polished.

I know this is old hat to many Twitter users, but it is a new paradigm to me. I’d love to have stats on this to make myself more accountable, but I wasn’t able to find an easy way to show my Twitter usage (new tweets vs replys vs retweets)–does anyone know one?

RSS Pick: Clay Shirky

Clay Shirky only posts once every few months, but when he does, he posts excellent long form articles about the intersection of the internet, journalism and society. Like Dion, he has also posted to Medium.

One of my favorites:

The most important fight in journalism today isn’t between short vs. long-form publications, or fast vs. thorough newsrooms, or even incumbents vs. start-ups.

If you are into big thinking about how the internet is going to change how information is controlled, distributed and charged for, Clay is your guy.

My Experiences with a Digital Sabbath

I’ve tried a digital sabbath a few times in the past year.  If you aren’t familiar with the concept, it means taking one day a week and putting away all your digital devices.  No smartphones, tablets, laptops or desktop computers (do people still use those?  I do!).  For one day, I even skipped making phone calls.  Focus on the here and now.  Read a book.  Play with your kids.  Go outside.  Do that home improvement project you’ve been meaning to get to.  Look, there’s even a website about the digital sabbath!

I’ve done this a few times and it is tough.  Why?  If I have any questions about anything, I reach for my phone or tablet–when does Home Depot open?  How do I cook sunchokes?  That is relatively easy to counter–just prepare ahead of time, or accept not knowing.  I’ve even been known to pull out a copy of the white pages (yes, they still distribute that).

I also feel I am ‘maximizing’ my time–when I can read about Clojure or respond to tweets while brushing my teeth, I feel like I’m doubling my time.  It’s the same feeling I have when I run the washing machine and the dishwasher–I can sit on the couch and read because I’m ‘doing’ two jobs already!  So, a sabbath removes a major source of attention fragmentation.

The harder part of a digital sabbath is the non informational uses of my phone.  Frankly, I use my phone to escape boredom and frustration.  Of course, it is still entirely possible to ‘check out’ with a book or even daydreaming, but using a phone makes it so dang easy.  I think it is because it feels like you are accomplishing something worthwhile easily–gaining new knowledge, interacting with someone across the world.  Maybe because those use to be hard hard tasks–you had to check a book out of the library, or write someone a letter or make an expensive phone call.  Now the effort/reward has a radically decreased numerator, but my brain is still in the 1980s and doesn’t recognize it.


While I can learn plenty and make plenty of friends through your phone/tablet/internet connected whatzit, a digital sabbath forces you to ive in the now and the here.  Escapism is fine in small doses, but a digital sabbath forced me to confront how often I use my phone for that purpose.

© Moore Consulting, 2003-2017 +