Skip to content

Making my Twitter feed richer with Zapier and hnrss

twitter photo
Photo by marek.sotak

I read Hacker News, a site for startups and technologies, and occasionally post as well.  A few months back, I realized that the items that I post to HN, I want to tweet as well.  While I could have whipped something up with the HN RSS feed and the Twitter API (would probably be easier than Twitversation), I decided to try to use Zapier (which I’ve loved for a while).  It was dead simple to set up a Zap reading from my HN RSS feed and posting to my Twitter feed.  Probably about 10 minutes of time, and now I doubled my posts to Twitter.

Of course, this misses out on one of the huge benefits of Twitter–the conversational nature of the app.  When my auto posts happen, I don’t have a chance to follow up, or to cc: the authors, etc.

However, the perfect is the enemy of the good, and I figured it was better to engage in Twitter haphazardly and imperfectly than not at all.

APIs | moore |

Consolidate external dependency notifications using Zapier

binoculars photo
Photo by M1key.me

As I wrote over at the Geek Estate Blog, if you build your business on vendors, you should monitor them.  In the past, I’ve used a variety of services to monitor vendor services, from pingdom to wget/cron to nagios.  These services are great about telling you when some external service is unavailable, but are not so hot at telling you when a service is going to be down (for planned maintenance) or back up.

For that, you need to be monitoring, and reading, vendor announcements, however the vendor has decided to provide them, whether that is as a blog/RSS feed, twitter feed, email newsletter, a status page or something else.

However, it can be tough to monitor and read announcements in two or more places.  Here, Zapier or a similar service can help.  Pick one place to be notified.  For me, that’s typically an email inbox, because, frankly, other data sources can be ignored (except phone texts), but I’ll always check my email.

Then, use Zapier’s zaps to transform any announcements from the other sources to emails.  For instance, there is an RSS trigger for new items in a feed and a Twitter trigger for tweets from a user.  Status pages often provide RSS feeds (Google’s does).  If the service provider doesn’t provide a structured method like an RSS feed to notify you of changes, but does provide a webpage of announcements, you could look at a service like changedetection.com and have the email sent to your inbox or parsed by Zapier and pushed to your notification location.

And for the output side, you can just use Zapier’s ‘send outbound email’ action.  If you want to have all notifications pushed to your phone, an RSS reader or Twitter acount, you can use Zapier to send texts, create RSS items or tweets as well.

APIs Business Useful Tools | moore |

Building an automated postcard mailing system with Lob and Zapier

Courtesy of smoothfluid
Courtesy of smoothfluid

I was looking at automated paper mailing systems recently (and listed what I found), and was especially impressed with Lob, especially the ease of its API.

Among other printing services, Lob will let you mail a postcard with a custom PDF on both sides, or a custom PDF on one side and a text message on the other, anywhere in the USA for $0.94.  (Sorry, not sure about international postcards) The company for which I work sends out tens of thousands of postcards every quarter. The vendor which we use charges them a similar fee (less, but in the same ballpark) but there’s a manual process to deliver the collateral and no API. So an on-demand, one by one post card sending system is very interesting to me.

Note that I haven’t received the Lob postcard which I sent myself, so I can’t speak to quality. Yet.

The Lob API is a bit weird, because the request is form encoded rather than a JSON payload.  It also uses basic auth, but only the username, not the password. But the API seems to have all the pieces you’d need to generate all kinds of postcards–reminder postcards, direct mail postcards, photo postcards, etc.

After testing out the service via the web interface and cURL examples, I thought that it’d be fun to build a Zapier zap. In particular, being able to send a postcard for an entry in a Google spreadsheet seemed like a useful use case. Plus, Zapier is awesome, and I’d wanted to test out their integration environment for myself.

So, I built a Zapier integration for Lob, using the Zapier developer docs in combination with the Lob developer docs. It was actually easy. The most complicated step was translating the Zapier action data, which is a one or two dimensional array of typed data, into the Lob data format, which wanted a couple of text fields and two address arrays. Zapier has a scripting environment that let me modify data from APIs pre and post send, and even had an example about form encoded APIs. Zapier’s JavaScript scripting development environment was full featured, including syntax and error highlighting. It had no real debugging available, but I could use the venerable debug-by-log-statement method fairly easily.

Where could I take this next? Everywhere people use postcards in real life. The postcards depend on PDF files (see a sample), so if you are generating a custom postcard for each interaction things become more complex, but there are a few APIs (based on a 30 second google search, here and here) available for dynamic PDF generation. There are also limits on API call throughput, if I stuck to the Zapier integration–I could send at most 300 postcards a day, unless I managed multiple spreadsheets.

I see reminders of high value events (dentist, house maintenance, etc), contests and marketing as key opportunities for this type of service. And if I had a product where direct mail was a key component, using Lob directly would be worth serious consideration.

Regarding the Zap, I believe I cannot make this Zap available to anyone else. Since I’m not a representative of Lob, I couldn’t commit to maintaining this Zap, and Zapier doesn’t want to have any of their customers depending on an integration that could disappear or be unsupported at any time–a fair position.

If the Zapier or Lob folks want to take this integration and run with it, I’d be happy to share my code–just leave a comment. If anyone else is interested in being able to generate Lob postcards from a Google spreadsheet (or any other compatible API) via Zapier integration, let me know and I’ll see what I can do.

APIs BPA Business Javascript Useful Tools | moore |

Zapier is awesome!

APIs are sprouting up everywhere. This is great for developers (and for end users) because it allows all kinds of automation. However, there are times when the investment of writing code to connect two APIs is too much.

Why might writing code be too much?

  • The problem is still fluid and writing code will lock in a solution.
  • There are more pressing business problems to solve.
  • There are no engineering resources available, and/or no money to hire a dev.

If any of these reasons apply to a problem you are facing, consider Zapier (or a competitor like IFTTT). These services are much like Excel macros–they require less software engineering expertise but can leverage some of the power of programming to automate away work. I’m only going to write about Zapier, since that is the solution with which I am familiar.

Zapier runs the connections between each service (called ‘Zaps’) at regular intervals. Zaps are built using a web only interface that leverages the APIs in a manner that, while not completely intuitive, is thoroughly manageable by anyone who can sum up a column in Excel.

Here’s the class of problems for which Zapier is good:

  • Connecting two services for which Zapier has connectors–this list is quite extensive.
  • Syncing needs to happen no more than every 5 minutes (15 for the free account).
  • No processing of the data during the transfer is needed (except possibly omitting some fields)–you are simply moving data from one place to another–no Yahoo! Pipes like transformations are possible in transit.
  • One way sync is OK (though there are workarounds).
  • You don’t need to bulk load initial data via Zapier–you either can disregard initial data or load it in some other fashion.
  • You have a reasonable number of sources and sinks–each linked source and sink will take up one Zap.
  • You have to have valid accounts with each source and sink.

That’s a fair number of limitations, but even so there are a large number of common problems that are solvable by Zapier. Some examples:

  • Syncing a list of contacts from one source to another.
  • Taking a google form submission and adding a user to a mailing list.
  • Moving a row from one Google spreadsheet to another.
  • Taking an email and adding it to a database.
  • Adding a customer who you have just invoiced in QuickBooks to an email list.

These are all types of problems that can be done manually, but if frequency or scale increases, the process can run people ragged.

I want to call special attention to the email processing ability of Zapier. If you have a well-formatted email that you often receive that you want to further process, Zapier can parse it into interesting fields and send that data along to a sink like a Google Spreadsheet. Examples of well-formatted emails include order confirmations, contact us forms, and newsletter subscriptions.

I have found the Zapier support folks very responsive, whether that was troubleshooting an issue with Google docs, finding out how to pronounce the company name, or explaining why having 100 Zaps reading from one Google spreadsheet was a bad idea. Having responsive support staff reassures me, because once Zapier gets embedded into business processes, ripping it out is going to be very painful and a lot of work.

You can also write your own Zaps if you have custom APIs that you’d like to integrate with. I haven’t explored this much–it does seem like a developer centric task.

Zapier is not an all purpose tool nor a total replacement for developers, but it is definitely a great app to have in your toolbox. Take a look and see what little (or big) niggling problem that you haven’t had time to write code for (or, if you can’t write code, what you haven’t been able to get an engineer to write code for) it might solve.

APIs Useful Tools Web Applications | moore |

Should your customers be your devrels?

Saw this spicy take:

If Twitter goes away, this is from Stefan Avram, who said “You shouldn’t have devrels. Your customers should be your devrels”. (Stefan is a cofounder of Wundergraph and head of growth. Good on him! It’s hard to found a company.)

I wanted to talk about this statement because, well, I’m a devrel (a developer relations professional) and I don’t agree with it.

First, let’s rule out a few obvious points.

  • If you aren’t targeting developers as users, you shouldn’t have devrel as a function.
  • Even if you have a devtool, if your main sales channel is top-down, devrel is not super useful.
  • If your product doesn’t have some kind of free version, devrel is going to have a tough time. Not everything has to be free, but it’s really tough to get a developer to offer both time and money to learn about your product.

I hope this goes without say, but it is fantastic if your customers are advocates for your product.

However, I think this approach is a bit naive. It’s a bit like saying “You shouldn’t have sales people. Your customers should be selling your product.”

The truth is that there will be some subset of your users who are enthusiastic and advocate for your product if it is good enough to buy. I have interviewed some for FusionAuth. You should do everything you can to encourage this behavior, including:

  • talking to them
  • learning about their companies and successes
  • giving them swag
  • sharing their stories far and wide

But these folks’ job isn’t to promote or support other users’ use of your product. They’re busy building their product, company or project. As they should.

A high functioning devrel team can offer:

  • quality content
  • support for a community
  • up to date, relevant code

This team should be working to improve all of these and make the experience of all of your potential users better and better.

And they’ll care about the company’s overall mission. And they’ll have inside access to the codebase, the engineering team, and the product roadmap. And they can offer reified, constructive feedback from users to your teams.

That’s a large number of important business functions. If they are performed by customers, who are, again, busy working on their own stuff, they’ll be done at best haphazardly and at worst not at all.

Think about it. How many blog posts have you written about a product? How many of them were deep dives?

I can say that I’ve done it for a few products (stripe, zapier, AWS, lob) and none of those are deep enough to qualify as anything other than the briefest introduction to a product. A developer who encounters posts like these will either be intrigued and want to dig in further or bounce, but they certainly won’t have enough info to make a decision or use the product.

By making documentation, community and code examples the developer relations team’s problem, you’ll get more focus and better results.

And more users and happier customers.

Devrel FusionAuth | moore |

A quick look at xkit

I was prototyping a small app in xkit and wanted to document this useful tool. When I first saw this launch on HackerNews, I couldn’t quite understand what the purpose was. But now that I’ve spent a bit of time playing with it, I understand it a bit more.

Suppose you are writing a recipe management SaaS and realize that you want to integrate with some other services. Perhaps you want to be able to export the steps of a recipe to a Trello board, or to a Google doc, or to a PDF.

These are all services available on the internet with an API which will allow end users to give your application access to their accounts. This lets you publish to each user’s Google docs account or Trello board.

(I’m not as familiar with services offering PDF generation functionality, but a quick search turns up some options, including some that you can self host.)

There’s a fair bit of hoop jumping in terms of setting up API keys and OAuth consent screens, however.

And this is the problem that xkit solves. If they’ve already written the connection (here’s a list), it is quite simple to add the ability for a user to connect to the service. With no previous experience, I was able to connect to Trello in about an hour. The user experience of connecting the external SaaS application is really smooth and far better than something I could whip up quickly.

If they haven’t written a connector, I don’t believe you can write one yourself. For example, for that PDF service, you’d need to contact the xkit folks and ask them to add one.

This is different than, say, Zapier, because it’s operating at a different level. Zapier is excellent (and has been for years) at letting users connect their apps. But xkit lets you let your users connect apps, basically letting you build a mini Zapier (in terms of connectivity, not functionality).

You can also host your own app catalog if you want to. I didn’t get into this too much, though, so it’s unclear what the benefits of that are.

They provide a user data store out of the box, but also integrate with a number of other providers (including FusionAuth). This means you can leverage your existing auth solution and still get the easy integration with other third party APIs.

Their pricing seems reasonable, given what they take off your plate.

Nothing’s perfect, however. I found a few documentation bugs, which I let them know about (they host their docs on readme.com and I found the suggestion process delightful). When I tried to sign up, the service was down, but a quick Tweet exchange resolved the issue within 30 minutes.

It is bizarre to me as an authentication focused company that they don’t have a “forgot password” link on their login pages. The documentation is javascript heavy, with nary a mention of other languages, but that’s understandable as they’re just starting out. It’s also strangely video heavy, which I found a bit distracting; that, however, could just be my learning style.

All in all, if you are looking to integrate third party APIs which require OAuth interactions on the part of your users, you’d be well served to take a look at xkit.

APIs Useful Tools Web Applications | moore |

Joining Transposit

I am starting a new job today. I joined Transposit as a developer advocate.

I’m excited for two main reasons.

I think that the company is in the right place to solve a real customer pain point. In my mind, this stands at the intersection of Heroku and Zapier. I love both these companies and have used them, but sometimes you need something that is more customizable than a chain of Zaps (perhaps something that maintains state or interacts with an API action that Zapier doesn’t support) and yet you don’t want to be responsible for the full SDLC of an app running on Heroku, including all the pain of deploying and building authentication. Even with Rails, you still need to snap together a number of components to build a real application on Heroku. You might reach for AWS Lambda, especially if you are only working within the AWS universe, but what if you need access to other APIs? You can pull down an SDK, but you just put yourself back in the land of more complexity.

I’ve encountered this myself and understand how much software doesn’t get built for these reasons. (Or it gets built and does half the job it could, or it gets built and turns into a maintenance problem in a year or two.)

Transposit threads this needle by creating a low code solution. You have all the power of Javascript (with the perils as well). It handles some of the things that pretty much every application is going to have (authentication, scheduled jobs, per user settings) and hosts your application for you. The big win, however, is the API composition abstraction. Every API they integrate with (full list) is just a database table. The syntax can be a bit weird at times, but the abstraction works (I’ve created a few apps). Authentication with an API is managed by Transposit as well (though you have to set it up) and you have the option of having the authentication be per user or application wide.

I think that Transposit is going to make it much easier to build software that will help automate business and make people’s lives easier. That’s something I’ve been thinking about for a long time. It’s free to signup and kick the tires, so you can go build something, like a slackbot that fits into a tweet.

The second reason I’m excited to join Transposit is because I’ll be shifting roles. After a couple of decades as a developer, CTO, engineering manager, tech lead and technology instructor (not all at the same time!) I’ll be trying out the developer advocate role. I’ll be doing a lot more writing and interaction with Transposit’s primary users, developers, to help make the platform into the best solution it can be.

PS, we’re hiring.

APIs Jobs | moore | | 1 Comment

Easily extracting conversations from a slack group

Man slack liningSlack is an amazing productivity tool when used correctly. One of the primary uses I’ve seen is for open source projects to provide support (Craft CMS, OG-AWS) or for communities to be built (Techfriends, Denver Devs). If you don’t have the luxury of the owner of your slack being Slack’s VP of engineering, the costs of $x/month/user can cause these types of slacks to remain on the free plan.

Which means that you are limited to the last 10k of messages.

And that’s fine for the vast majority of messages. Sometimes, however a discussion is so good that it deserves to be indexed and shared, which means it needs to be pulled out of the Slack walled garden and onto the web (I also wrote about how to do this with the Facebook Group walled garden last year). Sometimes you might just want to save it beyond the 10k message limit for your own selfish reasons.

You can of course do this extraction manually (I did so here and here). But that’s a lot of work.

Another option is to use Zapier. The slack integration is trivial to set up, and has a number of options. From there you can push to a google spreadsheet (if you want to do further reification) or directly to WordPress (or any of the other integrations).

The nice part about this is that the Zapier slack integration is that you have a variety of options that can trigger the publishing of a message to a spreadsheet:

  • a post of a public message in a specific channel
  • a post of a public message in any channel
  • starring of a message by you
  • attachment of a certain reaction emoji (I picked a floppy disk) to a message, no matter who adds the emoji

I’ve just started doing this but am excited to have a low friction way to pull high value conversations out of slack. Slack is great for synchronous communication and easy discussion. When real knowledge drops, it should be shared with the future and anyone who can type into a search box. Do make sure to let folks know because there may be some expectation of privacy that you’ll want to respect.

Tips Useful Tools | moore | | Leave a comment

Things I wish I knew about Stripe

Caterpillar
Striped, but not charging your credit card

So, at The Food Corridor, we’ve been using Stripe happily since we launched in June of 2016.  As a developer, I’d used Stripe before in a couple of different ways, but this has definitely been my most sustained use of the payment service.  (If you don’t know what Stripe is, it is an API that makes charging customers as easy as an API call.  More here.)

I wanted to outline some of the things I’ve learned from months of using Stripe.

  • Stripe supports pulling money directly from bank accounts, via ACH, but it really isn’t the same ACH as your bank lets you do.  This is because Stripe isn’t a bank.  The biggest thing to be aware of here is that Stripe ACH takes 7 days to arrive in your bank account.  Another issue is that you have to do verification.  They have two ways of doing verification–micro deposits and Plaid.  Plaid is instant, but only supports major banks, which was a non starter for us (updated 9/8: Plaid supports around 1000 banks now).  The code for micro deposits is straightforward, but be prepared for some customer support issues.  Stripe deposits two amounts and withdraws just one amount, which was confusing for some of our users.  It also takes a couple of days, so if your users are hot to spend money, Stripe ACH may not be a fit.  The win?  Definitely cheaper.  (And I didn’t find any other service that would support both credit card and ACH transactions that was developer friendly.)
  • Don’t forget to set up your webhooks out of the gate.  Stripe mentions this, but I glossed over it in the early days, and missed some events that were important.  (The most relevant is that ACH is asynchronous, so when an ACH transfer fails, it is reported via webhook.  If bank account verification doesn’t work, you’ll get a different kind of webhook.  Review the docs and set up webhooks for all the ACH events.)  If you don’t have time for a full featured webhook processing implementation, Zapier can just send the webhook data to your email. This can be a great stopgap solution.  Or you can use stripe_event.
  • Per support, if a webhook post fails (because your app is down, for example), they are retried once an hour for 72 hours.
  • Speaking of stopgap solutions, the Stripe Dashboard is fantastic for manual processes.  Just because you can automate everything via an API, doesn’t mean you should.  There can be some complicated edge cases with payment processing, especially around refunds, but they can easily be handled with a google doc of instructions and the Stripe Dashboard.  I have found only one use case that the API can handle that the dashboard cannot (a partial refund of an ACH transaction).
  • I have found Stripe support to be excellent, quick and knowledgeable.
  • Occasionally customer charges will be declined because of bank fraud triggers.  Expect to occasionally ask your customers to call their bank.  (I think this has happend about once every third month).
  • Disputes are a total pain, because the process is opaque and slow (expect a resolution in about two months and know you are not in possession of the payment during that time).
  • Make sure to capture the payment id anytime you charge a card or run ACH.  It will make future automation a lot easier.
  • Monthly plans are complicated, so if you can lean on Stripe for management, even if you are doing manual plan management (applying coupons, adding, or removing users from plans via the dashboard), do that.
  • The first payment you charge takes 7 days to move from stripe to your​ bank account.  This is for fraud protection.  Payments thereafter typically take 2 days (but it depends on your country and industry).

And here are some special tips if you are using Stripe Connect (their marketplace product).

  • Read the docs!
  • Remember that first payment timeline?  It applies to every one of the connected accounts.  Think about charging your own credit card as soon as you connect an account to help with customer cash flow.
  • Consider whether you want to use managed vs standalone accounts.  Managed accounts are a lot more work but allow you to have a seamless UX that you control.  Standalone accounts, which we use, are far quicker to setup.  I think this depends on the number of sellers you have in your marketplace.
  • You also want to think about whether to place the charges on the platform account or on the connected accounts.  A major factor there is who bears the Stripe fees, the platform or the sellers.  We charged on the platform account because we wanted all our data in one place.  If you are selling plans, you can’t charge on the platform and use Stripe plans.
  • If you are charging on the platform account, and are using standalone accounts (where the sellers have to set up a stripe account) your sellers won’t see charge descriptions unless you manually copy the description over.  The code looks like:

# this will let the sellers know what invoice the charge was for
transfer_id = charge.transfer
transfer = Stripe::Transfer.retrieve(transfer_id, expand: ['destination_
payment'])
payment_id = transfer.destination_payment
payment = Stripe::Charge.retrieve(payment_id, {stripe_account: destinati
on_account_id})
payment.description = description
payment.save

Happy charging!

APIs BPA Stripe The Food Corridor Tips Useful Tools | moore |

Getting the good content out of a Facebook group

I am astonished at how hard it is to get information out of Facebook groups.

The startup of which I am a part has created a Facebook group for dissemination of information of between commercial kitchen operators.  This was easier to get started than a forum and has the advantage of everyone being a “real person”, or at least real enough to get a Facebook account.  It also benefits from the ubiquity of Facebook–many many folks have it on their phones and get notifications about group activity.

However, it has the detriment of being a “walled garden”, with the content of the group being unavailable for searchers on the web.  Some might argue that privacy actually is a good thing, because it will encourage folks to be more honest, but really, anything you put on Facebook can be cut and pasted and made public, so I’m not sure I buy that argument.

Regardless, I wanted to find an easy, automated way to take the Facebook group content and pipe it elsewhere, where it could be reified and curated.  A human could do that, but I’d like an automated solution. And, other than the Facebook API, I haven’t found many.  Zapier (my go to integration choice) only recently released this as an option (in the last few months).  IFTTT doesn’t have it.  There’s no commercial solution that I could find that does this.  There are, however, some open source solutions.

The Facebook API makes it fairly easy to grab the posts of a group, and from the posts, the comments, but frankly, I really want a solution that doesn’t require coding up the JSON parsing/pagination handling/Oauth access.  I just tried the facebook-export tool and it seems to work just fine (though I had ended up having to update the leveldown/levelup versions to 1.5/1.3 to get past a compile error: leveldown.target.mk:114: recipe for target 'Release/obj.target/leveldown/src/batch.o' failed). It gives you all your posts as JSON.

Javascript Social Media | moore | | Leave a comment