Skip to content

Using WordPress as a CRUD Database, API Included

Ethernet cordBased on this HN discussion, which I discussed a while back, I looked at how to set up WP as a CRUD database accessible via API.

It wasn’t hard. Steps:

  1. Install WordPress (I used ec2 and the Cloudformation sample template)
  2. Install the following plugins
  3. I also installed the following optional plugins
  4. I created a custom post type of ‘todo’ and added a couple of custom fields.
  5. I was able to get the todos by going to these URLs (apparently you can have the API live at wp-json, but that required some rejiggering of url permalinks).
    • http://host/wordpress/?rest_route=/wp/v2/todo/8
    • http://host/wordpress/?rest_route=/wp/v2/todos

Here’s an example of the output:

{
  "id": 8,
  "date": "2018-03-05T02:38:26",
  "date_gmt": "2018-03-05T02:38:26",
  "guid": {
    "rendered": "http://host/wordpress/?post_type=todo&p=8"
  },
  "modified": "2018-03-05T02:40:01",
  "modified_gmt": "2018-03-05T02:40:01",
  "slug": "auto-draft",
  "status": "publish",
  "type": "todo",
  "link": "http://host/wordpress/todo/auto-draft/",
  "title": {
    "rendered": "Buy Milk"
  },
  "template": "",
  "acf": {
    "": false,
    "due_date": "20180308",
    "description": "please buy milk.",
    "who_owns_it": {
      "ID": "1",
      "user_firstname": "",
      "user_lastname": "",
      "nickname": "mooreds",
      "user_nicename": "mooreds",
      "display_name": "mooreds",
      "user_email": "...",
      "user_url": "",
      "user_registered": "2018-03-05 02:21:36",
      "user_description": "",
      "user_avatar": "..."
    },
    "done": false
  },
  "_links": {
    "self": [
      {
        "href": "http://host/wordpress/wp-json/wp/v2/todo/8"
      }
    ],
    "collection": [
      {
        "href": "http://host/wordpress/wp-json/wp/v2/todo"
      }
    ],
    "about": [
      {
        "href": "http://host/wordpress/wp-json/wp/v2/types/todo"
      }
    ],
    "wp:attachment": [
      {
        "href": "http://host/wordpress/wp-json/wp/v2/media?parent=8"
      }
    ],
    "curies": [
      {
        "name": "wp",
        "href": "https://api.w.org/{rel}",
        "templated": true
      }
    ]
  }
}

The custom post fields are all under the ACF key, and you can see that there was an expansion of the who_owns_it field. If you are going to do this, make sure have the the normal title tag be part of the custom post, otherwise the WP UX for editing the custom posts won’t be much use.

Not perfectly restful, but a super simple way to set up an API that non technical folks can use to create, update or delete records and that you can consume in other systems.

APIs Useful Tools Web Applications | moore |

“Choose boring technology”

Reading a boring bookThis article on choosing boring technology is so good. It’s from 2015 so some of the tech it references is more mature, but the thesis is that the technology underlying a business should be well known and boring to implement. It also discusses how important cohesive technology can be (I love the final footnote).

I think this is an interesting contrast to microservices, which is essentially solving repeatedly for local optimizations and then relying on the API or message boundary to allow scaling. Microservices, in my experience, often get a bit hand wavey when it comes to operationalization (“just put it on Kubernetes, it’ll be fine”).

I have definitely dealt with this in previous companies where I had to put the kibosh on new technologies being introduced (“nope, we’re going to do it in java”). This meant that as a small team we had a standard deployment stack and process and could leverage our logging knowledge and debugging tools.

Here are some arguments I hear for using the latest and greatest technologies:

  • “Our developers will get bored and we’ll have a hard time recruiting”
    • This is a great reason to have regular hackfests and or support developers working on open source or side projects.
  • “New tech will help us do things faster”
    • It’s definitely worth evaluating new technology periodically and adding it to your stack (especially if it is outsourced, a la RDS, or boring, a la memcached). As the original post mentions, if you get substantial benefits from the new technology, then use it full bore and plan for a migration. Don’t end up in a state where you are half in/half out. Also consider tooling and processes to get things done faster–these may have quicker iteration times with lower operational risk.
  • “Choose the right tool for the job”
    • Most turing complete languages can be used for any purpose. And you shouldn’t be optimizing for the right tool for the job, but rather the right tool for the business for the job.

Really, you should just go read the post. It’s good.

 

Business Programming Technology | moore | | Leave a comment

Easily visualizing location data with Google Fusion Tables

Hands with map on itSometimes you have a list of locations in a Google spreadsheet and want to visualize where the locations using a map. Google Fusion tables lets you do just that, for free, with no technical expertise needed.

To do so, you need to create a spreadsheet. I created a spreadsheet of birthplaces of the Presidents of the USA. Here’s the spreadsheet. You want to make sure you have column headers and that your location information is all in one column. You can see that I concatenated birth city and state into one column, because you can only map one column (unless you have lat/lng, in which case you can use those two columns). You also can’t concatenate any columns when the data has been pulled into fusion tables.

Then, create a new google fusion table (under the ‘more’ menu). Choose a spreadsheet as your data source and then past in the spreadsheet link.

Your location column may or may not have been given a data type of location. If not, use the ‘edit’ menu then ‘change columns’ to convert it to a location type.

Add a map using the red plus sign and select your column as the location. Wait for your column to be geocoded (if you have lat/lng you shouldn’t need to do this).

And there you are. Here’s the map I generated. You can see that if you click on the marker you will get information about each president, which is a nice bonus feature. You can also share this within your organization or with the wider world and do additional filtering.

Fusion tables is great when you have the structured data and just need a simple map representation.

Caveats:

  • Once your data is in fusion tables, you are extremely limited on what you can do with it (see the concatenation above, for example). Do whatever data massaging you need in the spreadsheet. This also means that you probably want the spreadsheet to be your source of record.
  • There’s no way to update your data. So when the next president enters office, I will either have to create a whole new fusion table or delete all the rows and re-import.
  • Fusion tables seems to no longer be under active development. At least I haven’t seen many feature changes over the past couple of years. It is out of beta. I think it’s fine to build adhoc tooling on top of this service, but if I were looking for the core of my business I’d avoid this.
Google Apps Useful Tools | moore | | Leave a comment

Passwordsafe, redux

Guy looking menacingly at a computerI have written before about passwordsafe (about a decade ago!). I just wanted to reiterate how having all your authentication information in one or more password safe files is such a lifesaver for transitions. You can easily change all the passwords at one go (using randomly generated strong passwords, of course), though you may have to deploy applications or change environment variables to do this. You also can easily see all the systems and external services that help make your application “go”.

I’ve heard about good things about tools like Lastpass and whatnot, but passwordsafe is open source, can live on a thumb drive, is multi platform and has a small feature set. It’s also extremely easy to set up and get going, especially if you store it in version control. And for a small team moving fast, the perfect solution is the enemy of the good.

Do something–even a shared google doc will get you some of the benefits.

Security Useful Tools | moore | | Leave a comment

HN Job Posting Thread

Person refusing bag of moneyIf you aren’t an avid follower of Hacker News, you might not be aware that once a month there’s a job posting thread on the site. This community has its rough edges, like any other, but this thread offers a great place to reach the audience–software developers, startup aficionados and technologists of all stripes. Here’s the April 2018 job thread.

I find this a bit more credible than the job sites because there’s community policing and because it expires there’s less incentive for recruiters or other bad actors gaming the system. Because it is free and specifies only employees of a company may post, I think that the job openings are more genuine. You often see hiring managers post their contact info, and an email to them is far more likely to get a response than submitting to an applicant tracking system. You can often find salary ranges, and the audience and posts are global. It’s also worth noting that this feeds into some other sites like whoishiring.io.

I have posted on behalf of a few companies for which I worked and have had a decent hit rate. Haven’t seen any real success numbers though.

Whether you are on the market, hiring, or just curious, more data is usually useful, which is why this is always a thread I peruse. If you want to look at previous months postings, you can see them in this bot’s submissions.

Jobs Recruiting | moore |

“Finding your purpose and living a meaningful life”

BatsA friend recently shared a letter from Hunter S. Thompson on finding your purpose. I found it to be quite insightful, especially on how he emphasizes a person should focus on “choos[ing] a path which will let his ABILITIES function at maximum efficiency toward the gratification of his DESIRES.” Choosing a goal is less important than choosing a way of life. I remember when I was chatting with my father about how he chose his career, he stated that in a world of choice you should choose some invariants and evaluate options based on those invariants. They don’t have to be job invariants, though sometimes they are. But if you don’t have a set of fixed standards, you run the risk of chasing after shiny objects again and again, or, worse, contorting yourself to some goal that previous you had settled upon. After some thought during this transition, these are mine:

  • Technology is the most fun and best when it helps people. I remember the joy of this in my first professional software project (mail merging insurance renewal letters). Watching people helped by software I built has been the highlights of my career.
  • No one every wishes they’d worked more on their deathbed. Work to live, don’t live to work.

It’s scary for me to lay out constraints because that means that you are ruling out possibilities. For instance, if a high flying Silicon Valley startup ever read this, they’d probably pass on my application due to my desire to not work 80+ hour weeks for the chance at winning the lottery. But that’s precisely the point. If the opportunity doesn’t meet my invariants, no matter how nice the paycheck or good the perks, it’s not a place that I’ll thrive.

PS Hunter S. Thompson also warns that every person’s advice is drawn from their experience and should be treated as subjecting–as Miles Law states: “where you stand depends a whole lot on where you sit“. Or to use Internet-ese: YMMV.

Lifehack | moore | | Leave a comment

Leaving well

Bird leaving eggLeaving a company in a way that is fair to both you and your company can be difficult. When employed, we spend a large portion of our waking hours at work. You may be leaving a group of people you loved, a toxic environment, a place you’ve outgrown, or a place you’ve loved and just need to move on from for personal reasons. Because of the amount of time invested and the multiplicity of emotional circumstances, it can be difficult to leave well. Below are some thoughts on this career transition, however, I’m not writing about why you should leave, just how the process should go once you’ve made that decision. (Note that some of these apply to transitioning positions within a company.)

Before you are thinking about leaving

  1. Prepare to leave well before you think about leaving by documenting your decisions, processes and systems. This has the added benefit of letting you do better in your current position. When you write down how you do a task, it gives you the chance to review it and consider optimizations, as well as revisit it in the future and perform the task just as well. Make sure to date all documents. When you revisit a system or process, revisit the document.
  2. Watch how other departing employees are treated. Expect to be treated in a similar manner. Some companies want to usher folks out quickly (to the point of just paying their standard two weeks notice immediately and having them depart) while others will be more flexible. Some managers will treat departing employees with compassion and respect. Others may not.
  3. You won’t be able to effect change at the company once you have publicly decided to depart. If you want to effect change, stay at the company work within the system.

Once you’ve decided to leave

  1. Save and put that money into a liquid savings account. How much? As much as you can. This will make the transition less scary and allow you greater flexibility.
  2. Decide on boundaries and stick to them. Being helpful with the transition doesn’t mean you have to be a doormat.
  3. It’s always easier to find a job when you have a job. Think about reactivating old networks, inviting folks for coffee, and checking out the job market while you are still in your position.
  4. When you decide to leave, give as much notice as possible. Since you’ve been observing how folks are treated and you know your own situation, adjust for those factors. However, I’ve found letting managers know about my departure with plenty of notice ensures a smooth departure. Personally, I’ve given up to two months of notice.
  5. I’ve never had a counteroffer, but I’ve read that accepting them is a poor choice.
  6. Make a plan with your manager. Take point on this, as you are the person who knows your job best. This plan should be your first task after you’ve told your manager you are departing.
  7. Keep a spreadsheet of departure tasks including owner, date to be completed and description. Sometimes important things are overlooked. This is where having documentation (see step 1) is helpful, but also look at your to-do lists, your and calendar entries.

Telling your fellow employees

  1. Let the company control the narrative about when you are leaving, including when to tell the team. However, if you are approaching your departure date and no one on the team knows, push your manager to publicize it.
  2. You will likely have many reasons for your departure. Pick a major, true, banal reason or two and answer with that when team members ask why you are leaving. There’s no need to get into every grievance, reason or issue you had.
  3. Your decisions will have less weight once you announce your departure. This is natural; team members that are staying discount your opinions because you won’t be living with the consequences. Prepare yourself for this.
  4. Consider offering to consulting to the company if it makes sense for you and the timing is right. Charge a fair market rate. Realize that stepping into this role may be difficult emotionally.
  5. Once your exit is public, your focus should be bringing other employees up to speed so they can do your job when you’re gone. It may feel good to bang out one more bugfix or initiative and if you have time to do that, great, but your primary focus should be on documentation and knowledge transfer.
  6. Realize that this transition will feel momentous to you, but that it is far less important to everybody else (both inside and outside the company). A company should have no irreplaceable employees.
  7. Treat everyone as fairly as possible. Remember that you may be working with some of these folks in a few years’ or decades’ time.
  8. Be professional and courteous (I can’t think of a time when this is bad advice, but at moments of transition it is especially important).

Leaving a job is a very personal decision and will impact your career. Spend time thinking about how to leave well, treat everyone with respect and have a plan.

Jobs Lifehack | moore | | Leave a comment

Let AWS RDS handle database scutwork

Amazon DatabasesRDS is a service I’ve mentioned in the past, but it’s fantastic. You can outsource large chunks of database administration to AWS. Tasks you can forget about include backups, failover, read only replicas, and OS and DB upgrades.

This is a great fit for spinning up databases for small scale to large scale systems and prototyping.

Things to keep in mind if you start using RDS:

  • The database is launched into a VPC and will have a security group around it. You’ll need to allow IP addresses or security groups access to the port the database is living on or your connections will time out.
  • The database RDS creates is a normal database that you can manage like you can any other database you have set up and installed, but there are certain limitations (for example, no MySQL UDFs). Read the documentation and understand the limitations, but be aware they are constantly changing. I suggest subscribing to the AWS Database blog RDS category for updates.
  • RDS uses EBS under the covers and has the performance constraints of that technology. For the largest scale production systems you’ll want to test before jumping in whole hog.
  • If you are using MySQL or PostgreSQL and are running into concurrency problems, Aurora may be worth evaluating.
  • If you want to have backups past thirty five days for peace of mind of compliance concerns, you’ll need manual snapshots.
  • RDS only supports certain RDBMS and limits databases to certain sizes. If you want to run anything else on AWS, you will need to self manage your DB on EC2 or look at other data management solutions. Here are some other gotchas.
  • When using RDS you aren’t freed from all database administration tasks. There are still users to manage, indices to add, and queries to tune. Most of your RDBMS skillset is applicable to RDS, however. You’ll also need to determine when to schedule DB and OS upgrades, backups and how to size your instances. You still need to set up the optimal architecture of an RDS system including standbys and read only replicas and do other configuration both at the network and database level.
  • You can manage RDS system attributes via cloudformation, terraform and the CLI in the same way you can manage other AWS infrastructure. That said, the RDS system is stateful so you can’t treat it entirely as “cattle”.

You can learn more about RDS in the extensive documentation.

Amazon RDS AWS Databases | moore | | Leave a comment

Swagger looks pretty good from here

SwaggerA few years ago I was working on an API that my client was going to make available to some of their clients. I used Swagger, which I’d heard about from a Gluecon presentation.

I was unimpressed. I recall having difficulty getting the online tool to work and the documentation generated was poor. This could very well have been user error or my misunderstanding of the sweet spot of the tool, but for whatever reason it wasn’t a fit for the problem.

Fast forward a few years and I was talking to a company about a position. They were planning to use Swagger to generate their API SDKs. They had an API which was crucial to their business and were currently supporting a large number of SDKs on a bespoke basis. I reviewed the documentation of Swagger and downloaded the 2.3 version. I was very quickly able to generate a number of client and server stubs using the codegen project. They have a long list of supported languages, but I quickly generated ruby, csharp and perl client bindings and a simple rails5 and spring server. I didn’t push these through to production and I’m sure that if I had I’d have learned about the rough edges (it’s always nice to check out Stack Overflow for rough edges for any technology–here’s questions for swagger). Regardless, the ability to take a simple JSON configuration file and create API front ends and backends in minutes was quite impressive.

Note that the codegen tools are all in java, so if leveraging that technology gives you the willies, you might want to look around for another solution. Note also that Swagger has now been moved to the OpenAPI project (as of version 2.0 of Swagger). If you want to know more about that, here’s the blog post announcing that move. Here’s a look at features of the next version.

If you are developing an API first company (and there are good reasons to be that), I’d recommend taking a long hard look at Swagger. The speed of adding SDK support as well as the community around this tooling look to be huge advantages.

APIs | moore | | Leave a comment

Blockchain now in the trough of disillusionment?

ChainsLooks like the blockchain may be headed into the trough of disillusionment. See also Kevin Owocki’s thoughts on ICOs.

This happens to every technology. There is wild optimism over the usefulness (remember Iridium?), which is overblown. It even happened in the 1800s with trains. That said, I’m sure there are uses for blockchain technology beyond stores of value, and am looking forward to seeing those emerge. For a fascinating read on the rollout of technology, see The Deployment Age.

Technology and Society | moore | | Leave a comment