MySQL and memcached: notes from a webinar

Just listened to another mysql webinar. This time it was about mysql and memcached. I got in a little late, but here are my notes.

Memcached is a distributed caching mechanism. Not as robust as Coherence, for example, but free, open source, and widely used and supported.

There is a project that hooks mysql UDFs to memecached.

The main speaker worked at Grazr, a feed manipulation company. He presented some lessons and stated that some seemed obvious, but we all know how it is, you learn mistakes the hard way.

1. Beware architecture momentum–early decisions influence later ones.
2. Your scaling plan is wrong in some way!–grazr overbuilt.
3. Testing is limited: they had issues with replication speed. Users are better at testing site than any perl script.
4. Replication is fast, until it isn’t. It’s asynchronous nature can make it hard to program to.
5. Memcached is your friend
6. Sphinx, an open source SQL full text search engine (like Lucene), is your friend. It also has a mysql storage engine–virtual table that you can join with other data sources. This sounds cool.
7. Do bulk insert and lazy writes–if you don’t need to immediately read it, do the write later.
8. Emphasis on scaling hurt user experience. Fast, data correct transaction are available with two options: one server, or a sync buffer like memcached.
9. Use instrumentation–he mentioned tools like nagios and cacti (cacti has mysql plugin [which I couldn’t find]) to help scaling–close the loop: when you make a change, find out what that change did.
10. Try new things–for example, the MySQL blackhole storage engine for data that is only read on slaves, not on the master.
11. Everyone has same problems–avoid Not Invented Here syndrome. Not just the obvious stuff (who writes a web server these days–oh wait), for example, there’s a batch job processor like gearman. Note that he also said they had to Invent a few things before they learned this lesson.
12. Accept change–design for now+1, not now+100
13. Listen–even folks not intimately familiar with your technology may have good ideas, and more brains are better.

End notes

Looks like a couple of interesting whitepapers are available (you’ll have to register and get pinged via email a few times by sales folks): “Designing and Implementing Scalable Applications with Memcached and MySQL”. And there are future webinars on the MySQL/memcached topic, one on SAAS applications and one on Scalable web architecture.

In a meta note, I enjoy the webinars because:

1. Easy to get to
2. Free
3. High quality
4. One can bail if you want (which I did a few weeks ago when MySQL proxy http://forge.mysql.com/wiki/MySQL_Proxy didn’t turn out to be all that interesting to me).
5. The give me something to blog about 🙂

[tags]webinar, memcached, users are the best testers[/tags]