Just listened to another mysql webinar. This time it was about mysql and memcached. I got in a little late, but here are my notes.
There is a project that hooks mysql UDFs to memecached.
The main speaker worked at Grazr, a feed manipulation company. He presented some lessons and stated that some seemed obvious, but we all know how it is, you learn mistakes the hard way.
- Beware architecture momentum–early decisions influence later ones.
- Your scaling plan is wrong in some way!–grazr overbuilt.
- Testing is limited: they had issues with replication speed. Users are better at testing site than any perl script.
- Replication is fast, until it isn’t. It’s asynchronous nature can make it hard to program to.
- Memcached is your friend
- Sphinx, an open source SQL full text search engine (like Lucene), is your friend. It also has a mysql storage engine–virtual table that you can join with other data sources. This sounds cool.
- Do bulk insert and lazy writes–if you don’t need to immediately read it, do the write later.
- Emphasis on scaling hurt user experience. Fast, data correct transaction are available with two options: one server, or a sync buffer like memcached.
- Use instrumentation–he mentioned tools like nagios and cacti (cacti has mysql plugin [which I couldn’t find]) to help scaling–close the loop: when you make a change, find out what that change did.
- Try new things–for example, the MySQL blackhole storage engine for data that is only read on slaves, not on the master.
- Everyone has same problems–avoid Not Invented Here syndrome. Not just the obvious stuff (who writes a web server these days–oh wait), for example, there’s a batch job processor like gearman. Note that he also said they had to Invent a few things before they learned this lesson.
- Accept change–design for now+1, not now+100
- Listen–even folks not intimately familiar with your technology may have good ideas, and more brains are better.
Looks like a couple of interesting whitepapers are available (you’ll have to register and get pinged via email a few times by sales folks): “Designing and Implementing Scalable Applications with Memcached and MySQL”. And there are future webinars on the MySQL/memcached topic, one on SAAS applications and one on Scalable web architecture.
In a meta note, I enjoy the webinars because:
- Easy to get to
- High quality
- One can bail if you want (which I did a few weeks ago when MySQL proxy http://forge.mysql.com/wiki/MySQL_Proxy didn’t turn out to be all that interesting to me).
- The give me something to blog about 🙂