lanehalley

When a name like Amazon gets into the business, everyone acts like everything is new and shiny again :)

Why, I can remember like it was, well... almost a decade ago when I first saw a hosting vendor get into the business of support MySQL databases.

The first vendor I remember? Rackspace. Why? Because way back when, they were the main sponsor for the second OpenSource Database Conference.

Looking through the options that Amazon provides, they do look pretty sharp. Database backups can be a pain, and providing snapshot based ones is an excellent idea. There was some talk of snapshotting when I was at the MySQL User Group this week talking about Drizze. The general consensus was that this was the superior way to do backups.

The other backup methods, like using a physical backup tool, certainly work (Innodb has sold one for years, and Percona has one that they provide). If logical backups are your thing you can always use mysqldump with Innodb to do an online backup (If you look through past blogs you can see how I use it and the distributed revision control system Mercurial to backup my databases and provide point in time recovery/selective restores).

Backup services to S3 are not new at all. Zmanda started to provide a product for backing it up years ago. Googling on "S3 mysql backup", I found several links for HOWTO to do it.

Looking at the API, I believe native driver support is available. I would be curious to find out if they are supporting just the ASCII protocol, or if they have enabled the binary one as well (which is an awesome way of crashing your database). I am going to assume these databases are all in a self contained instance. If not? Should be simple enough to crash you and your neighbor. It would be interesting to find out how many changes they had to make to the database (or still have to make... all of this reminds me that they were heavily recruiting at the MySQL User's Conference last year). It was pretty obvious back then that they were going to work on this service. There is no mention of SSL support. The SSL stuff built into MySQL would be a pain to make work, but I suspect they could with some hacking.

I wonder if libdrizzle's mysql bits will work? I suspect we will have to try that out. If so, anyone need sharding in their driver?

BTW if you follow their "use mysqldump" model for pushing data into the service, remember that --single-transaction will allow you to do a hot backup. There is no need to lock up your current database. I am still appalled at how few people know that. Years ago we should have renamed "mysqldump" to "mysqlbackup" and defaulted the settings for Innodb. Outside of licensing, I suspect "how to backup" my database was the most often requested question.

Oh, that and "are you worried now that Oracle has acquired Innodb?"

Renaming our dump tool to backup would have made for pretty slick marketing :)

Amazon's sizing numbers look good. They have hit the sweet spot for most users.

Monitoring is nice, but also, not that hard. If you have everything already in AWS I can see where the "one stop shopping" would be nice (or is that one-click?).

There is more you can find in an article by Jeff. Rightscale has an article up as well.

I'm really curious to find out what they have disabled in 5.1, I have a hard time believing they can support all of its features. Coming up with an upgrade path should be interesting to watch. I've seen no sign of any patches coming back from them, so I would assume they have rolled their own. This means either they will be on 5.1 for a long time, or will be spending a lot of engineering time maintaining their own version. (Most of the bigger MySQL shops do, so it is not all that surprising.) The number of folks on the planet that can really maintain custom versions well is pretty small. I don't believe Amazon has any of them on staff. I am sure though that they can find people who can figure it out.

I have seen mixed results when companies keep their own custom versions of MySQL. It can be pretty easy to code yourself into a corner when you don't play an active part in the community. 5.1 should provide many years worth of use for them though.

Kudos to Amazon. Having them run a database service won't provide the "high end" sort of usage that keeps the folks who tune databases in business, but there certainly are a lot of users for whom this type of service will work just fine.

At the very least this service will certainly up the ante for others in the MySQL hosting business.

Providing services like RDS and application is the future for cloud companies. It will be interesting to see what they will come up with next.

I'm surprised Memcached hasn't been done as a service yet, but perhaps that is why I have those SASL patches sitting in my inbox.

We still have a little while before Gearman will be announced :)

I have been asked a number of times "do you think there is a need for a column oriented database in the open source world?"

The answer has been yes!

Users and vendors have asked me this question a number of times. The problem has been most of the vendors were interested in creating closed source solutions around either Drizzle/MySQL, or, did their efforts in a way that made serious modifications to the backend (aka... made poor use of the storage engine interface).

For these reasons I have not really found myself all that thrilled to work with what has been out there. Also, I would often find that the commitment to open source was either luke warm or "we will do it, once we have some traction...".

My response to that? "Tell me more when you open source it. I'll see if it will work."

For this reason I was very happy to see Calpont do their release of Infinidb last week.

So as of this weekend?

We have a project to use their engine with Drizzle. Infinidb makes use of the storage engine interface I worked on for MySQL which is a subset of the interface we have built for Drizzle. We have had several engines ported already, but this will be the first column oriented engine we will have ported to Drizzle.

Building in different engines beyond the basic transactional engines is fun, because we get to see how the design stretches to fit additional needs. The core of Drizzle stays the same, but the micro-kernel nature of our design allows for others to expand the reach of where Drizzle can be used. Padraig started working on the engine on Friday and had it loading by the end of the weekend.

It should be fun to see what additional enhancements we can do out of the box with Infini engine :)

October 30, 2009 I attended Agilepalooza in Natick MA. The event was hosted by VersionOne. This was an openspace event, so the participant created and hosted the agenda. I wanted to talk about UX stuff so I proposed a talk "Visual Artifacts for Agile Teams." Here are my notes from the talk.

To start the talk, we did a silent brainstorm and collected topics for our backlog. I wrote out two posters that said "What is Working?" and "What are our Challenges?" to collect our comments. After that, the talk was fairly free form. As we ran down on a topic, we went to the backlog to see what else we wanted to talk about.

What are our Challenges?

The group identified a lot of challenges related to keeping everyone informed about the evolving product UI

1. Changes to mockup not communicated to QA, who test the wrong behavior
2. Marketing says "change this" to one developer, how does rest of team know?
3. Team decides to change product, how do you estimate impact of UI changes?
4. Going to code too soon. Failing to visualze and validate the user experience. 

What's Working?

Visual artifacts help keep the team focused on a shared product concept.

two kinds of visual artifacts
Product mandate: mission, scope, dependencies, agreements
Product look and behavior: sketches, storyboards, scenarios, user models

Integrate UX person into scrum team, attend standups, work with developers (50%-%100 works)
Get lightweight. Draw over printouts of existing visual design, rather than modify the source
When showing design concepts to people, don't ask "do you like it?" ask them to perform tasks so you understand what they can do and can't do.
Different artifacts for envision and construction phases of projects.

High res, Low res

We also got into a good discussion about the uses of low res and high res sketches and prototypes

Use of low res or high res depends on what you're trying to do. You can use high res during envisioning to help people imagine what it could be like, you can also use low res during envisioning to encourage people to participate in co-design.

Low res
More sketchy, observer can project/imagine
Easier to show to subject mater experts who can imagine what could be happening
Harder to show to managers and end users
Can be technology agnostic.

Hi res
Can be specific about look, information and behavior
More real, observer is responding
Easier to show to managers and end users who aren't good at imagining interactivity
If using for construction, you need to understand/take into account the technology platform so as to not waste effort creating things you can't build

Team Activities

At the end of our talk, we discussed some techniques you can use with your team to co-create visual artifacts and do visual problem solving

Design studio

Product box

We also talked about how scenarios can relate to user stories. In the early phases, a scenario can be a single story "buy house" later on, the scenario grows more specific and contains many user stories that define more specific activities.

It's good to have larger scenarios in your backlog before you break them down as they act as a placeholder you can keep in the back of your mind while you work on other things.

There are some photos of the event on flickr too.

Last night while working with someone I am tutoring we got into a conversation of "what is faster". Below is some assembler that was created from it, for two different programs which did exactly the same thing, but in different ways.

The question is, which is faster?
Second Question, which do you think was the more maintainable code?

	.file	"?"
	.text
.globl main
	.type	main, @function
main:
.LFB0:
	.cfi_startproc
	pushq	%rbp
	.cfi_def_cfa_offset 16
	movq	%rsp, %rbp
	.cfi_offset 6, -16
	.cfi_def_cfa_register 6
	subq	$2097184, %rsp
	leaq	-2097184(%rbp), %rax
	movl	$1048576, %edx
	movl	$64, %esi
	movq	%rax, %rdi
	call	memset
	movl	$0, -20(%rbp)
	jmp	.L2
.L5:
	leaq	-1048608(%rbp), %rax
	movq	%rax, -16(%rbp)
	leaq	-2097184(%rbp), %rax
	movq	%rax, -8(%rbp)
	jmp	.L3
.L7:
	nop
.L3:
	movq	-8(%rbp), %rax
	movzbl	(%rax), %edx
	movq	-16(%rbp), %rax
	movb	%dl, (%rax)
	movq	-16(%rbp), %rax
	leaq	1(%rax), %rdx
	movq	-8(%rbp), %rax
	addq	$1, %rax
	movzbl	(%rax), %eax
	movb	%al, (%rdx)
	movq	-16(%rbp), %rax
	leaq	2(%rax), %rdx
	movq	-8(%rbp), %rax
	addq	$2, %rax
	movzbl	(%rax), %eax
	movb	%al, (%rdx)
	movq	-16(%rbp), %rax
	leaq	3(%rax), %rdx
	movq	-8(%rbp), %rax
	addq	$3, %rax
	movzbl	(%rax), %eax
	movb	%al, (%rdx)
	movq	-16(%rbp), %rax
	leaq	4(%rax), %rdx
	movq	-8(%rbp), %rax
	addq	$4, %rax
	movzbl	(%rax), %eax
	movb	%al, (%rdx)
	movq	-16(%rbp), %rax
	leaq	5(%rax), %rdx
	movq	-8(%rbp), %rax
	addq	$5, %rax
	movzbl	(%rax), %eax
	movb	%al, (%rdx)
	movq	-16(%rbp), %rax
	leaq	6(%rax), %rdx
	movq	-8(%rbp), %rax
	addq	$6, %rax
	movzbl	(%rax), %eax
	movb	%al, (%rdx)
	movq	-16(%rbp), %rax
	leaq	7(%rax), %rdx
	movq	-8(%rbp), %rax
	addq	$7, %rax
	movzbl	(%rax), %eax
	movb	%al, (%rdx)
	addq	$8, -16(%rbp)
	addq	$8, -8(%rbp)
	leaq	-1048608(%rbp), %rax
	addq	$1048576, %rax
	cmpq	-16(%rbp), %rax
	jne	.L7
.L4:
	addl	$1, -20(%rbp)
.L2:
	cmpl	$4095, -20(%rbp)
	jbe	.L5
	movl	$0, %eax
	leave
	ret
	.cfi_endproc
.LFE0:
	.size	main, .-main
	.ident	"GCC: (GNU) 4.4.1 20090725 (Red Hat 4.4.1-2)"
	.section	.note.GNU-stack,"",@progbits

	.file	"?"
	.text
.globl main
	.type	main, @function
main:
.LFB0:
	.cfi_startproc
	pushq	%rbp
	.cfi_def_cfa_offset 16
	movq	%rsp, %rbp
	.cfi_offset 6, -16
	.cfi_def_cfa_register 6
	subq	$2097184, %rsp
	leaq	-2097184(%rbp), %rax
	movl	$1048576, %edx
	movl	$64, %esi
	movq	%rax, %rdi
	call	memset
	movl	$0, -20(%rbp)
	jmp	.L2
.L5:
	leaq	-1048608(%rbp), %rax
	movq	%rax, -16(%rbp)
	leaq	-2097184(%rbp), %rax
	movq	%rax, -8(%rbp)
	jmp	.L3
.L4:
	movq	-8(%rbp), %rax
	movzbl	(%rax), %edx
	movq	-16(%rbp), %rax
	movb	%dl, (%rax)
	addq	$1, -16(%rbp)
	addq	$1, -8(%rbp)
.L3:
	leaq	-1048608(%rbp), %rax
	addq	$1048576, %rax
	cmpq	-16(%rbp), %rax
	jne	.L4
	addl	$1, -20(%rbp)
.L2:
	cmpl	$4095, -20(%rbp)
	jbe	.L5
	movl	$0, %eax
	leave
	ret
	.cfi_endproc
.LFE0:
	.size	main, .-main
	.ident	"GCC: (GNU) 4.4.1 20090725 (Red Hat 4.4.1-2)"
	.section	.note.GNU-stack,"",@progbits