Summary: The three databases, MiniSQL, MySQL and DB2 are benchmarked for comparison using a variety of typical tasks and loads.

Performance measurements were conducted on a comparatively slow system, i.e. 133 mhz Pentium processor with 128 megabytes of memory and a single slow (5,000rpm) SCSI drive. DB2 and other heavy weight relational databases perform significantly better when they have access to multiple drives and multiple CPU's to take advantage of parallel operations, so these measurements distinctly favor mSQL and MySQL. However, the test system is more typical of what a light weight relational database would run on.

All operations were performed via SQL from the 'console' which comes with each database. This measures the performance of the console as well as the underlying database engine. This should not be important for the more intensive requests (which take much longer than a second). For very fast requests the console can be the primary bottleneck. However, for these fast requests, the actual path you take to get to the database engine will also likely be the primary performance bottleneck and you will have to take that into consideration. For most web based applications, the external delays of just getting the request to and from your server dominates over any but the most serious database operations.

As a general rule, the light weight relational databases were much faster on simple requests but the performance degraded as the requests became more complex and the tables being accessed got larger. The following operations were performed in the performance measurements:

Test	Description
Create Table	Create the test table with several columns and a primary key composed of an ID in two columns and a date. This also created a sorted index into the table based on these columns.
Load Data	Load the table with data of either 1,000, 10,000, or 100,000 records. The data is real data excerpts from an existing DB2 database. As I could not get mSQLimport to work correctly in time for the performance test, this step was completed by doing the required inserts from the mSQL console (via a script file). However this does not seem to have adversely delayed the load time as the corrected mSQLimport took almost three times as long as the mSQL console to load records in the 1,000 record load.
Run Stats	An operation unique to DB2, allowing the query optimization steps to choose the appropriate strategy later on.
Simple Update	Update a column in every records in the table based on the value in a different column.
Complex Update	Update a column in those records where there are later records in the same table with the same ID. This entails a table join (literally in the SQL) which should simplified to an index scan of the primary key. This can not be done in mSQL and has to be done in multiple steps in MySQL because of missing functions.
Simple Count	Count the number of records in the table meeting a simple criteria. Since mSQL does not support the count function, all records were retrieved. This would have to be done by your program in order to count the records returned.
Group Count	Count the number of records in the table after grouping them according to the value in a particular column. Since mSQL does not support the count function or 'group by' option, all records were retrieved after having mSQL order them with the 'order by' option. This would have to be done by your program in order to count the records returned.
Complex Count	Count number of records in the table meeting a complex criteria based on whether or not there are later records in the same table with the same ID and a later date. This entails a table join (literally in the SQL) which should simplified to an index scan of the primary key. mSQL does not support the count function, so all records were retrieved as this would have to be done by your program in order to count the records returned.
Index Select	Perform a series of quick selects (50) where the record requested can be found quickly through the index (primary key). This should not degrade significantly with table size and then mostly measures console overhead (the overhead of getting 50 requests into and out of the database engine).
Full Table Select	Perform a series of slow selects (10) where the records requested require a series of full table scans to identify the records. This should degrade linearly with the table size.

The raw numbers are as follows:

Test	MiniSQL			MySQL			DB2
Table Size	1000	10000	100000	1000	10000	100000	1000	10000	100000
Create Table [View Graph](.GIF, 10K)	2	2	2	1.7	2	1.7	0	0.3	0.0001
Load Data [View Graph](.GIF, 7K)	13.3	96.3	960.7	1.7	14.7	140.7	4	45.3	450
Run Stats	N/A	N/A	N/A	N/A	N/A	N/A	1	4.3	13
Simple Update [View Graph](.GIF, 6K)	0	4	144	11.7	118.7	1158.3	0.3	5	196.7
Complex Update	N/A	N/A	N/A	15.3	136.7	1416.7	1	8.3	239
Simple Count [View Graph](.GIF, 5K)	1.3	5	120.7	0	0	1	0	1	4.7
Group Count	2.7	41.7	Fail	0.3	1.3	14.3	1	4	12.3
Complex Count [View Graph](.GIF, 5K)	68.7	4295	Fail	0.3	7.3	86.3	0	2.3	51.7
Index Select [View Graph](.GIF, 7K)	0.3	0	1.7	1	0.7	1	15.3	15	15
Full Table Select [View Graph](.GIF, 7K)	0.3	2	13.3	0.7	1.7	9.3	4.7	6.7	54

On updates, mSQL could only do the simple updates, but it was blindingly fast. MySQL was a comparative laggard, but this is almost certainly due to forcing a disk write on each update. DB2 did better by caching all writes and then forcing the updates to disk in blocks on completion of the request. Also, the version of mSQL that I tested had a bug (reported and identified but not fixed by the time of this review) that cached too well in that I was unable to see recent additions to a table after an insert completed.

All tests were performed until the results stabilized and then measured three times with the reported results the average of these three. This was important as DB2 learns from previous requests the nature of the requests that are normally submitted and performance improves over time in a consistent environment. Each test was run as a script and all tests were run sequentially with a stop and start of the database engine in between.

mSQL supports table joins in selects, but does not seem to recognize that the specified join did not require building a new (and huge) logical table, but simply required an index scan. As such, the performance of the Complex Count (join) degraded as the square of the table size and was not tested for the 100,000 record table. This test was only run once for the 10,000 record test table due to the long time to complete. mSQL also aborted on the 'order by' clause of the 'Group Count' test with 100,000 records.

DB2 showed the worst performance on the Index Select, which was 50 very easy selects indicating that the normal path through the DB2 console to the database engine and back is about a third of a second. It seems to less than a hundredth of second for mSQL and about a fiftieth of a second for MySQL. However, console performance is seldom an overall throughput issue and the improved error messages by DB2 may warrant the increased delays by the console.