Sunday, April 16, 2006

Asynchronous Persistence

Major change is underway in order to clean the implementation of asynchronous persistence used through out the database classes.

Seemed to me that there was far too many methods and more than a little confusion in the implementation so I've simplified the arrangement of devices with respect to loading, saving and initialising DeviceBuffer objects.

These simplified BufferDevice objects will handle the low-level buffer load/save/init operations and be wrapped by a single class used to handle page-level processing.

This arrangement will be easier to test and have better performance due to a significant reduction in the number of method calls...

It means revising an awful lot of code which is more than a small pain but one well worth taking on!

Monday, April 10, 2006

Transaction Logging

Installation testing is going very well and more of the various subsystems are undergoing functional testing now. The maze of tasks involved with installing hierarchical devices have largely been solved so now attention has turned towards the creating .NET transactions for wrapping the overal install and getting the custom transaction implementation to enlist itself into this framework feature.

Well the enlistment part was fairly straightforward with the designed classes needing only minor modifications in order to start working however the changes needed to handle saving transacted pages involved a little more head scratching and code tweaking! The freshly written code was saving pages and their associated buffers directly to the underlying device - this is clearly illegal if you want a recoverable system!

Ultimately the code was modified for transacted buffers such that calls to SavePage will update the database transaction holder with information pertaining to the buffer and the current timestamp. During the Commit-Preparation phase these buffers can be committed (scratchpad data moved to write-pending area) and the transaction log records can be generated from the two images (or single image in the case of newly initialised buffers). Finally in Commit phase the Commit log record is written to validate the log records.

That's the idea at least - so far the distribution pages are following this regime fine but root pages seem to have a mind of their own - it's that or I'm not actually saving them...

All this work meant I needed to provide an implementation of the CheckPoint handler at long last and thankfully this has proved relatively easy - the only real problem seems to be with the cache management threads which don't seem to be unlocking the cache in all scenarios - still at least I know where the problem is - multithreaded mayhem can be a pain to debug but .NET gives us flexible tracing!

Development should move into overdrive following the arrival of a new desk and chair for comfortable programming however the country-wide mayhem that is Songkran now lies directly in my path so it's all on hold for the next 5 days or so - fun fun fun (with a water gun!)

Sunday, April 09, 2006

Installers Reach Runnable Stage

Been working like a demon despite being on holiday from my holiday in Hanoi!!!

The installation components are now being rigourously tested by a new test-harness and as a result the call sequence and the mount operations performed by devices have been debugged and tweaked so that now the installation completes without problem.

It is still not a complete success - I need to check the log-writer is logging the writes to the root pages and I need to check the data device root pages are correct. After all that is done I will be able to test the non-create mount operation and move delicately onto testing the recovery logic which could be quite painful!

Found that VS2005 disconnected check-outs from Source Control work like a dream and my edits are checked in successfully and with a minimum of fuss (well once it worked out the network drive was back online that is...)

Installers Reach Runnable Stage

Been working like a demon despite being on holiday from my holiday in Hanoi!!!

The installation components are now being rigourously tested by a new test-harness and as a result the call sequence and the mount operations performed by devices have been debugged and tweaked so that now the installation completes without problem.

It is still not a complete success - I need to check the log-writer is logging the writes to the root pages and I need to check the data device root pages are correct. After all that is done I will be able to test the non-create mount operation and move delicately onto testing the recovery logic which could be quite painful!

Found that VS2005 disconnected check-outs from Source Control work like a dream and my edits are checked in successfully and with a minimum of fuss (well once it worked out the network drive was back online that is...)