Thursday, December 07, 2006

CCR + DSSP = Distributed Scalable Buffer Devices

Technorati tags: , ,
After much fiddling around and moving of code I've finally got to a point where I can test the implementation of a feature I call distributed buffer device service. Now before I go on to describe what this you might want to read about the CCR and DSSP sub-systems upon which all the service implementations are based...
Okay so where was I? Ah yes, the past two months I investigated whether it was possible (and desirable) to write parts of the database engine as DSSP services.
I had already integrated and converted the code-base to utilise the CCR framework rather than using the difficult to code/debug Asynchronous Programming Model - shame really since I'd become rather good at writing those wrappers!
So this investigative process was really a continuation of existing work. The initial implementation of a Physical Buffer Service was simple enough and even compiled and built without too much hassle however the Container Buffer Service (which is regarded as the minimum service needed to perform useful testing) ran into difficult and damn obscure issues - all related to the generation of the proxy project code.
I have finally (after much hair pulling - very painful considering I've no hair on my head) got this Container Buffer Service to compile and the proxy service to build! Wow!
So what's the point of all this abstraction? Well DSSP allows services to communicate with each other using HTTP and hence each service need not exist on a single machine - now since our services are now DSSP services then we automatically get distributed physical device services - now we didn't have THAT before so this must be considered "progress"...
Now the test harness for these services is actually an NT service - I call this service the "Block File-System Service" and this could well form the underpinnings to the Audio Database service.
The implications of all this is the fact that the database file-group devices will maintain a one-to-many relationship with FileSystem service instances running on potentially multiple machines - sounds super uber scalable to me...
Once I have Container Buffer Services working - it will be time to look at the caching version. Note the caching implementation will provide caching at both ends of the network connection to increase networking performance.