What a difference two years makes

Preamble

When I started this project some 4 years ago I really didn’t think I would still be writing it but here I am… To be fair much has happened in the past two years that has kept me away from this creation – I’ve changed country, lifestyle and even computer systems!

Not only that but the .NET framework has evolved from .NET 1.1 when I started this beast into .NET 3.5 SP1. I imagine that the code will be ported to .NET 4.0 when that sees the light of day in due course too…

Following the last post, the low-level entities were looked at in great detail – so much so that it took some three months to fix the problems this “look” caused. However it was all worthwhile – the low-level memory management, sparse file and overlapped I/O logic has been successfully used in a separate project (a multi-request HTTP file downloader) to great effect. The resultant system was very fast, resilient and efficient!

The asynchronous code was successfully ported from the .NET APM (that’s Asynchronous Processing Model to you) to the Microsoft CCR which resulted in code that was much easier to test, maintain and extend. Licensing issues mean that I may yet change the underlying concurrency framework used but for the moment CCR is king!

So where are we now?

State = current

Well the index implementation was always half-complete – as in index entries can be added but not removed and if that wasn’t enough – some of the index models have not been implemented (such as clustered indexing)

So the first stage is creating a suitable test rig for playing with paged index trees and writing the necessary code to support adding/removing entries and hopefully balancing the trees too!

With working indices we then get finalised table handling and then I will at long last be able to look at the next phase – hosting a table-driven neural network (actually a page-based neural network would be rather nice and might come first)

C# Port to 3.5

The project has long since been migrated to .NET 3.5 and now it is running on 3.5SP1 but the language features are only being taken advantage of with new code – I need to revisit all code classes to ensure the best use of the language features are being made use of – in particular – more use of LINQ – instead of explicit loop constructs. This work does have a purpose – Parallel LINQ is already here and could form an alternative to using the CCR in certain cases plus there is much talk of merging the codebase of CCR with that of Parallel LINQ – so having CCR code and LINQ code in place makes the next transition much easier to make – when I am called upon to do so, and they say future proofing is impossible…

Code expose

Yes I may begin to expose some of the miles and miles of source code I have painstakingly put together for this project – however I will not be releasing the software into Code Project / CodePlex or any other open-source repository – it’s taken too much of my time to give freely!

Anyway – that’s enough for now – this post was really about catch-up! Now it’s time for sleep!