VO –> Vulcan regex

We’re about to do a bunch of ‘Transporting’ from VO to Vulcan in the next few days so I’m probably going to figure out a bunch of useful tricks for helping to automate the process.

So far things are looking good, our unit tests our now transporting cleanly and there’s no incompatibilities exposed by them.

One thing I did need to do was remove all the PSZ <-> STRING conversions.  The easiest way to do that was to use Visual Studio’s ‘Replace in files’ option and some regex’s.


They are the things you want to find, replace them with \1

Note that Visual Studio doesn’t appear to support .*? so those matches may be too eager.  In my case I just ended up with extra brackets left around my variables.  Syntactically okay but ugly looking.

Thinking about it, I have the feeling last time I gave up on Visual Studio and just used perl the last time I needed to do this.  I guess if I have any more complex things to do I will.

More Logging

Now that I’ve figured out what I want a logging framework to do I’ve taken a look at some interesting existing frameworks.  log4net is a .net logging framework based on a java framework – log4j.  It’s hosted on the apache website which gives it extra credibility in my view.  Looking at the feature set it looks like it covers pretty much what I need. 

As far as sending emails go there’s a convenient way to get emails out when there’s a problem with a sample config in the examples.

<appender name="SmtpAppender" type="log4net.Appender.SmtpAppender">
    <to value="to@domain.com" />
    <from value="from@domain.com" />

    <subject value="test logging message" />
    <smtpHost value="SMTPServer.domain.com" />
    <bufferSize value="512" />

    <lossy value="true" />
    <evaluator type="log4net.Core.LevelEvaluator">
        <threshold value="WARN"/>

    <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%newline%date [%thread] %-5level %logger [%property{NDC}] - %message%newline%newline%newline" />

What that does is send an email when a warning comes along and also sends upto the previous 512 messages (of any kind) as context. That wasn’t exactly what I was thinking of but it’s certainly good enough for the emails.

What I want from a logging framework

I keep being unsatisfied with our applications logging and I’ve decided to take a look at what I really want.

Obviously we want different levels of severity,

  • Exception
  • Error
  • Warning
  • Note
  • Debug

Those severities imply that we can filter by them and that we can possibly do different things based on encountering them. We also need to be able to specify different targets – file, debug view, event log and email.

There’s also another aspect that we don’t often think about but which I think is quite important, tying together the log messages into some sort of unit. This might be something like the import of an order on a system or perhaps a web page request.

There’s another thing related to that, timing. Not everything wants to be logged immediately. When we log straight to a file or a debug log we normally just keep writing the stream as we encounter it. If we want to log to an email we might want to batch together some of those message though. That also ties in quite neatly with the idea of units of work.

This would mean that if I have an error occur within a task I could see all the messages within that task, not just the errors. This would be an ‘intelligent’ mode of operation, rather than just one tied to a severity level.

This gives us a set of features something like this,

  1. Severity filtering
  2. Tying messages together into a unit of work or task
  3. Delayed sending
  4. Possibly optional logging based on severity, i.e. only display the notes if an error occurs.

Honestly we probably also want to have several separate configurations in action at once too so there’s the another feature to consider,

  1. A flexible configuration mechanism – this should be able to deal with multiple setups running at the same time

This makes me think a simple interface like this would be best,

public interface Logger
	void StartLevel(string name);
	void EndLevel(string name);
	void Error(string message);
	void Note(string message);
	void Warning(string message);
	void Debug(string message);
	void Exception(string message);

All that we would do is to create new classes that implement that to do the functionality we want. We presumably want them to be able to be composed of different ones and have different configurations but essentially when we log we log through an interface like that and it gets sent to a bunch of objects exposing that API.

In practice we would probably use it something like this,

Log.StartLevel("Interface run");
Log.Note("Interface running");
Log.StartLevel("Order Import");
Log.Note("Importing orders...");
Log.Note("Importing order XXXXX1");
Log.Note(" for destination WHS1");
Log.Error("Product D898223 does not exist, order XXXXX1 not imported");
Log.Note("Importing order XXXXX2");
Log.Note("Importing order XXXXX3");
Log.Note("Importing order XXXXX4");
Log.EndLevel("Order Import");
Log.EndLevel("Interface run"); // an email gets sent if an error occurred

One thing that irks is that if we have several paths storing the messages for some condition to become true, like sending an email when the interface has run or logging if an error occurs. We could even be storing all the debug messages when we are highly unlikely to send them. In practice though I think it’s worth keeping it simple, if there are really enough messages building up between flushes I suspect that we must have the applications architecture wrong. If not it can always be optimized then.

The configuration is the tricky bit.

<!-- uncomment this to push debug output so that you can monitor it via DbgView 
<debug-output />
<subject>Interface problems</subject>
<trigger-level>Interface run</trigger-level>

I think that makes sense as a basic API and configuration mechanism. I think in practice it will need tidying and some more complexity adding but that should be the basics. All that I now need to do is the magic in between to make it work.

There’s also the problem of what to do in the case of exceptions to resolve. Ideally in those cases you would shortcut the trigger levels; especially since if we’ve just caught an exception it’s probably just busted through all those carefully constructed levels. Alternatively perhaps you simply add an alternative configuration block for them. Then it’s explicit and there’s no hard wired behaviour or complex contortions to try to express something complex in xml. Perhaps we’d just add a ‘flush log’ to implicitly dump all the previous log messages stored along with the actual exception report.