Extending with Plugins

Extension points

NFT is leveraging MEF (Microsoft Extension Framework) for dependency injection (inversion of control). There are four interfaces which may be used to hook into NFT:

public interface ISourceReaderFactory
{
    bool CanReadSource(string source);

    ISourceReader CreateReader(string source, string config);
    bool SupportsQuery { get; }
}

public interface ITargetWriterFactory
{
    bool CanWriteTarget(string target);
    ITargetWriter CreateWriter(string target, string[] fieldNames, int[] fieldSizes, string config);
}

public interface IOperator : IEvaluator
{
    ExpressionType Type { get; }
    string Name { get; }
    int ParamCount { get; }
    ParamType[] ParamTypes { get; }
    ParamType ReturnType { get; }

    void Configure(string config);
}

// Derived from
public interface IEvaluator
{
    string Evaluate(IEvaluator eval, IExpression expression, IContext context);
}

public interface ILoggerFactory
{
    string LogType { get; }
    ILogger CreateLogger(string configuration, LogLevel logLevel);
}

As to connectivity, i.e. reading and writing, both interfaces make use of methods to find out whether the reader/writer can read/write the source/target, based on the format of the source/target string.

As an example, the CSV Plugin returns true for URIs starting with file:// and ending with .csv.

For operators, things are even simpler. All you need to do is to implement the IOperator interface, using the ExpressionType.Custom expression type, have it exported as IOperator via MEF, and the new operator will be picked up automatically by NFT. See below for more information on writing operators.

Also loggers can be plugged in to NFT, using the ILoggerFactory interface, producing ILogger instances which can be used for logging purposes.

Writing plugins

Writing a plugin consists of following these steps:

• Create a new assembly for .NET 4.0 and reference NoFrillsTransformation.Interfaces and System.ComponentModel.Composition (this is the MEF framework)
• Implement ISourceReaderFactory and/or ITargetWriterFactory
• Mark the classes as [Export(typeof(ISourceReaderFactory))] or [Export(typeof(ITargetWriterFactory))], respectively (see below)
• Implement the ISourceReader and IRecord interfaces for reading sources
• And/or implement the ITargetWriter interface for writing to targets

Make sure that the assembly resides in the same directory as the EXE file; NFT should now automatically pick up the plugin and start reading and/or writing from/to the new source/target.

Snippet from the CsvWriterFactory:

[Export(typeof(NoFrillsTransformation.Interfaces.ITargetWriterFactory))]
public class CsvWriterFactory : ITargetWriterFactory
{
    public bool CanWriteTarget(string target)
    {
        if (string.IsNullOrWhiteSpace(target))
            return false;
        string temp = target.ToLowerInvariant();
        if (!temp.StartsWith("file://"))
            return false;
        if (!temp.EndsWith(".csv") && !temp.EndsWith(".txt"))
            return false;
        return true;
    }

    public ITargetWriter CreateWriter(IContext context, string target, string[] fieldNames, int[] fieldSizes, string config)
    {
        return new CsvWriterPlugin(context, target, fieldNames, fieldSizes, config);
    }
}

Possible extension plugins could be stuff like:

• ODBC reader/writer
• SQLite reader/writer
• XML reader/writer
• Salesforce SOQL reader, Salesforce writer

Writing Plugin operators

Writing custom operators isn't difficult, actually. You have two possibilities how to get started:

• Forking the repository and adding the operators directly to NoFrillsTransformation.Operators
• Writing your own assembly containing operators which adher to the IOperator interface.

If you go for the second method (which enables you to use the latest NFT releases and hook into that), you need to follow these steps:

• Create a new assembly for .NET 4.0 and reference NoFrillsTransformation.Interfaces and System.ComponentModel.Composition (this is the MEF framework)
• Implement IOperator
• Mark the class as [Export(typeof(IOperator))]
• Make sure your assembly is located side by side with NoFrillsTransformation.exe

Example operator implementation

The following code shows a sample implementation of an operator. Obviously all using statements and namespace definitions have been left out. Another place to check for the code is inside the NoFrillsTransformation.Operators assembly.

public class ReverseOperator
{
	public ReverseOperator()
	{
		_paramTypes = new ParamType[] { ParamType.Any };
	}

	private ParamType[] _paramTypes;

	public ExpressionType Type { get { return ExpressionType.Custom; } }
	public string Name { get { return "reverse"; } }
	public string ParamCount { get { return 1; } }
	public ParamType[] ParamTypes { get { return _paramTypes; } }
	public ParamType ReturnType { get { return ParamType.String; } }

	public string Evaluate(IEvaluator eval, IExpression expression, IContext context)
	{
		string parameter = eval.Evaluate(eval, expression.Arguments[0], context);
		char[] charArray = parameter.ToCharArray();
		Array.Reverse(charArray);
		return new string(charArray);
	}

	public void Configure(string config)
	{
		// This operator does not need configuring
	}
}

This operator implements a reverse operator. Please note the use of the IEvaluator in the Evaluate method of the operator. You will get passed the parameter into your Evaluate method; this has not yet been evaluated, thus you need to call that method recursively first, before you actually perform your operator on the output.

This gives you full flexibility in implementing custom operators. In fact, only the three "special" operators are not implemented in the Operators assembly, in the way shown above, and those are the "field name", the "string literal" and the "lookup" operators, as these require sepcial functionality. All others rely on this pattern, some taking information from the passed IContext parameter (e.g. the TargetRowNow and SourceRowNum operators).

#### Operator Configuration

NFT offers a way to configure operators. Most built in operators do not support any kind of configuration, but some do, as the Equals Operator, or the FileTextWrite Operator.

Configuration on operator level can be passed from the configuration XML file, in this way:

<?xml version="1.0" encoding="utf-8"?>
<Transformation>
  <!-- left out the rest -->
  <OperatorConfigs>
    <OperatorConfig name="equals">ignorecase</OperatorConfig>"
  </OperatorConfigs>
</Transformation>

The string in the OperatorConfig tags will be passed on to the operator's Configure method, as shown in the IOperator interface above.

### Writing plugin loggers

Writing custom loggers is also possible. As with the operators, you have two possibilities how to get started:

• Forking the repository and adding the loggers directly to NoFrillsTransformation.Logging
• Writing your own assembly containing operators which adher to the ILoggerFactory and ILogger interface.

If you go for the second method (which enables you to use the latest NFT releases and hook into that), you need to follow these steps:

• Create a new assembly for .NET 4.0 and reference NoFrillsTransformation.Interfaces and System.ComponentModel.Composition (this is the MEF framework)
• Implement ILoggerFactory
• Mark the class as [Export(typeof(ILogger))]
• Implement your logger class, implementing the ILogger interface
• Make sure your assembly is located side by side with NoFrillsTransformation.exe
• Add a Logger type into your NFT configuration with your newly created log type string, e.g. loggregate, odbc or whatever you chose as LogType.