This document covers how to write a TypeScript Transformer.
- Introduction
- The basics
- Transformer API
- Writing your first transformer
- Types of transformers
- Consuming transformers
- Transformation operations
- Tips & tricks
- Testing
- Known bugs
TypeScript is a typed superset of Javascript that compiles to plain Javascript. TypeScript supports the ability for consumers to transform code from one form to another, similar to how Babel does it with plugins.
Follow me @itsmadou for updates and general discourse
There are multiple examples ready for you to use through this handbook. When you want to take the dive make sure to:
- clone the repo
- install deps with
yarn - build the example you want
yarn build example_name
A transformer when boiled down is essentially a function that takes and returns some piece of code, for example:
const Transformer = code => code;The difference though is that instead of code being of type string -
it is actually in the form of an abstract syntax tree (AST),
described below.
With it we can do powerful things like updating,
replacing,
adding,
& deleting nodes.
Abstract Syntax Trees, or ASTs, are a data structure that describes the code that has been parsed. When working with ASTs in TypeScript I'd strongly recommend using an AST explorer - such as ts-ast-viewer.com.
Using such a tool we can see that the following code:
function hello() {
console.log('world');
}In its AST representation looks like this:
-> SourceFile
-> FunctionDeclaration
- Identifier
-> Block
-> ExpressionStatement
-> CallExpression
-> PropertyAccessExpression
- Identifier
- Identifier
- StringLiteral
- EndOfFileToken
For a more detailed look check out the AST yourself! You can also see the code can be used to generate the same AST in the bottom left panel, and the selected node metadata in the right panel. Super useful!
When looking at the metadata you'll notice they all have a similar structure (some properties have been omitted):
{
kind: 288, // (SyntaxKind.SourceFile)
pos: 0,
end: 47,
statements: [{...}],
}{
kind: 243, // (SyntaxKind.FunctionDeclaration)
pos: 0,
end: 47,
name: {...},
body: {...},
}{
kind: 225, // (SyntaxKind.ExpressionStatement)
pos: 19,
end: 45,
expression: {...}
}
SyntaxKindis a TypeScript enum which describes the kind of node. For more information have a read of Basarat's AST tip.
And so on.
Each of these describe a Node.
ASTs can be made from one to many -
and together they describe the syntax of a program that can be used for static analysis.
Every node has a kind property which describes what kind of node it is,
as well as pos and end which describe where in the source they are.
We will talk about how to narrow the node to a specific type of node later in the handbook.
Very similar to Babel - TypeScript however has five stages, parser, binder, checker, transform, emitting.
Two steps are exclusive to TypeScript, binder and checker. We are going to gloss over checker as it relates to TypeScripts type checking specifics.
For a more in-depth understanding of the TypeScript compiler internals have a read of Basarat's handbook.
Before we continue we need to quickly clarify exactly what a Program is according to TypeScript.
A Program is a collection of one or more entrypoint source files which consume one or more modules.
The entire collection is then used during each of the stages.
This is in contrast to how Babel processes files - where Babel does file in file out, TypeScript does project in, project out. This is why enums don't work when parsing TypeScript with Babel for example, it just doesn't have all the information available.
The TypeScript parser actually has two parts,
the scanner,
and then the parser.
This step will convert source code into an AST.
SourceCode ~~ scanner ~~> Token Stream ~~ parser ~~> AST
The parser takes source code and tries to convert it into an in-memory AST representation which you can work with in the compiler. Also: see Parser.
The scanner is used by the parser to convert a string into tokens in a linear fashion, then it's up to a parser to tree-ify them. Also: see Scanner.
Creates a symbol map and uses the AST to provide the type system which is important to link references and to be able to know the nodes of imports and exports. Also: see Binder.
This is the step we're all here for. It allows us, the developer, to change the code in any way we see fit. Performance optimizations, compile time behavior, really anything we can imagine.
There are three stages of transform we care about:
before- which run transformers before the TypeScript ones (code has not been compiled)after- which run transformers after the TypeScript ones (code has been compiled)afterDeclarations- which run transformers after the declaration step (you can transform type defs here)
Generally the 90% case will see us always writing transformers for the before stage,
but if you need to do some post-compilation transformation,
or modify types,
you'll end up wanting to use after and afterDeclarations.
Tip - Type checking should not happen after transforming. If it does it's more than likely a bug - file an issue!
This stage happens last and is responsible for emitting the final code somewhere. Generally this is usually to the file system - but it could also be in memory.
When wanting to modify the AST in any way you need to traverse the tree - recursively. In more concrete terms we want to visit each node, and then return either the same, an updated, or a completely new node.
If we take the previous example AST in JSON format (with some values omitted):
{
kind: 288, // (SyntaxKind.SourceFile)
statements: [{
kind: 243, // (SyntaxKind.FunctionDeclaration)
name: {
kind: 75 // (SyntaxKind.Identifier)
escapedText: "hello"
},
body: {
kind: 222, // (SyntaxKind.Block)
statements: [{
kind: 225, // (SyntaxKind.ExpressionStatement)
expression: {
kind: 195, // (SyntaxKind.CallExpression)
expression: {
kind: 193, // (SyntaxKind.PropertyAccessExpression)
name: {
kind: 75 // (SyntaxKind.Identifier)
escapedText: "log",
},
expression: {
kind: 75, // (SyntaxKind.Identifier)
escapedText: "console",
}
}
},
arguments: [{
kind: 10, // (SyntaxKind.StringLiteral)
text: "world",
}]
}]
}
}]
}If we were to traverse it we would start at the SourceFile and then work through each node.
You might think you could meticulously traverse it yourself,
like source.statements[0].name etc,
but you'll find it won't scale and is prone to breaking very easily -
so use it wisely.
Ideally for the 90% case you'll want to use the built in methods to traverse the AST. TypeScript gives us two primary methods for doing this:
Generally you'll only pass this the initial SourceFile node.
We'll go into what the visitor function is soon.
import * as ts from 'typescript';
ts.visitNode(sourceFile, visitor);This is a special function that uses visitNode internally.
It will handle traversing down to the inner most node -
and it knows how to do it without you having the think about it.
We'll go into what the context object is soon.
import * as ts from 'typescript';
ts.visitEachChild(node, visitor, context);The visitor pattern is something you'll be using in every Transformer you write,
luckily for us TypeScript handles it so we need to only supply a callback function.
The simplest function we could write might look something like this:
import * as ts from 'typescript';
const transformer = sourceFile => {
const visitor = (node: ts.Node): ts.Node => {
console.log(node.kind, `\t# ts.SyntaxKind.${ts.SyntaxKind[node.kind]}`);
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};Note - You'll see that we're returning each node. This is required! If we didn't you'd see some funky errors.
If we applied this to the code example used before we would see this logged in our console (comments added afterwords):
288 # ts.SyntaxKind.SourceFile
243 # ts.SyntaxKind.FunctionDeclaration
75 # ts.SyntaxKind.Identifier
222 # ts.SyntaxKind.Block
225 # ts.SyntaxKind.ExpressionStatement
195 # ts.SyntaxKind.CallExpression
193 # ts.SyntaxKind.PropertyAccessExpression
75 # ts.SyntaxKind.Identifier
75 # ts.SyntaxKind.Identifier
10 # ts.SyntaxKind.StringLiteralTip - You can see the source for this at /example-transformers/log-every-node - if wanting to run locally you can run it via
yarn build log-every-node.
It goes as deep as possible entering each node, exiting when it bottoms out, and then entering other child nodes that it comes to.
Every transformer will receive the transformation context.
This context is used both for visitEachChild,
as well as doing some useful things like getting a hold of what the current TypeScript configuration is.
We'll see our first look at a simple TypeScript transformer soon.
Most of this content is taken directly from the Babel Handbook as the same principles apply.
Next let's introduce the concept of a scope. Javascript has lexical scoping (closures), which is a tree structure where blocks create new scope.
// global scope
function scopeOne() {
// scope 1
function scopeTwo() {
// scope 2
}
}Whenever you create a reference in Javascript, whether that be by a variable, function, class, param, import, label, etc., it belongs to the current scope.
var global = 'I am in the global scope';
function scopeOne() {
var one = 'I am in the scope created by `scopeOne()`';
function scopeTwo() {
var two = 'I am in the scope created by `scopeTwo()`';
}
}Code within a deeper scope may use a reference from a higher scope.
function scopeOne() {
var one = 'I am in the scope created by `scopeOne()`';
function scopeTwo() {
one = 'I am updating the reference in `scopeOne` inside `scopeTwo`';
}
}A lower scope might also create a reference of the same name without modifying it.
function scopeOne() {
var one = 'I am in the scope created by `scopeOne()`';
function scopeTwo() {
var one = 'I am creating a new `one` but leaving reference in `scopeOne()` alone.';
}
}When writing a transform we want to be wary of scope. We need to make sure we don't break existing code while modifying different parts of it.
We may want to add new references and make sure they don't collide with existing ones. Or maybe we just want to find where a variable is referenced. We want to be able to track these references within a given scope.
References all belong to a particular scope; this relationship is known as a binding.
function scopeOnce() {
var ref = 'This is a binding';
ref; // This is a reference to a binding
function scopeTwo() {
ref; // This is a reference to a binding from a lower scope
}
}When writing your transformer you'll want to write it using TypeScript.
You'll be using the typescript package to do most of the heavy lifting.
It is used for everything,
unlike Babel which has separate small packages.
First, let's install it.
npm i typescript --saveAnd then let's import it:
import * as ts from 'typescript';Tip - I strongly recommend using intellisense in VSCode to interrogate the API, it's super useful!
These methods are useful for visiting nodes - we've briefly gone over a few of them above.
ts.visitNode(node, visitor)- useful for visiting the root node, generally theSourceFilets.visitEachChild(node, visitor, context)- useful for visiting each child of a nodets.isXyz(node)- useful for narrowing the type of anode, an example of this ists.isVariableDeclaration(node)
These methods are useful for modifying a node in some form.
-
ts.createXyz(...)- useful for creating a new node (to then return), an example of this ists.createIdentifier('world')Tip - Use ts-creator to quickly get factory functions for a piece of TypeScript source - instead of meticulously writing out an AST for a node you can write a code string and have it converted to AST for you.
-
ts.updateXyz(node, ...)- useful for updating a node (to then return), an example of this ists.updateVariableDeclaration() -
ts.updateSourceFileNode(sourceFile, ...)- useful for updating a source file to then return -
ts.setOriginalNode(newNode, originalNode)- useful for setting a nodes original node -
ts.setXyz(...)- sets things -
ts.addXyz(...)- adds things
Covered above, this is supplied to every transformer and has some handy methods available (this is not an exhaustive list, just the stuff we care about):
getCompilerOptions()- Gets the compiler options supplied to the transformerhoistFunctionDeclaration(node)- Hoists a function declaration to the top of the containing scopehoistVariableDeclaration(node)- Hoists a variable declaration to the tope of the containing scope
This is a special property that is available when writing a Program transformer. We will cover this kind of transformer in Types of transformers. It contains metadata about the entire program, such as (this is not an exhaustive list, just the stuff we care about):
getRootFileNames()- get an array of file names in the projectgetSourceFiles()- gets allSourceFiles in the projectgetCompilerOptions()- compiler options from thetsconfig.json, command line, or other (can also get it fromcontext)getSourceFile(fileName: string)- gets aSourceFileusing itsfileNamegetSourceFileByPath(path: Path)- gets aSourceFileusing itspathgetCurrentDirectory()- gets the current directory stringgetTypeChecker()- gets ahold of the type checker, useful when doing things with Symbols
This is the result of calling program.getTypeChecker().
It has a lot of interesting things on in that we'll be interested in when writing transformers.
getSymbolAtLocation(node)- useful for getting the symbol of a nodegetExportsOfModule(symbol)- will return the exports of a module symbol
It's the part we've all be waiting for! Let's write out first transformer.
First let's import typescript.
import * as ts from 'typescript';It's going to contain everything that we could use when writing a transformer.
Next let's create a default export that is going to be our transformer,
our initial transformer we be a transformer factory (because this gives us access to context) -
we'll go into the other kinds of transformers later.
import * as ts from 'typescript';
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
// transformation code here
};
};
export default transformer;Because we're using TypeScript to write out transformer -
we get all the type safety and more importantly intellisense!
If you're up to here you'll notice TypeScript complaining that we aren't returning a SourceFile -
let's fix that.
import * as ts from "typescript";
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
// transformation code here
+ return sourceFile;
};
};
export default transformer;Sweet we fixed the type error!
For our first transformer we'll take a hint from the Babel Handbook and rename some identifiers.
Here's our source code:
babel === plugins;Let's write a visitor function,
remember that a visitor function should take a node,
and then return a node.
import * as ts from 'typescript';
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
+ const visitor = (node: ts.Node): ts.Node => {
+ return node;
+ };
+
+ return ts.visitNode(sourceFile, visitor);
-
- return sourceFile;
};
};
export default transformer;Okay that will visit the SourceFile...
and then just immediately return it.
That's a bit useless -
let's make sure we visit every node!
import * as ts from 'typescript';
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
const visitor = (node: ts.Node): ts.Node => {
- return node;
+ return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};
};
export default transformer;Now let's find identifiers so we can rename them:
import * as ts from 'typescript';
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
const visitor = (node: ts.Node): ts.Node => {
+ if (ts.isIdentifier(node)) {
+ // transform here
+ }
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};
};
export default transformer;And then let's target the specific identifiers we're interested in:
import * as ts from 'typescript';
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
const visitor = (node: ts.Node): ts.Node => {
if (ts.isIdentifier(node)) {
+ switch (node.escapedText) {
+ case 'babel':
+ // rename babel
+
+ case 'plugins':
+ // rename plugins
+ }
}
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};
};
export default transformer;And then let's return new nodes that have been renamed!
import * as ts from 'typescript';
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
const visitor = (node: ts.Node): ts.Node => {
if (ts.isIdentifier(node)) {
switch (node.escapedText) {
case 'babel':
+ return ts.createIdentifier('typescript');
case 'plugins':
+ return ts.createIdentifier('transformers');
}
}
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};
};
export default transformer;Sweet! When ran over our source code we get this output:
typescript === transformers;Tip - You can see the source for this at /example-transformers/my-first-transformer - if wanting to run locally you can run it via
yarn build my-first-transformer.
All transformers end up returning the TransformerFactory type signature.
These types of transformers are taken from ttypescript.
Also known as raw,
this is the same as the one used in writing your first transformer.
// ts.TransformerFactory
(context: ts.TransformationContext) => (sourceFile: ts.SourceFile) => ts.SourceFile;When your transformer needs config that can be controlled by consumers.
(config?: YourPluginConfigInterface) => ts.TransformerFactory;When needing access to the program object this is the signature you should use,
it should return a TransformerFactory.
It also has configuration available as the second object,
supplied by consumers.
(program: ts.Program, config?: YourPluginConfigInterface) => ts.TransformerFactory;Amusingly TypeScript has no official support for consuming transformers via tsconfig.json.
There is a GitHub issue dedicated to talking about introducing something for it.
Regardless you can consume transformers it's just a little round-about.
This is the recommended approach! Hopefully in the future this can be officially supported in
typescript.
Essentially a wrapper over the top of the tsc CLI -
this gives first class support to transformers via the tsconfig.json.
It has typescript listed as a peer dependency so the theory is it isn't too brittle.
Install:
npm i ttypescript typescript -DAdd your transformer into the compiler options:
{
"compilerOptions": {
"plugins": [{ "transform": "my-first-transformer" }]
}
}Run ttsc:
ttscttypescript supports tsc CLI,
Webpack,
Parcel,
Rollup,
Jest,
& VSCode.
Everything we would want to use TBH.
Using either awesome-typescript-loader or ts-loader you can either use the getCustomTransformers() option (they have the same signature) or you can use ttypescript:
{
test: /\.(ts|tsx)$/,
loader: require.resolve('awesome-typescript-loader'),
// or
loader: require.resolve('ts-loader'),
options: {
compiler: 'ttypescript' // recommended, allows you to define transformers in tsconfig.json
// or
getCustomTransformers: program => {
before: [yourBeforeTransformer(program, { customConfig: true })],
after: [yourAfterTransformer(program, { customConfig: true })],
}
}
}Use ttypescript with the parcel-plugin-ttypescript plugin.
See: https://github.com/cevek/ttypescript#parcel
There is a wide variety of helper methods that can assert what type a node is.
When they return true they will narrow the type of the node,
potentially giving you extra properties & methods based on the type.
Tip - Abuse intellisense to interrogate the
tsimport for methods you can use, as well as TypeScript AST Viewer to know what type a node is.
import * as ts from 'typescript';
const visitor = (node: ts.Node): ts.Node => {
if (ts.isJsxAttribute(node.parent)) {
// node.parent is a jsx attribute
// ...
}
};Identifiers are created by the parser and are always unique.
Say, if you create a variable foo and use it in another line, it will create 2 separate identifiers with the same text foo.
Then, the linker runs through these identifiers and connects the identifiers referring to the same variable with a common symbol (while considering scope and shadowing). Think of symbols as what we intuitively think as variables.
So, to check if two identifiers refer to the same symbol - just get the symbols related to the identifier and check if they are the same (by reference).
Short example -
const symbol1 = typeChecker.getSymbolAtLocation(node1);
const symbol2 = typeChecker.getSymbolAtLocation(node2);
symbol1 === symbol2; // check by referenceFull example -
This will log all repeating symbols.
import * as ts from 'typescript';
const transformerProgram = (program: ts.Program) => {
const typeChecker = program.getTypeChecker();
// Create array of found symbols
const foundSymbols = new Array<ts.Symbol>();
const transformerFactory: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
const visitor = (node: ts.Node): ts.Node => {
if (ts.isIdentifier(node)) {
const relatedSymbol = typeChecker.getSymbolAtLocation(node);
// Check if array already contains same symbol - check by reference
if (foundSymbols.includes(relatedSymbol)) {
const foundIndex = foundSymbols.indexOf(relatedSymbol);
console.log(
`Found existing symbol at position = ${foundIndex} and name = "${relatedSymbol.name}"`
);
} else {
// If not found, Add it to array
foundSymbols.push(relatedSymbol);
console.log(
`Found new symbol with name = "${
relatedSymbol.name
}". Added at positon = ${foundSymbols.length - 1}`
);
}
return node;
}
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};
};
return transformerFactory;
};
export default transformerProgram;Tip - You can see the source for this at /example-transformers/match-identifier-by-symbol - if wanting to run locally you can run it via
yarn build match-identifier-by-symbol.
While there doesn't exist an out of the box method you can basically roll your own. Given a node:
const findParent = (node: ts.Node, predicate: (node: ts.Node) => boolean) => {
if (!node.parent) {
return undefined;
}
if (predicate(node.parent)) {
return node.parent;
}
return findParent(node.parent, predicate);
};
const visitor = (node: ts.Node): ts.Node => {
if (ts.isStringLiteral(node)) {
const parent = findParent(node, ts.isFunctionDeclaration);
if (parent) {
console.log('string literal has a function declaration parent');
}
return node;
}
};Will log to console string literal has a function declaration parent with the following source:
function hello() {
if (true) {
'world';
}
}- Be careful when traversing after replacing a node with another -
parentmay not be set. If you need to traverse after transforming make sure to setparenton the node yourself.
Tip - You can see the source for this at /example-transformers/find-parent - if wanting to run locally you can run it via
yarn build find-parent.
In the visitor function you can return early instead of continuing down children, so for example if we hit a node and we know we don't need to go any further:
const visitor = (node: ts.Node): ts.Node => {
if (ts.isArrowFunction(node)) {
// return early
return node;
}
};if (ts.isVariableDeclaration(node)) {
return ts.updateVariableDeclaration(node, node.name, node.type, ts.createStringLiteral('world'));
}-const hello = true;
+const hello = "updated-world";Tip - You can see the source for this at /example-transformers/update-node - if wanting to run locally you can run it via
yarn build update-node.
Alternatively we can mutate the node via getMutableClone(node) FYI there is a bug in ts-loader that makes this not work well,
strong advise for now is to NOT use this:
if (ts.isVariableDeclaration(node)) {
const newNode = ts.getMutableClone(node) as ts.VariableDeclaration;
newNode.initializer = ts.createStringLiteral('mutable-world');
return newNode;
}-const hello = true;
+const hello = "mutable-world";Tip - You can see the source for this at /example-transformers/update-mutable-node - if wanting to run locally you can run it via
yarn build update-mutable-node.
You'll notice that you can't mutate unless you getMutableClone -
this is by design.
Maybe instead of updating a node we want to completely change it. We can do that by just returning... a completely new node!
if (ts.isFunctionDeclaration(node)) {
// Will replace any function it finds with an arrow function.
return ts.createVariableDeclarationList(
[
ts.createVariableDeclaration(
ts.createIdentifier(node.name.escapedText),
undefined,
ts.createArrowFunction(
undefined,
undefined,
[],
undefined,
ts.createToken(ts.SyntaxKind.EqualsGreaterThanToken),
ts.createBlock([], false)
)
),
],
ts.NodeFlags.Const
);
}-function helloWorld() {}
+const helloWorld = () => {};Tip - You can see the source for this at /example-transformers/replace-node - if wanting to run locally you can run it via
yarn build replace-node.
Interestingly, a visitor function can also return a array of nodes instead of just one node. That means, even though it gets one node as input, it can return multiple nodes which replaces that input node.
export type Visitor = (node: Node) => VisitResult<Node>;
export type VisitResult<T extends Node> = T | T[] | undefined;Let's just replace every expression statement with two copies of the same statement (duplicating it) -
const transformer: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
const visitor = (node: ts.Node): ts.VisitResult<ts.Node> => {
// If it is a expression statement,
if (ts.isExpressionStatement(node)) {
// Return it twice.
// Effectively duplicating the statement
return [node, node];
}
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};
};So,
let a = 1;
a = 2;becomes
let a = 1;
a = 2;
a = 2;Tip - You can see the source for this at /example-transformers/return-multiple-node - if wanting to run locally you can run it via
yarn build return-multiple-node.
The declaration statement (first line) is ignored as it's not a ExpressionStatement.
Note - Make sure that what you are trying to do actually makes sense in the AST. For ex., returning two expressions instead of one is often just invalid.
Say there is a assignment expression (BinaryExpression with with EqualToken operator), a = b = 2. Now returning two nodes instead of b = 2 expression is invalid (because right hand side can not be multiple nodes). So, TS will throw an error - Debug Failure. False expression: Too many nodes written to output.
This is effectively same as the previous section. Just return a array of nodes including itself and other sibling nodes.
What if you don't want a specific node anymore?
Return an undefined!
if (ts.isImportDeclaration(node)) {
// Will remove all import declarations
return undefined;
}import lodash from 'lodash';
-import lodash from 'lodash';Tip - You can see the source for this at /example-transformers/remove-node - if wanting to run locally you can run it via
yarn build remove-node.
Sometimes your transformation will need some runtime part, for that you can add your own import declaration.
ts.updateSourceFileNode(sourceFile, [
ts.createImportDeclaration(
/* decorators */ undefined,
/* modifiers */ undefined,
ts.createImportClause(
ts.createIdentifier('DefaultImport'),
ts.createNamedImports([
ts.createImportSpecifier(undefined, ts.createIdentifier('namedImport')),
])
),
ts.createLiteral('package')
),
// Ensures the rest of the source files statements are still defined.
...sourceFile.statements,
]);+import DefaultImport, { namedImport } from "package";Tip - You can see the source for this at /example-transformers/add-import-declaration - if wanting to run locally you can run it via
yarn build add-import-declaration.
Sometimes you may want to push a VariableDeclaration so you can assign to it.
Remember that this only hoists the variable -
the assignment will still be where it was in the source.
if (ts.isVariableDeclaration(node) && ts.isIdentifier(node.name)) {
context.hoistVariableDeclaration(node.name);
return node;
}function functionOne() {
+ var innerOne;
+ var innerTwo;
const innerOne = true;
const innerTwo = true;
}Tip - You can see the source for this at /example-transformers/hoist-variable-declaration - if wanting to run locally you can run it via
yarn build hoist-variable-declaration.
You can also do this with function declarations:
if (ts.isFunctionDeclaration(node)) {
context.hoistFunctionDeclaration(node);
return node;
}+function functionOne() {
+ console.log('hello, world!');
+}
if (true) {
function functionOne() {
console.log('hello, world!');
}
}Tip - You can see the source for this at /example-transformers/hoist-function-declaration - if wanting to run locally you can run it via
yarn build hoist-function-declaration.
TODO - Is this possible?
TODO - Is this possible?
Sometimes you want to add a new variable that has a unique name within its scope, luckily it's possible without needing to go through any hoops.
if (ts.isVariableDeclarationList(node)) {
return ts.updateVariableDeclarationList(node, [
...node.declarations,
ts.createVariableDeclaration(
ts.createUniqueName('hello'),
undefined,
ts.createStringLiteral('world')
),
]);
}
return ts.visitEachChild(node, visitor, context);-const hello = 'world';
+const hello = 'world', hello_1 = "world";Tip - You can see the source for this at /example-transformers/create-unique-name - if wanting to run locally you can run it via
yarn build create-unique-name.
TODO - Is this possible in a concise way?
sourceFile.getLineAndCharacterOfPosition(node.getStart());
TODO - Is this possible?
It's possible!
// We need to use a Program transformer to get ahold of the program object.
const transformerProgram = (program: ts.Program) => {
const transformerFactory: ts.TransformerFactory<ts.SourceFile> = context => {
return sourceFile => {
const visitor = (node: ts.Node): ts.Node => {
if (ts.isImportDeclaration(node) && ts.isStringLiteral(node.moduleSpecifier)) {
const typeChecker = program.getTypeChecker();
const importSymbol = typeChecker.getSymbolAtLocation(node.moduleSpecifier);
const exportSymbols = typeChecker.getExportsOfModule(importSymbol);
exportSymbols.forEach(symbol =>
console.log(
`found "${
symbol.escapedName
}" export with value "${symbol.valueDeclaration.getText()}"`
)
);
return node;
}
return ts.visitEachChild(node, visitor, context);
};
return ts.visitNode(sourceFile, visitor);
};
};
return transformerFactory;
};Which will log this to the console:
found "hello" export with value "hello = 'world'"
found "default" export with value "export default 'hello';"
You can also traverse the imported node as well using ts.visitChild and the like.
Tip - You can see the source for this at /example-transformers/follow-imports - if wanting to run locally you can run it via
yarn build follow-imports.
Like following TypeScript imports for the code that you own, sometimes we may want to also interrogate the code inside a module we're importing.
Using the same code above except running on a node_modules import we get this logged to the console:
found "mixin" export with value:
export declare function mixin(): {
color: string;
};"
found "constMixin" export with value:
export declare function constMixin(): {
color: 'blue';
};"
Hmm what - we're getting the type def AST instead of source code... Lame!
So it turns out it's a little harder for us to get this working (at least out of the box). It turns out we have two options :
- Turn on
allowJsin the tsconfig and the delete the type def... which will give us the source AST... but we now won't have type defs... So this isn't desirable. - Create another TS program and do the dirty work ourselves
Spoiler: We're going with option 2. It's more resilient and will work when type checking is turned off - which is also how we'll follow TypeScript imports in that scenario!
const visitor = (node: ts.Node): ts.Node => {
if (ts.isImportDeclaration(node) && ts.isStringLiteral(node.moduleSpecifier)) {
// Find the import location in the file system using require.resolve
const pkgEntry = require.resolve(`${node.moduleSpecifier.text}`);
// Create another program
const innerProgram = ts.createProgram([pkgEntry], {
// Important to set this to true!
allowJs: true,
});
console.log(innerProgram.getSourceFile(pkgEntry).getText());
return node;
}
return ts.visitEachChild(node, visitor, context);
};Which will log this to the console:
export function mixin() {
return { color: 'red' };
}
export function constMixin() {
return { color: 'blue' }
}
Awesome! The cool thing about this btw is that since we've made a program we will get all of its imports followed for free! However it'll have the same problem as above if they have type defs - so watch out if you need to jump through multiple imports - you'll probably have to do something more clever.
Tip - You can see the source for this at /example-transformers/follow-node-modules-imports - if wanting to run locally you can run it via
yarn build follow-node-modules-imports.
TypeScript can also transform JSX - there are a handful of helper methods to get started. All previous methods of visiting and manipulation apply.
ts.isJsxXyz(node)ts.updateJsxXyz(node, ...)ts.createJsxXyz(...)
Interrogate the typescript import for more details. The primary point is you need to create valid JSX - however if you ensure the types are valid in your transformer it's very hard to get it wrong.
Useful when wanting to know what the file pragma is so you can do something in your transform.
Say for example we wanted to know if a custom jsx pragma is being used:
const transformer = sourceFile => {
const jsxPragma = sourceFile.pragmas.get('jsx');
if (jsxPragma) {
console.log(`a jsx pragma was found using the factory "${jsxPragma.arguments.factory}"`);
}
return sourceFile;
};The source file below would cause 'a jsx pragma was found using the factory "jsx"' to be logged to console.
/** @jsx jsx */Tip - You can see the source for this at /example-transformers/pragma-check - if wanting to run locally you can run it via
yarn build pragma-check.
Currently as of 29/12/2019 pragmas is not on the typings for sourceFile -
so you'll have to cast it to any to gain access to it.
Sometimes during transformation you might want to change the pragma back to the default (in our case React). I've found success with the following code:
const transformer = sourceFile => {
sourceFile.pragmas.clear();
delete sourceFile.localJsxFactory;
};If you're like me sometimes you want to split your big transformer up into small more maintainable pieces. Well luckily with a bit of coding elbow grease we can achieve this:
const transformers = [...];
function transformer(
program: ts.Program,
): ts.TransformerFactory<ts.SourceFile> {
return context => {
const initializedTransformers = transformers.map(transformer => transformer(program)(context));
return sourceFile => {
return initializedTransformers.reduce((source, transformer) => {
return transformer(source);
}, sourceFile);
};
};
}TODO - Is this possible like it is in Babel? Or we use a language service plugin?
Generally with transformers the the usefulness of unit tests is quite limited. I recommend writing integration tests to allow your tests to be super useful and resilient. This boils down to:
- Write integration tests over unit tests
- Avoid snapshot tests - only do it if it makes sense - the larger the snapshot the less useful it is
- Try to pick apart specific behavior for every test you write - and only assert one thing per test
If you want you can use the TypeScript compiler API to setup your transformer for testing, but I'd recommend using a library instead.
This library makes testing transformers easy.
It is made to be used in conjunction with a test runner such as jest.
It simplifies the setup of your transformer,
but still allows you to write your tests as you would for any other piece of software.
Here's an example test using it:
import { Transformer } from 'ts-transformer-testing-library';
import transformerFactory from '../index';
import pkg from '../../../../package.json';
const transformer = new Transformer()
.addTransformer(transformerFactory)
.addMock({ name: pkg.name, content: `export const jsx: any = () => null` })
.addMock({
name: 'react',
content: `export default {} as any; export const useState = {} as any;`,
})
.setFilePath('/index.tsx');
it('should add react default import if it only has named imports', () => {
const actual = transformer.transform(`
/** @jsx jsx */
import { useState } from 'react';
import { jsx } from '${pkg.name}';
<div css={{}}>hello world</div>
`);
// We are also using `jest-extended` here to add extra matchers to the jest object.
expect(actual).toIncludeRepeated('import React, { useState } from "react"', 1);
});EmitResolver cannot handle JsxOpeningLikeElement and JsxOpeningFragment that didn't originate from the parse tree
If you replace a node with a new jsx element like this:
const visitor = node => {
return ts.createJsxFragment(ts.createJsxOpeningFragment(), [], ts.createJsxJsxClosingFragment());
};It will blow up if there are any surrounding const or let variables.
A work around is to ensure the opening/closing elements are passed into ts.setOriginalNode:
ts.createJsxFragment(
- ts.createJsxOpeningFragment(),
+ ts.setOriginalNode(ts.createJsxOpeningFragment(), node),
[],
- ts.createJsxJsxClosingFragment()
+ ts.setOriginalNode(ts.createJsxJsxClosingFragment(), node)
);See microsoft/TypeScript#35686 for more information.
There seems to be a problem with ts-loader where it causes type checking to be triggered a second time when using getMutableClone(node) -
this leads to undefined behavior in transformers and generally ends up having it blow up.
Strong advice to steer clear of this method for now.
See: {tbd raised issue here}