Using ts-rs bindings in TypeScript

js
import typescript from "@rollup/plugin-typescript";
import tsRsBundler from "./lib/ts-rs-bundler.js";

export default {
  input: "index.ts",
  output: { dir: "dist", format: "es", sourcemap: true },
  plugins: [
    tsRsBundler(),
    typescript(),
  ],
};

js
import path from "node:path";
import { glob } from "node:fs/promises";

export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  const circularDependencyAvoider = new Map();

  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) {
      const [protocol, rawPath] = id.split(":");
      if (protocol === magicProtocol) {
        const importPath = path.resolve(path.dirname(importer), rawPath);
        if (!circularDependencyAvoider.has(importPath)) {
          circularDependencyAvoider.set(importPath, new Set());
        }
        circularDependencyAvoider.get(importPath).add(importer);
        return `${magicProtocol}:${importPath}`;
      }
      return null;
    },
    async load(id) {
      const [protocol, rawPath] = id.split(":");
      if (protocol === magicProtocol) {
        const paths = [];
        for await (const entry of glob(rawPath)) {
          if (!circularDependencyAvoider.get(rawPath)?.has(entry)) {
            paths.push(entry);
          } else {
            console.warn(
              `Circular dependency detected: ${entry} -> ${rawPath}`
            );
          }
        }
        const res = paths.map((path) => `export * from "${path}";\n`).join("");
        this.emitFile({
          type: "asset",
          fileName: "index.d.ts",
          source: res,
        });
        return res;
      }
      return null;
    },
  };
}

json
{
  "compilerOptions": {
    "outDir": "./dist",
    "declaration": true,
  }
}

ts
export * from "ts-rs-bundler:./*.ts";

I'm currently creating a small photo management software on the side and for that I'm writing the backend in Rust using the Axum framework. The frontend is written in TypeScript and Lit.

The problem of building an API surface

When building one project in two languages a bug problem is, that you want to define a common interface which both sides use to interact. That way you can have type hints on both sides and you "only" need to check your types against that common interface for compatibility. It also makes it easier to swap or mock either side. Most commonly you'd use something like OpenAPI (formerly known as Swagger) to define your API and at some point I will probably switch over to that, but right now it seems to be more effort from the Rust side, because there is no great tool that generates an OpenAPI spec for Axum at build/test time. Things like utoipa or aide exist, but they seem like too much overhead at this stage of the project.

Generating types

I selected to use the ts-rs crate to generate TypeScript types from my Rust types which works pretty good. It will generate a bindings folder next to you target folder which holds your types as *.ts files. These look like this:

ts
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.

export type User = { id: string, username: string, is_admin: boolean, };

Example project structure

- bindings/
  - event.ts
  - user.ts
  - ...
- frontend/
  - src/
    - ...
  - package.json
  - ...
- src/
  - main.rs
  - ...
- target/
- ...
- cargo.toml

How do you use the generated types?

This was the question I asked myself after generating the first set of bindings. Fundamentally I knew some answers and how to get something running, but I wanted a "clean" solution, so something I wouldn't need to touch manually or copy files manually or something like that. Ideally it should run automatically or at least with just one build command.

Getting a baseline

My idea was to make the bindings into their own npm package. That way I wouldn't need to touch the generated files. To achieve this I started off by creating a package.json and adding the correct fields to export some dist folder and switching to ESM by default. The rest is at this stage the default of npm init -y.

json
{
  // ...
  "main": "dist/index.js",
  "files": [
    "dist/*"
  ],
  "type": "module",
  // ...
}

Now we need to fill this dist folder. For this I added TypeScript and Rollup to the package:

json
{
  // ...
  "scripts": {
    "build": "rollup -c",
    "build:watch": "rollup -c --watch"
  },
  "devDependencies": {
    "@rollup/plugin-typescript": "^11.1.6",
    "rollup": "^4.17.2",
    "tslib": "^2.6.2",
    "typescript": "^5.4.5"
  }
  // ...
}

The dev dependencies can be installed via npm i -D rollup tslib typescript @rollup/plugin-typescript.

Now only two things were missing - a rollup.config.js

js
import typescript from "@rollup/plugin-typescript";

export default {
  input: "index.ts",
  output: { dir: "dist", format: "es", sourcemap: true },
  plugins: [
    typescript(),
  ],
};

...and an index.ts

ts
export * from "./event.ts";
export * from "./user.ts";
// ...

Using it in the frontend

Now we can use it in our frontend by importing it directly as a path:

json
{
  "devDependencies": {
    // ...
    "api-types": "../bindings" 
  }
}

The problem with this setup is, that every time I add, rename or delete an API type, I'd have to manually update the index.ts. This is not only cumbersome, but also invites mistakes. So let's to it better...

Generating index.ts

Now to the part why I'm writing this blogpost:

Rollup plugins are not magic

One of the big reasons I use rollup by default is, that it's just so easy to create plugins for it (I've done so many times before).
Let's start out with a basic plugin that replaces every call to a special import with a console.log("magic");.

The basic plugin might look like this:

js
export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) {
      const [protocol, message] = id.split(":");
      if (protocol === magicProtocol) {
        return id;
      }
      return null;
    },
    async load(id) {
      const [protocol, message] = id.split(":");
      if (protocol === magicProtocol) {
        return `console.log("${message}");`;
      }
      return null;
    },
  };
}

Let's go over this quickly.

A pluguin

The following exports a function which generates one instance of our plugin. It is called from inside the rollup.config.js file and that way we can use its parameters to configure the plugin instance (a simple object). Here we use that to set the magicProtocol which will be used later on to know what to replace.

js
export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  return { /**/ };
}

The instance

The returned plugin instance consists of a name which is used for internal logging and hooks that get called by rollup during the build.
The two hooke that are relevant to us are resolveId() for checking if a plugin wants to be involved with the loading of a module and load() which is used to actually load the code for a module.

js
export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  return { 
    name: "ts-rs-bundler",
    resolveId(id, importer) { /**/ },
    async load(id) { /**/ },
  };
}

Which magic do we want?

Before we take a closer look at the implementation of the hooks, let's think about how we want everything to work in the end.

Basically when doing something like this:

js
import x from "some-module.js";
import "ts-rs-bundler:magic";
console.log("hi");

We want the result to be something like:

js
import x from "some-module.js";
console.log("Hello magic");
console.log("hi");

So we want to replace a specific module with code we own and use the parameters from it.

Resolving modules

This is the resolveId function. It gets called for every module that rollup tries to resolve and using this we can tell rollup if we want to handle that module further by returning a string or null, if we don't want to handle it (simplified, full explanation is in the rollup docs). Let's jump into the code:

js
export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) { 
      const [protocol, message] = id.split(":");
      if (protocol === magicProtocol) {
        return id;
      }
      return null;
    },
    async load(id) { /**/ },
  };
}

We use the special character ":" here to split the path into a "protocol" we use in combination with the magicProtocol for detecting if this module is called and if the magicProtocol matches, we tell rollup by returning the id, that we want to handle this module.

Loading modules

Loading also gets fed every module id after it was already resolved by resolveId(). Here id is, what resolveId() returned. load() can return null and by that tell rollup to use another module for loading, or a string to actually load a module (or a Promise to either).
Our loading function looks like this:

js
export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) { /**/ },
    async load(id) { 
      const [protocol, message] = id.split(":");
      if (protocol === magicProtocol) {
        return `console.log("${message}");`;
      }
      return null;
    },
  };
}

And that's all. Next step is to apply this to our problem.

A solution in sight

Now that we have a working rollup plugin, let's do the actual implementation.

For this the idea is, that we want to find all relevant *.ts files and export all of their exports. To make this even more reusable, I chose to use the "message" part of our old plugin to provide a glob of files we want to import to the plugin. This means my complete index.ts file looks like this:

ts
export * from "ts-rs-bundler:./*.ts";

To get this to work, we need to change our implementation for resolveId and load.

Let's start with resolveId, which we need to change to actually resolve relative globs relative to the importing script. Luckily node's path.resolve() can help us here.

js
import path from "node:path";

export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) { 
      const [protocol, rawPath] = id.split(":");
      if (protocol === magicProtocol) {
        const importPath = path.resolve(path.dirname(importer), rawPath); 
        return `${magicProtocol}:${importPath}`;
      }
      return null;
    },
    async load(id) { /**/ },
  };
}

As you can see, we're now returning a different id from the resolver. That way it's absolute and we know where we have to search during loading. Speaking of, the loader is also not that hard:

js
import path from "node:path";
import { glob } from "node:fs/promises"; 

export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) { /**/ },
    async load(id) { 
      const [protocol, rawPath] = id.split(":");
      if (protocol === magicProtocol) {
        const paths = []; 
        for await (const entry of glob(rawPath)) {
          paths.push(entry);
        }
        return paths.map((path) => `export * from "${path}";\n`).join("");
      }
      return null;
    }, 
  };
}

Finally we also need to tell typescript, that we actually want to generate declarations for our types in the output by making this our tsconfig.json:

json
{
  "compilerOptions": {
    "outDir": "./dist",
    "declaration": true,
  }
}

If we now run a npm run build we get... Nothing.

Looking at the console output of the build, we can see that rollup detected a circular loop. This is, because index.ts itself matches *.ts relative to index.ts. That's an easy fix, we just have to ignore the importers of a glob. To do this, I introduced a simple mapping, which stores the paths that import each glob.

js
import path from "node:path";
import { glob } from "node:fs/promises";

export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  const circularDependencyAvoider = new Map(); 

  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) { 
      const [protocol, rawPath] = id.split(":");
      if (protocol === magicProtocol) {
        const importPath = path.resolve(path.dirname(importer), rawPath);
        if (!circularDependencyAvoider.has(importPath)) { 
          circularDependencyAvoider.set(importPath, new Set());
        }
        circularDependencyAvoider.get(importPath).add(importer);
        return `${magicProtocol}:${importPath}`;
      }
      return null;
    },
    async load(id) { 
      const [protocol, rawPath] = id.split(":");
      if (protocol === magicProtocol) {
        const paths = [];
        for await (const entry of glob(rawPath)) {
          if (!circularDependencyAvoider.get(rawPath)?.has(entry)) { 
            paths.push(entry);
          } else { 
            console.warn(`Circular dependency detected: ${entry} -> ${rawPath}`);
          }
        }
        const res = paths.map((path) => `export * from "${path}";\n`).join("");
        return res;
      }
      return null;
    },
  };
}

This resolves the circular dependency warning and prints our intentional warning from line 30 instead. Sadly this doesn't fix the issue, that this solution still doesn't work this way.

One last problem

Right now the generated output of a build is an empty dist/index.js file (which is fine, because we're only interested in the types) and this dist/index.d.ts:

ts
export * from "ts-rs-bundler:./*.ts";

But... that's exactly our input file and not what we loaded!

The issue is, that the rollup loaded version of the module is not used for the TypeScript compiler declaration process.

And one last fix

Gladly rollup once again rescues us. We can just emit the file with the wanted content during loading. This will create two warnings, because tsc will keep complaining that it can't load our magic module during declaration building and it will create a warning that we're emitting and overwriting a file already emitted by another module.
The fix is just adding this to our loading:

js
import path from "node:path";
import { glob } from "node:fs/promises";

export default function tsRsBundler({ magicProtocol = "ts-rs-bundler" } = {}) {
  const circularDependencyAvoider = new Map();

  return {
    name: "ts-rs-bundler",
    resolveId(id, importer) { /**/ },
    async load(id) { 
      const [protocol, rawPath] = id.split(":");
      if (protocol === magicProtocol) {
        const paths = [];
        for await (const entry of glob(rawPath)) {
          if (!circularDependencyAvoider.get(rawPath)?.has(entry)) {
            paths.push(entry);
          } else {
            console.warn(`Circular dependency detected: ${entry} -> ${rawPath}`);
          }
        }
        const res = paths.map((path) => `export * from "${path}";\n`).join("");
        this.emitFile({ 
          type: "asset",
          fileName: "index.d.ts",
          source: res,
        });
        return res;
      }
      return null;
    },
  };
}

Now we're done and a simple npm run build will generate everything we need and we can just import that module and use the generated bindings.

If you want everything you need to do in one place, just check the TLDR at the start of this post.