Add CommonJS support as a plugin option

docs/generated_code.md

@@ -76,19 +76,14 @@ we generate `foo/bar_pb.js`.
 
 By default, we generate JavaScript _and_ TypeScript declaration files, so the generated
 code can be used in JavaScript or TypeScript projects without transpilation. If you
-prefer to generate TypeScript, use the plugin option `target=ts`.
-
-Note that we generate ECMAScript modules, which means we use `import` and `export` statements.
-All import paths include a `.js` extension, so you can use the generated code in Node.js
-with `"type": "module"` in your project's `package.json` without transpilation.
-If you do require support for the legacy CommonJS format, you can generate TypeScript and
-transpile it, for example with the extremely fast [esbuild](https://github.com/evanw/esbuild)
-bundler.
-
-It is also possible to modify the extension used in the import paths via the 
-[`import_extension`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#import_extensionjs) plugin option.  
-This option allows you to choose which extension will used in the imports, 
-providing flexibility for different environments.
+prefer to generate TypeScript, use the plugin option `[target=ts`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#target).
+
+By default, we generate ECMAScript modules, which means we use `import` and `export` statements. 
+If you need CommonJS, set the plugin option [`js_import_style=legacy_commonjs`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#js_import_style).
+
+All import paths include a `.js` extension by default. You can remove or change the
+extension via the [`import_extension`](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoc-gen-es#import_extensionjs) 
+plugin option.  
 
 
 ### Messages

docs/writing_plugins.md

@@ -97,7 +97,12 @@ In most cases, implementing the `generateTs` function only and letting the plugi
 As mentioned, if you do not provide a `generateJs` and/or a `generateDts` function and either `js` or `dts` is specified as a target out, the plugin framework will use its own TypeScript compiler to generate these files for you.  This process uses a stable version of TypeScript with lenient compiler options so that files are generated under most conditions.  However, if this is not sufficient, you also have the option of providing your own `transpile` function, which can be used to override the plugin framework's transpilation process.  
 
 ```ts
-transpile(fileInfo: FileInfo[], transpileJs: boolean, transpileDts: boolean): FileInfo[]
+transpile(
+  fileInfo: FileInfo[], 
+  transpileJs: boolean, 
+  transpileDts: boolean,
+  jsImportStyle: "module" | "legacy_commonjs",
+): FileInfo[]
 ```
 
 The function will be invoked with an array of `FileInfo` objects representing the TypeScript file content
@@ -247,13 +252,13 @@ function generateTs(schema: Schema) {
 
 ### Importing
 
-Generating import statements is accomplished via a combination of the `print` function and another function on the generated file object:  `import`.  The approach varies depending on the type of import you would like to generate.  
+Generating import statements is accomplished via a combination of the methods `import` and `print` on the generated file 
+object. 
 
 #### Importing from an NPM package
 
-To generate an import statement from an NPM package dependency, you first invoke the `import` function, passing the name of the import and the package in which it is located.
-
-For example, to import the `useEffect` hook from React:
+To import from an NPM package, you first invoke the `import` function, passing the name of the symbol to import, and the 
+package in which it is located. For example, to import the `useEffect` hook from React:
 
 ```ts
 const useEffect = f.import("useEffect", "react");
@@ -273,7 +278,7 @@ When the `ImportSymbol` is printed (and only when it is printed), an import stat
 
 #### Importing from `protoc-gen-es` generated code 
 
-Imports in this way work similarly.  Again, the `print` statement will automatically generate the import statement for you when invoked.
+To import a message or enumeration from `protoc-gen-es` generated code, you can simply pass the descriptor to `import()`:
 
 ```ts
 declare var someMessageDescriptor: DescMessage;
@@ -322,48 +327,44 @@ Note that some of the `ImportSymbol` types in the schema runtime (such as `JsonV
 
 The natural instinct would be to simply print your own import statements as `f.print("import { Foo } from 'bar'")`, but this is not the recommended approach.  Using `f.import()` has many advantages such as:
 
-- **Conditional imports**
-   - Import statements belong at the top of a file, but you usually only find out later whether you need the import, such as further in your code in a nested if statement.  Conditionally printing the import symbol will only generate the import statement when it is actually used.
+- **Conditional imports**: Import statements belong at the top of a file, but you usually only find out later whether you need the import, such as further in your code in a nested if statement.  Conditionally printing the import symbol will only generate the import statement when it is actually used.
 
-- **Preventing name collisions**
-    - For example if you `import { Foo } from "bar"`  and `import { Foo } from "baz"` , `f.import()` will automatically rename one of them `Foo$1`, preventing name collisions in your import statements and code.
+- **Preventing name collisions**: For example if you `import { Foo } from "bar"`  and `import { Foo } from "baz"` , `f.import()` will automatically rename one of them `Foo$1`, preventing name collisions in your import statements and code.
 
-- **Extensibility of import generation**
-    - Abstracting the generation of imports allows the library to potentially offer other import styles in the future without affecting current users.
+- **Import styles**: If the plugin option `js_import_style=legacy_commonjs` is set, code is automatically generated
+  with `require()` calls instead of `import` statements.
 
 
 ### Exporting
 
-Working with exports is accomplished via the `export` function on the generated file object.  Let's walk through an example:
-
-Suppose you generate a validation function for every message.  If you have a nested message, such as:
+To export a declaration from your code, use `exportDecl`:
 
-```proto
-message Bar {
-   Foo foo = 1;
-}
+```typescript
+const name = "foo";
+f.exportDecl("const", name);
 ```
 
-You may want to import and use the validation function generated for message `Foo` when generating the code for message `Bar`.  To generate the validation function, you would use `export` as follows:
+This method takes two arguments:
+1. The declaration, for example `const`, `enum`, `abstract class`, or anything 
+   you might need.
+2. The name of the declaration, which is also used for the export.
 
-```ts
-const fn = f.export("validateFoo");
-f.print("function ", fn, "() {");
-f.print("   return true;");
-f.print("}");
+The return value of the method can be passed to `print`: 
+
+```typescript
+const name = "foo";
+f.print(f.exportDecl("const", name), " = 123;");
 ```
 
-Note that `export` returns an `ImportSymbol` that can then be used by another dependency.  The trick is to store this `ImportSymbol` and use it when you generate the validation function for `Bar`.  Storing the symbol is as simple as putting it in a global map:
+The example above will generate the following code:
 
-```ts
-const exportMap = new Map<DescMessage, ImportSymbol>()
+```typescript
+export const foo = 123;
 ```
 
-That way, when you need to use it for `Bar`, you can simply access the map:
+If the plugin option `js_import_style=legacy_commonjs` is set, the example will
+automatically generate the correct export for CommonJS.
 
-```ts
-const fooValidationFn = exportMap.get(bar);  // bar is of type DescMessage
-```
 
 ### Parsing plugin options
 

packages/protoc-gen-es/README.md

@@ -98,12 +98,25 @@ By default, [protoc-gen-es](https://www.npmjs.com/package/@bufbuild/protoc-gen-e
 uses a `.js` file extensions in import paths, even in TypeScript files.
 
 This is unintuitive, but necessary for [ECMAScript modules in Node.js](https://www.typescriptlang.org/docs/handbook/esm-node.html).
-Unfortunately, not all bundlers and tools have caught up yet, and Deno 
-requires `.ts`. With this plugin option, you can replace `.js` extensions 
+Unfortunately, not all bundlers and tools have caught up yet, and Deno
+requires `.ts`. With this plugin option, you can replace `.js` extensions
 in import paths with the given value. For example, set
 
-- `import_extension=none` to remove the `.js` extension
-- `import_extension=.ts` to replace the `.js` extension with `.ts`
+- `import_extension=none` to remove the `.js` extension.
+- `import_extension=.ts` to replace the `.js` extension with `.ts`.
+
+### `js_import_style`
+
+By default, [protoc-gen-es](https://www.npmjs.com/package/@bufbuild/protoc-gen-es)
+(and all other plugins based on [@bufbuild/protoplugin](https://www.npmjs.com/package/@bufbuild/protoplugin))
+generate ECMAScript `import` and `export` statements. For use cases where 
+CommonJS is difficult to avoid, this option can be used to generate CommonJS 
+`require()` calls.
+
+Possible values:
+- `js_import_style=module` generate ECMAScript `import` / `export` statements - 
+  the default behavior.
+- `js_import_style=legacy_commonjs` generate CommonJS `require()` calls.
 
 
 ### `keep_empty_files=true`

packages/protoc-gen-es/src/javascript.ts

@@ -52,7 +52,7 @@ export function generateJs(schema: Schema) {
 function generateEnum(schema: Schema, f: GeneratedFile, enumeration: DescEnum) {
   const protoN = getNonEditionRuntime(schema, enumeration.file);
   f.print(makeJsDoc(enumeration));
-  f.print("export const ", enumeration, " = ", protoN, ".makeEnum(")
+  f.print(f.exportDecl("const", enumeration), " = ", protoN, ".makeEnum(")
   f.print(`  "`, enumeration.typeName, `",`)
   f.print(`  [`)
   if (enumeration.sharedPrefix === undefined) {
@@ -74,7 +74,7 @@ function generateEnum(schema: Schema, f: GeneratedFile, enumeration: DescEnum) {
 function generateMessage(schema: Schema, f: GeneratedFile, message: DescMessage) {
   const protoN = getNonEditionRuntime(schema, message.file);
   f.print(makeJsDoc(message));
-  f.print("export const ", message, " = ", protoN, ".makeMessageType(")
+  f.print(f.exportDecl("const", message), " = ", protoN, ".makeMessageType(")
   f.print(`  `, literalString(message.typeName), `,`)
   if (message.fields.length == 0) {
     f.print("  [],")

packages/protoc-gen-es/src/typescript.ts

@@ -53,7 +53,7 @@ export function generateTs(schema: Schema) {
 function generateEnum(schema: Schema, f: GeneratedFile, enumeration: DescEnum) {
   const protoN = getNonEditionRuntime(schema, enumeration.file);
   f.print(makeJsDoc(enumeration));
-  f.print("export enum ", enumeration, " {");
+  f.print(f.exportDecl("enum", enumeration), " {");
   for (const value of enumeration.values) {
     if (enumeration.values.indexOf(value) > 0) {
       f.print();
@@ -84,7 +84,7 @@ function generateMessage(schema: Schema, f: GeneratedFile, message: DescMessage)
     JsonValue
   } = schema.runtime;
   f.print(makeJsDoc(message));
-  f.print("export class ", message, " extends ", Message, "<", message, "> {");
+  f.print(f.exportDecl("class", message), " extends ", Message, "<", message, "> {");
   for (const member of message.members) {
     switch (member.kind) {
       case "oneof":

packages/protoplugin-example/src/protoc-gen-twirp-es.ts

@@ -16,13 +16,13 @@
 
 import { createEcmaScriptPlugin, runNodeJs } from "@bufbuild/protoplugin";
 import { version } from "../package.json";
+import type { Schema } from "@bufbuild/protoplugin/ecmascript";
 import {
   literalString,
-  makeJsDoc,
   localName,
+  makeJsDoc,
 } from "@bufbuild/protoplugin/ecmascript";
 import { MethodKind } from "@bufbuild/protobuf";
-import type { Schema } from "@bufbuild/protoplugin/ecmascript";
 
 const protocGenTwirpEs = createEcmaScriptPlugin({
   name: "protoc-gen-twirp-es",
@@ -39,19 +39,16 @@ function generateTs(schema: Schema) {
       Message,
       JsonValue
     } = schema.runtime;
-    // Convert the Message ImportSymbol to a type-only ImportSymbol
-    const MessageAsType = Message.toTypeOnly();
     for (const service of file.services) {
-      const localServiceName = localName(service);
       f.print(makeJsDoc(service));
-      f.print("export class ", localServiceName, "Client {");
+      f.print(f.exportDecl("class", localName(service) + "Client"), " {");
       f.print("    private baseUrl: string = '';");
       f.print();
       f.print("    constructor(url: string) {");
       f.print("        this.baseUrl = url;");
       f.print("    }");
       f.print();
-      f.print("    async request<T extends ", MessageAsType, "<T>>(");
+      f.print("    async request<T extends ", Message.toTypeOnly(), "<T>>(");
       f.print("        service: string,");
       f.print("        method: string,");
       f.print("        contentType: string,");

packages/protoplugin-test/src/byo-transpile.test.ts

@@ -0,0 +1,95 @@
+// Copyright 2021-2023 Buf Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import { describe, expect, test } from "@jest/globals";
+import { CodeGeneratorRequest } from "@bufbuild/protobuf";
+import { createEcmaScriptPlugin } from "@bufbuild/protoplugin";
+import type { FileInfo } from "@bufbuild/protoplugin/ecmascript";
+
+describe("bring your own transpile", () => {
+  test("does not transpile target=ts", () => {
+    const lines = testGenerate("target=ts");
+    expect(lines).toStrictEqual(["fake typescript source"]);
+  });
+
+  test("transpiles to target js", () => {
+    const lines = testGenerate("target=js");
+    expect(lines).toStrictEqual(["fake transpiled to js"]);
+  });
+
+  test("transpiles to target dts", () => {
+    const lines = testGenerate("target=dts");
+    expect(lines).toStrictEqual(["fake transpiled to dts"]);
+  });
+
+  function testGenerate(parameter: string): string[] {
+    const plugin = createEcmaScriptPlugin({
+      name: "test",
+      version: "v1",
+      generateTs: (schema) => {
+        const f = schema.generateFile("test.ts");
+        f.print("fake typescript source");
+      },
+      transpile: (files, transpileJs, transpileDts) => {
+        const out: FileInfo[] = [];
+        for (const file of files) {
+          if (transpileJs) {
+            switch (file.content) {
+              case "fake typescript source\n":
+                out.push({
+                  name: "test.js",
+                  preamble: file.preamble,
+                  content: "fake transpiled to js\n",
+                });
+                break;
+              default:
+                out.push({
+                  name: "test.js",
+                  preamble: file.preamble,
+                  content: "failed to transpile to js\n",
+                });
+                break;
+            }
+          }
+          if (transpileDts) {
+            switch (file.content) {
+              case "fake typescript source\n":
+                out.push({
+                  name: "test.d.ts",
+                  preamble: file.preamble,
+                  content: "fake transpiled to dts\n",
+                });
+                break;
+              default:
+                out.push({
+                  name: "test.js",
+                  preamble: file.preamble,
+                  content: "failed to transpile to js\n",
+                });
+                break;
+            }
+          }
+        }
+        return out;
+      },
+    });
+    const req = new CodeGeneratorRequest({
+      parameter,
+    });
+    const res = plugin.run(req);
+    expect(res.file.length).toBeGreaterThanOrEqual(1);
+    const content = res.file[0]?.content ?? "";
+    return content.trim().split("\n");
+  }
+});