Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix jest-haste-map-types #7951

Merged
merged 3 commits into from Feb 22, 2019
Merged

fix jest-haste-map-types #7951

merged 3 commits into from Feb 22, 2019

Conversation

jeysal
Copy link
Contributor

@jeysal jeysal commented Feb 21, 2019

Summary

jest-haste-map wants to do both export = HasteMap and multiple export type SomeType. Until we migrate it over to use a proper ESM default export, the way to do this in TypeScript is namespaces - see this StackOverflow answer. This fixes our jest-haste-map/build/index.d.ts:

Diff
diff --git a/tmp/old/index.d.ts b/tmp/new/index.d.ts
index ff4589f52..d2fe5bd75 100644
--- a/tmp/old/index.d.ts
+++ b/tmp/new/index.d.ts
@@ -4,9 +4,180 @@
  * This source code is licensed under the MIT license found in the
  * LICENSE file in the root directory of this source tree.
  */
+/// <reference types="node" />
+import EventEmitter from 'events';
+import { Config } from '@jest/types';
+import H from './constants';
 import HasteFS from './HasteFS';
 import HasteModuleMap, { SerializableModuleMap as HasteSerializableModuleMap } from './ModuleMap';
-export declare type ModuleMap = HasteModuleMap;
-export declare type SerializableModuleMap = HasteSerializableModuleMap;
-export declare type FS = HasteFS;
+import { HasteMap as HasteMapObject, HasteRegExp, InternalHasteMap, Mapper } from './types';
+declare type HType = typeof H;
+declare type Options = {
+    cacheDirectory?: string;
+    computeDependencies?: boolean;
+    computeSha1?: boolean;
+    console?: Console;
+    dependencyExtractor?: string;
+    extensions: Array<string>;
+    forceNodeFilesystemAPI?: boolean;
+    hasteImplModulePath?: string;
+    ignorePattern?: HasteRegExp;
+    mapper?: Mapper;
+    maxWorkers: number;
+    mocksPattern?: string;
+    name: string;
+    platforms: Array<string>;
+    providesModuleNodeModules?: Array<string>;
+    resetCache?: boolean;
+    retainAllFiles: boolean;
+    rootDir: string;
+    roots: Array<string>;
+    throwOnModuleCollision?: boolean;
+    useWatchman?: boolean;
+    watch?: boolean;
+};
+declare namespace HasteMap {
+    type ModuleMap = HasteModuleMap;
+    type SerializableModuleMap = HasteSerializableModuleMap;
+    type FS = HasteFS;
+}
+/**
+ * HasteMap is a JavaScript implementation of Facebook's haste module system.
+ *
+ * This implementation is inspired by https://github.com/facebook/node-haste
+ * and was built with for high-performance in large code repositories with
+ * hundreds of thousands of files. This implementation is scalable and provides
+ * predictable performance.
+ *
+ * Because the haste map creation and synchronization is critical to startup
+ * performance and most tasks are blocked by I/O this class makes heavy use of
+ * synchronous operations. It uses worker processes for parallelizing file
+ * access and metadata extraction.
+ *
+ * The data structures created by `jest-haste-map` can be used directly from the
+ * cache without further processing. The metadata objects in the `files` and
+ * `map` objects contain cross-references: a metadata object from one can look
+ * up the corresponding metadata object in the other map. Note that in most
+ * projects, the number of files will be greater than the number of haste
+ * modules one module can refer to many files based on platform extensions.
+ *
+ * type HasteMap = {
+ *   clocks: WatchmanClocks,
+ *   files: {[filepath: string]: FileMetaData},
+ *   map: {[id: string]: ModuleMapItem},
+ *   mocks: {[id: string]: string},
+ * }
+ *
+ * // Watchman clocks are used for query synchronization and file system deltas.
+ * type WatchmanClocks = {[filepath: string]: string};
+ *
+ * type FileMetaData = {
+ *   id: ?string, // used to look up module metadata objects in `map`.
+ *   mtime: number, // check for outdated files.
+ *   size: number, // size of the file in bytes.
+ *   visited: boolean, // whether the file has been parsed or not.
+ *   dependencies: Array<string>, // all relative dependencies of this file.
+ *   sha1: ?string, // SHA-1 of the file, if requested via options.
+ * };
+ *
+ * // Modules can be targeted to a specific platform based on the file name.
+ * // Example: platform.ios.js and Platform.android.js will both map to the same
+ * // `Platform` module. The platform should be specified during resolution.
+ * type ModuleMapItem = {[platform: string]: ModuleMetaData};
+ *
+ * //
+ * type ModuleMetaData = {
+ *   path: string, // the path to look up the file object in `files`.
+ *   type: string, // the module type (either `package` or `module`).
+ * };
+ *
+ * Note that the data structures described above are conceptual only. The actual
+ * implementation uses arrays and constant keys for metadata storage. Instead of
+ * `{id: 'flatMap', mtime: 3421, size: 42, visited: true, dependencies: []}` the real
+ * representation is similar to `['flatMap', 3421, 42, 1, []]` to save storage space
+ * and reduce parse and write time of a big JSON blob.
+ *
+ * The HasteMap is created as follows:
+ *  1. read data from the cache or create an empty structure.
+ *
+ *  2. crawl the file system.
+ *     * empty cache: crawl the entire file system.
+ *     * cache available:
+ *       * if watchman is available: get file system delta changes.
+ *       * if watchman is unavailable: crawl the entire file system.
+ *     * build metadata objects for every file. This builds the `files` part of
+ *       the `HasteMap`.
+ *
+ *  3. parse and extract metadata from changed files.
+ *     * this is done in parallel over worker processes to improve performance.
+ *     * the worst case is to parse all files.
+ *     * the best case is no file system access and retrieving all data from
+ *       the cache.
+ *     * the average case is a small number of changed files.
+ *
+ *  4. serialize the new `HasteMap` in a cache file.
+ *     Worker processes can directly access the cache through `HasteMap.read()`.
+ *
+ */
+declare class HasteMap extends EventEmitter {
+    private _buildPromise;
+    private _cachePath;
+    private _changeInterval?;
+    private _console;
+    private _options;
+    private _watchers;
+    private _whitelist;
+    private _worker;
+    constructor(options: Options);
+    static getCacheFilePath(tmpdir: Config.Path, name: string, ...extra: Array<string>): string;
+    getCacheFilePath(): string;
+    build(): Promise<HasteMapObject>;
+    /**
+     * 1. read data from the cache or create an empty structure.
+     */
+    read(): InternalHasteMap;
+    readModuleMap(): HasteModuleMap;
+    /**
+     * 2. crawl the file system.
+     */
+    private _buildFileMap;
+    /**
+     * 3. parse and extract metadata from changed files.
+     */
+    private _processFile;
+    private _buildHasteMap;
+    private _cleanup;
+    /**
+     * 4. serialize the new `HasteMap` in a cache file.
+     */
+    private _persist;
+    /**
+     * Creates workers or parses files and extracts metadata in-process.
+     */
+    private _getWorker;
+    private _crawl;
+    /**
+     * Watch mode
+     */
+    private _watch;
+    /**
+     * This function should be called when the file under `filePath` is removed
+     * or changed. When that happens, we want to figure out if that file was
+     * part of a group of files that had the same ID. If it was, we want to
+     * remove it from the group. Furthermore, if there is only one file
+     * remaining in the group, then we want to restore that single file as the
+     * correct resolution for its ID, and cleanup the duplicates index.
+     */
+    private _recoverDuplicates;
+    end(): Promise<void>;
+    /**
+     * Helpers
+     */
+    private _ignore;
+    private _isNodeModulesDir;
+    private _createEmptyMap;
+    static H: HType;
+    static ModuleMap: typeof HasteModuleMap;
+}
+export = HasteMap;
 //# sourceMappingURL=index.d.ts.map
\ No newline at end of file

@babel/plugin-transform-typescript does not support namespaces (because it's not possible to really support them), but we only want to put a few types in them, so I wrote a Babel plugin that strips them away in those simple cases. This way, yarn build works and jest-haste-map/build/index.js is unchanged.

Test plan

Manual inspection of jest-haste-map/build/index.{js,d.ts}

@@ -33,10 +33,10 @@ beforeEach(() => {
roots: ['./packages/jest-resolve-dependencies'],
});
return Runtime.createContext(config, {maxWorkers, watchman: false}).then(
(hasteMap: any) => {
(runtimeContext: any) => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was never actually a HasteMap instance lol, HasteMaps don't even have a .resolver property 😄

Copy link
Member

@SimenB SimenB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fantastic, thank you so much for fixing it!

@SimenB SimenB merged commit 4ea66e2 into jestjs:master Feb 22, 2019
@jeysal
Copy link
Contributor Author

jeysal commented Feb 22, 2019

:) do any other locations where we could get rid of any or ts-ignore come to your mind that I missed? I think most haste map usages are still in JS files

@SimenB
Copy link
Member

SimenB commented Feb 22, 2019

I think most haste map usages are still in JS files

Yeah, jest-runtime is the one I hit, and jest-runner and @jest/core are still JS. So I think we're good 🙂

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants