Polymod

i added pako fingerprint detection

turns out webpack arbitrarily decided to inline 1 threejs class into polytrack’s custom wrapper module???? i’m pretty sure the only way to account for this is by splicing out three.js at the declaration-level using a cross-module control flow graph so i’m now working on the necessary refactorings for that

as seen below, i basically need to fingerprint every single variable/function/class declaration across every module, instead of only checking exports of each

i re-refactored everything again and did a bunch more testing. turns out some three.js modules were being missed due to terser’s peephole optimizations on template expressions and string concatenation :(

the three.js detector can now fingerprint and detect the majority of three.js modules! still, many of the fingerprints don’t have enough entropy or got mangled by webpack/terser, so it doesn’t completely work.

as it turns out, the bundle still had a lot of undiscovered media files that needed new scanning infrastructure to detect. however, after that was written, i was able to get asset linking and js-sha version extraction working. unfortunately fingerprinting Three.js’s hundreds of modules down to a precise version is very hard so that is still not done.

full bundle->cjs->esm works now! some modules are dynamically loaded from other chunk files because Polytrack’s author enabled source splitting, so i restructured the driver code to support multi-bundle recursive module/chunk loading. also, i originally had the converter unwrap umd wrappers, but i realized that’s completely pointless because the entire module will get spliced out by npm detection, so it just marks such modules as “complex.” unbundling wise, the only thing left to write is npm package detection.

loader detection works now! i had to rewrite the entire unpacking library with custom semantic AST traversal to keep everything zero-copy without unsafe blocks

module splicing works now, so the webpack unbundler is able split the bundle into multiple commonjs chunks in <0.1s! next is to add webpack helper inlining, webpack loader detection (for non-js assets), and cjs->esm conversion.

static JS analysis is hard to do so all you get is a screenshot of code :(

i took a dump of Polytrack’s code and found that it is distributed in webpack bundles. so, i’m currently working on a rust script that uses oxc to parse an AST, simplify the bundle, and then split it into separate linked files.