Skip to content

Decline proxy inlining that would duplicate an argument - #86

Open
ryoheinamiki515 wants to merge 3 commits into
ben-sb:mainfrom
ryoheinamiki515:decline-unsafe-proxy-inlining
Open

Decline proxy inlining that would duplicate an argument#86
ryoheinamiki515 wants to merge 3 commits into
ben-sb:mainfrom
ryoheinamiki515:decline-unsafe-proxy-inlining

Conversation

@ryoheinamiki515

@ryoheinamiki515 ryoheinamiki515 commented Sep 9, 2026

Copy link
Copy Markdown

Fixes the exponential inlining reported in #27, and the argument duplication it comes from.

The correctness bug

ProxyFunction.getReplacement pastes the argument node at every usage of its parameter. When the body uses a parameter more than once, the argument runs more than once:

function p(d, _) { return d << _ | d >>> 32 - _; }
var x = p(f(), 2);

1.1.7 returns var x = f() << 2 | f() >>> 30;. f now runs twice.

The growth

replaceProxyFunctionUsages recurses into each replacement, so a nested chain of such calls compounds the duplication. Input is p(p(...p(a,b)...,b),b) with the p above, under the default config.

Nesting depth Input 1.1.7 This branch
12 105 B 90,099 B / 71 ms 150 B / 5 ms
16 125 B 1,441,779 B / 830 ms 174 B / 1 ms
18 135 B 5,767,155 B / 3,628 ms 186 B / 1 ms
20 145 B 23,068,659 B / 16,667 ms 198 B / 1 ms
64 365 B no result after 2 minutes 462 B / 2 ms

Depth 64 is not synthetic. The common minified MD5 snippet chains 64 rounds as nested calls, f = md5_ii(f = md5_ii(f = md5_hh(.... Each round passes through md5_cmn and bit_rol, and bit_rol uses its first parameter twice.

One real page script that hits it: https://apg01.newzware.com/insights/common/insights.js, 15,555 bytes. On 1.1.7 it returns no result inside a 5-second deadline. On this branch it returns in 30 ms. #36 reports the same shape on a different script.

The change

ProxyFunction.isSafeToInline(args) declines the inline when a parameter that the body uses more than once receives an argument that is not safe to duplicate. ProxyRemover then leaves the call in place. Inlining of every other call is unchanged.

ProxyFunction.isDuplicable(node) states the precondition that getReplacement never checked. An argument is safe to duplicate when both hold.

  1. It evaluates with no side effect. A call, new, an assignment, ++, -- and delete each have one. A member expression can run a getter. A regular expression, an array, an object and a function each have their own identity, which a duplicate would split.
  2. It holds at most maxDuplicableNodes, which is 4.

Condition 1 alone stops the exponential chain, because a chain needs a nested call and a call is never duplicable. Condition 2 bounds the duplication whatever the pass order is. The confirmed trigger inlines a twice-used parameter to a 9 node expression, so the budget of 4 stops that expression from duplicating again.

A declined call still names the proxy function, so removeProxyFunctions must keep that declaration. retainedProxyFunctionIds records it, next to the existing cyclicProxyFunctionIds.

The pass order

ProxyRemover runs first, and ArrayUnpacker runs third, so a string-array argument still reads as a member expression when the guard sees it. That declines the dominant obfuscation shape:

var _0x1=['aa','bb'];
function p(d,_){return d<<_|d>>>32-_}
var x=p(_0x1[0],2);

This branch pushes a second ProxyRemover after the array unpacking, next to the second ExpressionSimplifier that already exists for the same reason. The argument then reaches the guard as the string 'aa', and it inlines. Output for the script above is var x = 0;, the same as 1.1.7.

The alias binding

findAliases removed an alias declaration such as var q = p; before any inline decision. A call that is not inlined then names an identifier that the output no longer declares. 1.1.7 already emits that shape for an aliased cyclic proxy function:

function a(x) { return b(x); }
function b(x) { return a(x); }
var c = a;
var z = c(1);

1.1.7 drops var c = a; and keeps var z = c(1);.

aliasDeclarations records each alias, and removeAliasDeclarations removes it after replacement, only for a proxy function whose calls were all inlined. The alias of a retained or cyclic proxy function stays.

Verification

tsc is clean. I ran the table above, the cases below, and the real script through the compiled output of this branch. I also extracted the 3,561-byte MD5 helper set from that script on its own. 1.1.7 returns no result for it inside a 10-second deadline. This branch returns 5,433 bytes in 22 ms.

Input Result
p(f(), 2) stays p(f(), 2), declaration kept
p(a, 2) inlines to a << 2 | a >>> 30
p(-1, 2) inlines. A negated literal is pure and small
p(a.b, 2) stays. A member read can run a getter
p(i++, 2) stays. ++ has a side effect
the string-array script above var x = 0;, same as 1.1.7
q(f()) with function q(d){return d+1} inlines to f() + 1
p(a, 2) and p(f(), 2) in one script first inlines, second stays, declaration kept
var q = p; q(f(), 2) stays, and var q = p; is kept
var q = p; q(a, 2) inlines, and var q = p; is removed
the aliased cyclic case above var c = a; is now kept

replaceProxyFunctionUsages inlines every proxy call. getReplacement pastes
the argument at each usage of its parameter, so a parameter the body uses
twice duplicates the argument. A nested chain of such calls grows
exponentially, and a side effect in the argument runs twice.

Inline such a call only when every multi-use parameter receives a literal,
an identifier or this. Keep the declaration of a proxy function whose call
was declined, so the remaining call still resolves.
findAliases removed an alias declaration such as var q = p before any
inline decision. A call that is not inlined then names an identifier that
the output no longer declares. 1.1.7 already emits that shape for an
aliased cyclic proxy function.

Record the alias declarations and remove them after replacement, and only
for a proxy function whose calls were all inlined.
The first version of this branch allowed six node types. That declined a
string-array read, which is the dominant obfuscation shape, because
ProxyRemover runs before ArrayUnpacker.

State the real precondition instead. An argument is safe to duplicate when
it evaluates with no side effect and holds at most 4 nodes. The purity half
stops the exponential chain, because a chain needs a nested call. The size
half bounds the duplication whatever the pass order is.

Run the proxy pass again after the array unpacking, next to the second
expression simplifier that exists for the same reason. A string-array
argument then reaches the guard as a literal, and it inlines.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant