Alephx support by lishawnl · Pull Request #2 · NorthernLabs/markets_ccxt

lishawnl · 2024-10-02T13:13:17Z

synced from upstream
added Alephx support in Typescript and transpiled to Python/Javascript/PHP
added examples and tested in Typescript and Python

…ge property descriptions (ccxt#27576) * Fix documentation for fetchMarkets return type (#1) Clarify that fetchMarkets returns an array of Market objects as defined by the Market structure, addressing issue ccxt#27491 which noted that the documentation mentioned 'Array of object' but didn't specify that the object is Market. * Clarify rateLimit documentation (#2) Update documentation to clearly explain what the rateLimit value represents: - It is the number of milliseconds to wait between consecutive requests - For example, if rateLimit is 1000, it means 1 request per second is allowed This addresses issue ccxt#24556 which noted that the documentation didn't clearly define what the rateLimit number means. * docs: add anchor link to market structure definition --------- Co-authored-by: Bryan Nuñez <149286981+cryptoganster@users.noreply.github.com> Co-authored-by: Igor Kroitor <igor.kroitor@gmail.com>

* feat(java): add typed wrapper layer, rate limiter, and type classes - Add 60 type classes in io.github.ccxt.types (Ticker, Trade, Order, OrderBook, OHLCV, MarketInterface, Position, etc.) matching C#/Go - Add ExchangeTyped wrapper (189 methods, sync + async) generated from TypeScript via build/generateJavaWrappers.ts - Add Throttler (leaky bucket + rolling window) wired into Exchange - Fix createSafeDictionary to return ConcurrentHashMap (thread safety) - Add unit tests (types, edge cases, throttler) and live integration tests (exception propagation, concurrency, rate limiting, Binance) * feat(java): add per-exchange typed wrappers (Binance, OKX, Poloniex) Generate per-exchange typed wrapper classes that extend ExchangeTyped and only expose methods each exchange actually implements: - Binance: 69 typed methods - OKX: 63 typed methods - Poloniex: 29 typed methods Users can now write: var binance = new Binance(); binance.loadMarkets(); Ticker t = binance.fetchTicker("BTC/USDT"); Changes: - Extend generateJavaWrappers.ts to scan exchange files and generate per-exchange wrappers in io.github.ccxt.wrappers - Change ExchangeTyped.exchange field from private to protected - Add example script (FetchOrderBooksExample) demonstrating usage * fix(java): fix verbose default and string comparison bug - Change verbose default from true to false (matches JS/Python/Go/C#) - Fix string reference comparison in Crypto.java ECDSA: `hash != "sha256"` → `!"sha256".equals(hash)` * feat(java): non-blocking I/O with httpClient.sendAsync and virtual threads Replace blocking httpClient.send() inside CompletableFuture.supplyAsync() with httpClient.sendAsync() for true non-blocking I/O. This eliminates thread exhaustion under high concurrency. Key changes: - fetch(): use sendAsync().thenApply() instead of supplyAsync+send() - fetch2(): use thenCompose chain instead of supplyAsync+join() - sleep(): use delayedExecutor instead of Thread.sleep() - request(): direct delegation to fetch2 (remove redundant wrapper) - loadMarkets/loadAccounts: thenCompose chains instead of blocking .get() - Add VIRTUAL_EXECUTOR for transpiled exchange methods - Add HTTP timeout (was missing entirely) - Fix GZIP decompression newline stripping (readAllBytes vs readLine) - Add ConcurrencyStressTest proving the fix * refactor(java): optimize Throttler — single lock per iteration, exact sleep, batch completion Rewrite leakyBucketLoop and rollingWindowLoop to match the quality of Go/C# implementations while keeping the same architecture (queue + loop + lock) used across all CCXT languages. - Reduce lock acquisitions from 5 per element to 1 per iteration - Replace 1ms busy-wait polling with exact computed sleep duration - Batch-complete all affordable requests per iteration - Complete futures outside the lock to reduce hold time - Fix fragile manual lock/unlock in rollingWindowLoop - Add timing and high-concurrency throttler tests * fix(java): add thread safety for shared mutable state in async contexts - Mark last_* debug fields as volatile (written from fetch/fetch2 async callbacks) - Mark market data fields as volatile (replaced atomically in setMarkets) - Mark loadMarkets flags as volatile (written from thenApply callback) - Convert options map to ConcurrentHashMap (mutated from async loadMarkets chain) - lastRestRequestTimestamp marked volatile (written from fetch2 thenCompose) * chore: remove generated exchange files (transpiled at build time) * feat(java): add WebSocket infrastructure with Netty Core WebSocket support matching the architecture of C#, Go, JS, Python: - ws/Future.java: Promise with explicit resolve/reject/race - ws/WsClient.java: Netty-based WebSocket client with: - Async connect/send via Netty NIO event loop - permessage-deflate compression support - GZIP/deflate binary frame decompression - Ping/pong keep-alive on virtual thread - SSL/TLS (WSS) and HTTP/SOCKS5 proxy support - Shared NioEventLoopGroup across all connections - Message dispatch offloaded to VIRTUAL_EXECUTOR - ws/OrderBookSide.java: Sorted price levels with O(log n) bisect - ws/WsOrderBook.java: Order book with snapshot+delta and cache - ws/ArrayCache.java: FIFO cache with dedup and newUpdates tracking - Client.java: Bridge extending WsClient for transpiled code - Exchange.java: watch(), watchMultiple(), client() factory, handleMessage(), ping(), onClose(), onError(), spawn(), orderBook factory methods - build.gradle.kts: Add Netty codec-http + handler-proxy deps * feat(java): enable WS exchange transpilation pipeline Enable the --ws flag in javaTranspiler.ts for transpiling WebSocket exchange classes from TypeScript pro/ sources: - Implement transpileWS() — reads exchanges.json, filters to exchanges with existing REST parents, transpiles to exchanges/pro/ - Add getJavaWsRegexes() — Java-specific regex transforms for WS code - Add getJavaImports() WS variant — ws package imports, FQN for parent - Fix class name collision (REST vs WS same name) via FQN extends - Auto-filter to only transpile exchanges with REST parents available - Add newException() to Exchange.java for dynamic exception construction Note: transpiled WS exchange code has ~100 type errors per exchange that need further transpiler regex work (watch/watchMultiple signatures, Object→int casts, Client vs WsClient types). The base infrastructure and transpiler pipeline are in place — the output polishing is iterative. * feat(java): Binance WS exchange compiles — transpiler fixes + manual patches Transpiler improvements (101 → 0 errors for Binance): - ArrayCache inner classes use FQN (ArrayCache.ArrayCacheByTimestamp etc) - Dynamic method dispatch via Helpers.callDynamically for Object-typed variables (.append, .reset, .limit, .storeArray, .store, .getLimit) - .cache/.nonce property access via Helpers.GetValue - Method references in subscriptions converted to string names - handler.call(this, args) → Helpers.callDynamically reflection dispatch - spawn(this.method, args) → spawn(() -> this.method(args)) lambda - this.delay() → spawn with Thread.sleep - CompletableFuture<Void> → CompletableFuture<Object> - void supplyAsync blocks get return null insertion - Client type consistency (client() returns Client, not WsClient) - Object→String assignments relaxed to Object type - watch/watchMultiple missing args filled with null Manual fixes for Binance.java: - return null in void async lambdas - Client casts for Helpers.GetValue results - Future.getFuture().join() for typed future access - ArrayCache cast for .hashmap access - final variable copies for lambda captures Also: add extend() alias for deepExtend(), newException() for dynamic exception construction, hashmap field made public in ArrayCache * test(java): add WebSocket unit tests and live test scaffold - FutureTest: resolve, reject, race, async resolve, double resolve - OrderBookSideTest: ascending asks, descending bids, update, delete, limit - ArrayCacheTest: append, eviction, newUpdates tracking, ByTimestamp, BySymbolById - LiveWsTest: watchTicker + watchTrades against Binance (gated by CCXT_LIVE_WS_TESTS env) * fix(java): address 20 quality audit issues across WS infrastructure Critical fixes: - Inflater resource leak: try-finally guarantees end() on exception - Future.race(): AtomicBoolean prevents concurrent double-resolution - Ping loop: try-catch around callback calls onError() + breaks loop - EventLoopGroup: shutdown hook for graceful cleanup on JVM exit - Reconnection: verified startedConnecting reset in onError() High-priority fixes: - watch() TOCTOU race: atomic putIfAbsent() replaces containsKey+put - Stale rejections: rejectionsMap cleared on resolve() to prevent new futures from inheriting old errors - ArrayCache thread safety: AtomicInteger for newUpdates, ConcurrentHashMap for newUpdatesBySymbol, synchronized append/getLimit - ArrayCache O(n) indexOf: position index HashMap for O(1) lookup in ArrayCacheByTimestamp and ArrayCacheBySymbolById Medium fixes: - cleanupWsClient(): clear() instead of unsafe keySet iteration - watch/watchMultiple: .exceptionally() handler on connect chain - Documentation for: BinaryFrame auto-release, limit() explicit call, watchMultiple single-message design, send() future handling Low fixes: - Verbose logging on connection lifecycle events - JSON serialization error logging * chore: remove generated WS exchange files (transpiled at build time) * fix(java): revert ConcurrentHashMap to HashMap for options and createSafeDictionary ConcurrentHashMap does not allow null keys or values, but transpiled TypeScript code maps `undefined` to `null` and stores it in these maps. This would cause NPE at runtime. Additionally, the `options` field was being overwritten with a plain HashMap in initializeProperties() anyway, silently losing the intended thread safety. HashMap is the correct choice here, matching what all other CCXT languages use (plain dict/map). * fix(java): sync WS infrastructure with latest fixes - Helpers.addElementToObject: reflection fallback for WsOrderBook field access - Future: no-arg resolve() overload - WsClient: reset() method, public onPong(), shutdown hook, ping loop exception handling, Inflater resource leak fix, Future.race AtomicBoolean - Exchange: loadOrderBook, crc32, isBinaryMessage, decodeProtoMsg stubs - Transpiler: full postProcessWsJava with all 77-exchange fix patterns - LiveWsTest: dynamic class loading (works without generated pro/ files) * fix(java): clean transpiler (no C# contamination) + addElementToObject reflection - Rebuild javaTranspiler.ts WS support without touching Java type conversion methods - Add Helpers.addElementToObject reflection fallback for WsOrderBook field access - Add watchOrderBook gradle task with env var passthrough - LiveWsTest uses dynamic class loading for pro/ independence Tested: 5/5 public + 2/4 private Binance WS endpoints passing (2 private failures are sandbox-specific, not infrastructure bugs) * fix(java): fix apiKey field access in WS API — was converting to string literal The method reference regex was too broad — it converted Binance.this.apiKey to the string "apiKey" instead of keeping it as field access. Restrict the regex to only match known callback method names (ping, negotiate, etc.), not field access like apiKey, secret, etc. Verified: createOrderWs now works end-to-end on Binance demo trading. * fix(java): fix callDynamically double-wrapping in WS transpiled code - Helpers.getArg: add null check on varargs array (fixes NPE when Java passes null for varargs, e.g. handleParamString(params, "timezone", null)) - javaTranspiler postProcessWsJava: add .join() to callDynamically calls used as return values or assignments inside supplyAsync lambdas, preventing CompletableFuture double-wrapping that broke watchOrderBook and watchOHLCV - Add BinanceDemoWsTest: comprehensive live WS test (12/12 pass on sandbox) - Add BinanceDemoRestTest: comprehensive REST test (17/17 pass on sandbox) - Add gradle wsTest/restTest tasks for running demo tests * fix(java): make callDynamically return Object (sync) like C# callDynamically was wrapping every reflective call in CompletableFuture, requiring .join() hacks in the transpiler for sync methods like append, store, limit, getLimit. Now returns Object directly — callers dispatching to async methods cast explicitly. Removes 19 lines of regex workarounds from javaTranspiler.ts. * fix(java): sync Binance upstream + switch tests to enableDemoTrading Update Binance exchange with new endpoints (algo orders, rpi depth, papiV2, conditional orders). Switch demo/rest tests from sandbox mode to enableDemoTrading for compatibility with demo.binance.com API keys. * feat(java): add Binance WS exchange, BybitApi, and WatchOrderBook example Re-add pro/Binance.java WS exchange (with callDynamically .join() removals applied), BybitApi typed wrapper, and WatchOrderBookExample test harness. * chore: remove generated/unrelated files from PR Revert go/v4/exchange_metadata.go, Binance.java, BinanceApi.java to pre-PR state and remove BybitApi.java — these are auto-generated and should not be part of this PR. * fix(java): restore HTTP proxy support in REST client initHttpClient() was changed to HttpClient.newHttpClient() with proxy logic commented out, breaking REST proxy support for all users. Restore the original proxy configuration using HttpClient.newBuilder() with ProxySelector, and fix the string comparison bug (was using != instead of .isEmpty() for empty string check). * Discard changes to go/tests/base/cache/cache.go * Discard changes to go/tests/base/cache/orderbook.go * fix(java): fix proxy init order — httpClient was built before proxy fields set initHttpClient() was called before initializeProperties(), so httpProxy/ httpsProxy were always null when the HttpClient was constructed. Move initHttpClient() after initializeProperties() so the proxy fields are populated before the HttpClient.Builder reads them. Add ProxyTest with 6 tests: no proxy, httpProxy config, httpsProxy config, conflicting proxy exceptions, proxyUrl+httpProxy conflict, and end-to-end proxy routing verification (dead proxy = connection refused, not bypassed). * test(java): add live proxy integration test for REST endpoints ProxyLiveTest fetches real Binance data (ticker, orderbook, trades) through an HTTP CONNECT proxy, verifying both httpProxy and httpsProxy configurations route traffic correctly. Requires a local proxy on port 18911 (e.g. tinyproxy, squid, or the included Python test proxy). Add gradle proxyLiveTest task. * fix main build in java * fix tests build * disable lighter * lighter helpers * fix(java): fix transpiler callDynamically cast, test file output bugs - Add CompletableFuture cast for callDynamically().join() calls across all 5 transpilation paths (base, exchanges, exchange tests, base tests, main tests) — fixes 44 compilation errors - Fix test file extension .cs → .java in transpileExchangeTestsToJava - Fix test file naming to use className (TestX) not finalName (testX) * fix(java): add missing Exchange.java method stubs + re-transpile base Hand-written stubs (Aftermath, Grvt exchanges need these): - binaryToBase64: delegate to Encode.binaryToBase64 - exceptionMessage: format exception with class name and stack - ethGetAddressFromPrivateKey: stub (throws UnsupportedOperationException) Transpiled section re-generated with callDynamically cast fix. * feat(java): add transpiled test files for CI build Base tests (TestInit, TestSafeMethods, etc.) and exchange tests (TestMain, TestSharedMethods, etc.) needed for CI — most lack AUTO_TRANSPILE_ENABLED so the transpiler doesn't regenerate them. * fix(java): exclude live tests from CI unit test run Tag ConcurrencyStressTest and ExchangeTypedTest as @Tag("live") and exclude them from the default test task — they require network access to exchange APIs and fail on CI runners. * fix(java): route live unit tests through proxy for CI ConcurrencyStressTest and ExchangeTypedTest hit live exchange APIs (loadMarkets). Read CCXT_HTTPS_PROXY env var to route through proxy on CI, matching how other language live tests work. Remove @Tag("live") exclusion since tests now work on CI runners. * fix(java): skip live unit tests gracefully when exchange unreachable Use JUnit Assumptions.assumeTrue to skip ConcurrencyStressTest and ExchangeTypedTest when loadMarkets fails (e.g. on CI runners that can't reach exchange APIs). Tests pass locally, skip on CI. * fix(java): fix flaky ConcurrencyStressTest for CI - Wrap loadMarkets in testExceptionPropagationUnderConcurrency with assumeTrue so it skips when exchange is unreachable - Relax thread growth threshold in testPlatformThreadCountStaysBounded to avoid false failures from parallel test suite thread noise * fix(java): implement ethGetAddressFromPrivateKey using web3j Replace stub with real implementation using web3j's Sign.publicKeyFromPrivate and Keys.getAddress. Needed by GRVT exchange for request signing. * fix bitfinex build * fix(java): SafeValueN reflection fallback for WsOrderBook fields + examples SafeValueN only supported Map and List inputs, returning null for arbitrary Java objects like WsOrderBook. This caused safeInteger/ safeFloat/safeString to fail silently on WsOrderBook fields, which meant handleOrderBook buffered all WS deltas forever without processing them — making watchOrderBook hang on the second call. - Add reflection fallback to SafeValueN for reading public fields from arbitrary Java objects, matching Helpers.GetValue behavior - Fix thread-unsafe empty check: String.valueOf() on a mutable List races with WS threads, replaced with instanceof String check - Add SafeMethodsTest (38 JUnit tests) covering Map, List, and object field access consistency - Add 16 examples (11 REST, 5 WebSocket) as a new Gradle module * docs: add Java to README and wiki installation guide Add Java 21+ to the supported languages list, install section, and a new Java subsection with Gradle setup, REST/async/WebSocket usage examples, and links to the examples directory. * docs: add Java usage examples alongside other languages in README Add Java tab to the multi-language code examples section with REST (typed API, order book, OHLCV, balance, create/cancel order), async (CompletableFuture), and WebSocket (watchTicker) examples. * docs: add Java across all documentation files - CONTRIBUTING.md: add Java to install section, dependencies, language list, module entry points, transpiled files section, examples dir - wiki/Manual.md: update overview, add Java tabs to instantiation, loading markets, async/sync, order book, ticker, OHLCV, balance - wiki/FAQ.md: add Java to transpilation language list - wiki/README.md: add C#, Go, Java to install links - wiki/_coverpage.md: add C#, Go, Java to supported languages * docs: add Java tabs to remaining Manual.md and WebSocket manual sections Manual.md: add Java to exchange properties, sandbox mode, rate limit, symbols/markets, market price, all tickers, specific tickers, public trades, personal trades, fetch order, API keys setup, error handling. ccxt.pro.manual.md: add Java to introduction, imports, instantiation, watchOrderBook, watchTicker examples. * docs: add Java to all remaining tab blocks in Manual and WS manual Manual.md: add Java to 26 more sections — precision formatting, sharing markets, market cache, overriding params, pagination (date, id, cursor), market depth, mark/index OHLCV, querying orders, market orders, trigger/stop-loss/take-profit/trailing orders, custom params, clientOrderId, order trades, withdrawal, deposits, transactions, margin mode. ccxt.pro.manual.md: add Java to 14 more sections — watchTickers, watchOHLCV, watchOHLCVForSymbols, watchTrades, watchTradesForSymbols, watchBalance, watchOrders, watchMyTrades, watchPositions, createOrderWs, editOrderWs, cancelOrderWs, cancelOrdersWs, cancelAllOrdersWs. Coverage: 48/52 Manual.md blocks, 18/20 WS manual blocks have Java. Remaining 6 are language-specific (method overriding, parser override, string math, exception class definition, custom WS handler). * feat(java): add typed method overloads directly on Exchange class Replace the separate ExchangeTyped wrapper with typed method overloads injected directly into Exchange.java. Users now get typed returns (List<Trade>, Ticker, OrderBook, etc.), exchange-specific implicit API methods, and properties all on a single object. - Rewrite generateJavaWrappers.ts to inject ~200 typed overloads into Exchange.java (sync + async + convenience variants) - Add castUnifiedApiArgs() post-processor to javaTranspiler.ts that casts all args to (Object) in internal calls, preventing Java overload resolution from picking typed methods over untyped varargs - Export method list to java-typed-methods.json as single source of truth consumed by the transpiler - Auto-detect zero-arg internal calls to avoid conflicting convenience overloads - Delete ExchangeTyped.java and wrappers/ directory - Update all examples to use direct Exchange API - Add ImplicitApi.java example showing both unified and exchange-specific API usage - Fix tests referencing ExchangeTyped * feat(java): typed subclass pattern + docs + skill Replace ExchangeTyped wrapper with typed subclass pattern following Go's approach. Each exchange now has BinanceCore (transpiled, untyped) and Binance extends BinanceCore (generated, typed). Safe by design: Java resolves overloads at compile time, so internal Core code never sees typed methods. Build system: - javaTranspiler.ts: rename transpiled classes to *Core, fix Api extends for derived exchanges, fix ClassName.this self-references - generateJavaWrappers.ts: generate per-exchange typed subclasses with typed overloads delegating via super.method() Docs & examples: - Update README, Install.md, Manual.md to new pattern - REST examples use typed Binance, WS examples use pro.Binance - Add ImplicitApi.java showing unified + exchange-specific API - Add ccxt-java Claude Code skill * fix(java): SafeMethods cleanup, move toTypedList to Exchange - SafeValueN: remove unreachable List<String>/List<Integer> branches after List<Object> (type erasure makes them dead code) - SafeIntegerN: fix racy String.valueOf(result).length() == 0 pattern to use instanceof String s && s.isEmpty() (consistent with SafeValueN) - Remove ~160 lines of commented-out vararg overload experiments - Move toTypedList helper from per-exchange generated classes to Exchange base class (eliminates 110 duplicate copies) * feat(java): typed WS exchange wrappers (pro.Binance extends pro.BinanceCore) Extend the typed subclass pattern to WebSocket exchanges: - WS Core classes now extend typed REST class (pro.BinanceCore extends Binance) so WS inherits REST typed methods - Generate typed WS wrappers (pro.Binance extends pro.BinanceCore) with typed watch method overloads - WS watch overloads use (Object) casts + null-coalesce params to route to untyped WS implementation (avoids hitting inherited REST typed overloads) - Fix ClassName."method" broken references in WS transpiler output - Update WS examples to use typed API (watchTicker → Ticker, watchOHLCV → List<OHLCV>) User API is now consistent: var exchange = new io.github.ccxt.exchanges.pro.Binance(); exchange.loadMarkets(false); Ticker ticker = exchange.fetchTicker("BTC/USDT"); // typed REST Ticker live = exchange.watchTicker("BTC/USDT"); // typed WS * fix(java): WS watchOrderBook/watchTrades typed returns - WsOrderBook.limit(): return this instead of void (matches TS chaining behavior). The transpiled code does return orderbook.limit() which returned null for void methods, causing watchOrderBook to resolve null. - OrderBook constructor: handle WsOrderBook instances by extracting bids/asks/symbol/timestamp directly from the WS object fields. * fix: remove orphaned java-typed-methods.json, fix Binance.java in PR diff - Delete build/java-typed-methods.json (leftover from previous approach, no longer referenced by any build script) - Stage current typed wrapper Binance.java (the PR diff was showing the old transpiled version from a prior commit in branch history) * fix(java): SSL validation, timeout error mapping, SOCKS proxy for REST 1. SSL: Use system default trust manager for WSS connections instead of InsecureTrustManagerFactory. Add validateServerSsl flag (default true) matching the JS/TS pattern. Only disables validation when explicitly set to false. 2. Timeout: Map HttpTimeoutException to RequestTimeout instead of NetworkError, consistent with Python/JS and the HTTP 408/504 mapping. RequestTimeout extends NetworkError so this is backward compatible. 3. SOCKS: Wire socksProxy into initHttpClient() using a custom ProxySelector with Proxy.Type.SOCKS. Previously socksProxy was validated in checkProxySettings() but silently dropped for REST. * fix(java): implement Ed25519 signing (was returning empty string) Replace the stub that returned "" for all EdDSA signatures with a working implementation using Java 21's built-in Ed25519 support (java.security.Signature). No external dependencies needed. Handles secret formats matching TS behavior: - Raw 32-byte seed (used directly) - PKCS#8 encoded key (extracts last 32 bytes as seed) - Base64 string (decodes, then extracts last 32 bytes) Used by: Backpack, Binance (Ed25519 API keys), Woofipro, and other exchanges requiring Ed25519 authentication. * fix(java): callDynamically prefers varargs + numeric coercion Fix reflection-based method dispatch to handle typed subclass overloads: 1. findMethod: prefer varargs methods (the untyped transpiled methods returning CompletableFuture<Object>) over non-varargs typed overloads (returning sync typed objects). This prevents callDynamically from invoking typed methods that return wrong types for the test harness. 2. coerceArgs: handle Integer→Long, Integer→Double and other numeric type mismatches from JSON parsing. Reduces "argument type mismatch" errors in response tests. Response test improvement: 494 → 467 failures (-27). Remaining failures are pre-existing transpiler numeric type issues. * fix(java): prefer varargs in test harness method resolution callExchangeMethodDynamically in BaseTest.java picked the first method by name, often finding typed overloads (fetchTrades(String, Long, Long, Map)) instead of the untyped varargs (fetchTrades(Object, Object...)). JSON-parsed Integer args don't match Long params → "argument type mismatch" on Method.invoke(). Fix: prefer varargs methods which accept any Object type. Response test failures: 464 → 26 (all "argument type mismatch" eliminated). * fix(java): Ed25519 handle List<Byte> from arraySlice The Backpack exchange calls arraySlice(base64ToBinary(secret), 0, 32) to extract the 32-byte Ed25519 seed. arraySlice on byte[] returns List<Byte>, not byte[]. The Eddsa function only accepted byte[] and String, causing "Ed25519 secret must be byte[] or base64 String" for all Backpack authenticated endpoints. Response test failures: 26 → 6 (all Backpack Ed25519 failures fixed). * fix(java): closePosition typed overloads, parse8601 fractional seconds - Add 'close' to ALLOWED_PREFIXES so closePosition/closeAllPositions get typed overloads in generated wrapper classes - Fix parse8601 SPACE_FORMAT to handle optional fractional seconds (e.g., "2026-04-03 20:07:58.823000") — was returning null Response test failures: 6 → 0. * fix(java): upgrade ast-transpiler to 0.0.80, fix unreachable return null - Upgrade ast-transpiler from 0.0.78 to 0.0.80 - Add post-processing to remove unreachable "return null;" after throw statements in transpiled Java code (ast-transpiler 0.0.80 regression) - removeUnreachableReturnNull handles if/else blocks where the else contains a throw (guaranteed termination) Fixes CI build failures for Lighter and Pacifica (fixed in 0.0.80). Remaining: CryptocomCore (1 unreachable — else with return, needs ast-transpiler fix) and BlofinCore (missing API file, CI handles this). * chore: upgrade ast-transpiler to 0.0.82 Fixes unreachable "return null;" after if/else/else-if chains where all branches terminate. Handles if/else, if/else-if/else, and nested chains. Build now passes with 0 compilation errors. * chore: upgrade ast-transpiler to 0.0.83 Fixes all Java compilation errors: - Unreachable return null after if/else/else-if chains - Final variable declarations for loop-scoped variables - Duplicate final variable declarations across scopes - Ternary expressions inside anonymous inner classes - VariableDeclarationList in for-loop initializers Build: 0 compilation errors. Live tests: 6/6 PASS. * fix(java): add generateJavaWrappers to transpile pipeline The CI pipeline runs transpileJava but never ran generateJavaWrappers.ts, so typed wrapper classes (Binance extends BinanceCore) were never generated in CI. dynamicallyCreateInstance returned null → NPE in id-tests and all test runners. Fix: chain generateJavaWrappers.ts after javaTranspiler.ts in both transpileJava and transpileJavaSingle npm scripts. * chore(java): retranspile with ast-transpiler fixes for final-var hoisting Incorporates ast-transpiler fixes for: - BinaryExpression identifier substitution inside object literals - AwaitExpression-wrapped CallExpression object-literal detection - Async-wrapper keyword remap for reassigned params Also adds lighterCreateClient, lighterSignApproveIntegrator, lighterGenerateApiKey, lighterSignChangePubkey stubs and fixes loadLighterLibrary arity in Exchange.java. Disables lighter in Java request static tests (already disabled in responses). * fix(java): address PR review blockers in core/runtime/build Apply the eight blockers raised during the deep review of #27071: - bitget: restore the multiple-trigger validation in createOrder/editOrder using a transpiler-friendly boolean expression. Previously the check was commented out (and the leftover comment used && + comma instead of this.sum() > 1, so it would not even re-enable correctly), leaving JS, Python, PHP, Go and C# users without server-side trigger conflict detection. - build: pin jackson-databind to 2.18.2 (CVE-2024-50379) and centralize jackson, web3j and netty versions in gradle/libs.versions.toml. Drops the dynamic 2.17.+ range that broke build reproducibility. - Exchange.loadMarkets(): wrap the dedup + state transition in a synchronized block, reset marketsLoading to null on failure (so callers can retry), and add an exceptionallyCompose to clear reloadingMarkets. Fixes a race that returned null on concurrent first-callers and lost the failure state on errors. - WsClient.connect(): replace volatile boolean + non-atomic check-then-set with AtomicBoolean.compareAndSet so concurrent connect() calls collapse onto a single createConnection task. - WsClient.onError(): complete the existing connected future exceptionally before installing a fresh one. The previous "if isDone, just replace" branch silently dropped errors and left awaiters stuck. - WsClient.close() and reject(error, null): snapshot keys before mutating futuresMap to prevent ConcurrentModificationException under concurrent watch/resolve. Also interrupt the ping thread so it exits without waiting for the next keepAlive tick. - Exchange.randNumber(): use the existing static SecureRandom field instead of a fresh, unseeded Random per call. Also force a non-zero leading digit so Integer.parseInt round-trips the requested width. Tests: extend ConcurrencyStressTest with concurrent-loadMarkets and randNumber-width regressions and add WsClientConcurrencyTest covering connect()'s atomicity, the close()/reject(error,null) CME, and onError()'s complete-then-replace contract. All 144 lib tests pass. * chore: bump ast-transpiler to ^0.0.84 * fix(java): yymmdd() single-arg default infix should be '' to match TS TS defines: const yymmdd = (timestamp, infix = '') => ... Java was defaulting to '-', which made paradex option symbols compute as 'BTC/USD:USDC-26-05-29-320000-P' instead of the correct 'BTC/USD:USDC-260529-320000-P'. Fixes paradex fetchMarkets static response test, unblocking the Java CI build. * fix(java): address PR review blockers in core/runtime/build - Crypto.Jwt ES256: implement P-256 ECDSA signing via JCA SHA256withECDSAinP1363Format. Unblocks Coinbase Advanced Trade authentication, which was unconditionally throwing UnsupportedOperationException on every authenticated call. Parses both PKCS#8 and SEC1 ("-----BEGIN EC PRIVATE KEY-----") PEM formats. - Encode.binaryToBase58: implement proper Base58 encoding via BigInteger against the existing B58 alphabet table. Previously returned hex, silently breaking Pacifica signature auth and Waves attachment payloads. - Encode.urlencodeWithArrayRepeat: URL-encode keys and list items (not only scalar values). Previously raw '&', '=', and spaces in parameter keys/array items broke HMAC signatures for binance / coinbase / krakenfutures batch endpoints. - Time.parse8601: replace over-broad contains("+0") offset strip (which silently dropped +0100, +0530, +0900, etc.) with a proper OffsetDateTime.ISO_OFFSET_DATE_TIME parse that normalizes +HHMM to +HH:MM before parsing. - Throttler.rollingWindowLoop: add fallback sleep when the inner loop makes no progress and timestamps is empty (e.g. head cost > maxWeight, which triggers by default when maxWeight=0). Prevents a 100% CPU busy-spin. * fix(java): typed WS API on pro.* now dispatches correctly Reported: pro.Binance.watchTicker("BTC/USDT") threw NotSupported because the typed REST wrapper Binance.java:818 delegated via super.watchTicker(...), where super is BinanceCore (REST) — not the pro.Binance subclass. super is lexically bound in Java, so runtime subclass overrides can never be reached that way. Fix (three small changes): 1. build/generateJavaWrappers.ts: filter watch methods out of the REST typed wrapper. They now live only on the WS typed wrapper (pro/<Exchange>.java), which extends pro.<Exchange>Core where the real WS implementation lives. super.watchTicker(...) there resolves directly to the WS Core method. 2. build/javaTranspiler.ts: add fixVoidReturnNull postprocess for WS classes. insertReturnNullInSupplyAsync was leaking "return null;" into void event-handler methods (handleMessage, handleLiquidation, etc.) because its supplyAsync nesting counter didn't decrement cleanly across methods. New pass rewrites "return null;" → "return;" inside void method bodies using brace-depth tracking to stay within the method scope. 3. Regenerate exchanges/Binance.java, exchanges/Bybit.java (watch methods removed), and exchanges/pro/Binance.java (now a small typed wrapper; transpiled impl moved to untracked pro/BinanceCore.java). Class hierarchy after fix: pro.Binance → pro.BinanceCore → exchanges.Binance → BinanceCore (WS transpiled) (REST typed) (REST transpiled) Verified: - compile green - request tests: 4404/4404 - response tests: 1406/1406 - pro/Binance.java has typed watchTicker(String) that resolves through super to pro.BinanceCore.watchTicker(Object, Object...) (the real WS implementation), not to Exchange.watchTicker stub. No REST regressions — non-watch methods follow the unchanged code path super.fetchX(...) → BinanceCore.fetchX(...). * fix(java): WS transpile passes + helpers for all-exchanges build Adds post-processing passes in build/javaTranspiler.ts so all 80 WS exchanges compile (previously 11 cores had to be moved aside): - fixVoidReturnNull: convert `return null;` → `return;` inside `public void` method bodies. Brace tracking now strips line/block comments and string literals, so the `{` chars inside `//` JSON-sample comment blocks no longer leak `inVoid` state into downstream methods. Fixes the supplyAsync-void-lambda and missing-return-value errors in Ascendex, Cryptocom, Coinex, Kucoin. - collectMethodNamesInClass + base-class whitelist: replaces the fragile hardcoded `callbackMethods` list. Rewrites `this.<method>` used as a map value / assignment RHS to the `"<method>"` string literal. Fixes `cannot find symbol` for `resolveData`, `actionAndMarketMessageHash`, `actionAndOrderIdMessageHash`, etc. - redirectToAsyncOnJoin: `(this.restMethod(arg)).join()` inside a WS Core dispatched to the typed REST overload (returning typed `Balances`/`List<Position>` directly) and broke `.join()`. Casts each arg to `(Object)` to force dispatch to the inherited untyped `CompletableFuture<Object>` varargs. Scoped to methods that exist in the exchange's REST typed wrapper file to leave WS-core local helpers alone. Fixes Bingx, Bitmart, Hashkey, Toobit, Gate. - splitTopLevelArgs: comma splitter that respects generics `<...>`, parens, brackets, and string literals — used by redirectToAsyncOnJoin. - rewriteDelayWithStringCallback: balanced-paren rewrite of `this.delay(ms, "name", ...args)` to `this.scheduleCallback(...)`. The existing regex only handled 3-arg form with a method-ref callback; fails once the method-ref pass converts refs to strings. Supports any arg count. - Spawn-as-expression with string callback: new regex rewrites `this.spawn("name", args)` used as an expression value (not a statement) to `this.spawnWithResult("name", args)`. Fixes Kucoin `urls[connectId] = this.spawn(this.negotiateHelper, ...)`. Adds two helpers to Exchange.java that the transpile output now targets. Putting the lambda inside a helper method makes the captured args effectively-final method params, avoiding lambda-capture errors that an inline rewrite would produce with reassignable locals: - scheduleCallback(Object delayMs, String methodName, Object... args): sleep-then-callDynamically on the virtual executor. - spawnWithResult(String methodName, Object... args) -> io.github.ccxt.ws.Future: async dynamic dispatch, returns a Future that resolves with the callee's result (auto-unwrapping CompletableFuture returns) or rejects on exception. Verified: - compile green on all 80 WS cores + typed wrappers - request tests: 4404/4404 - response tests: 1406/1406 - binance WS live: 20 ticker updates streamed * fix(java): run WS transpile in transpileJava npm script The typed WS wrappers (exchanges/pro/<Exchange>.java, tracked) extend the transpiled WS Core (exchanges/pro/<Exchange>Core.java, generated). Previously the CI only ran `npm run transpileJava` which dispatched to `build/javaTranspiler.ts --multi` (REST only), so pro/*Core.java files were never generated and compile failed with `cannot find symbol: class BinanceCore` from pro/Binance.java. Add `--ws` invocation to both `transpileJava` and `transpileJavaSingle` so the full WS pipeline runs. Adds a standalone `transpileJavaWs` alias for ad-hoc WS regeneration. Verified: - rm all pro/*.java, run `npm run transpileJava`: regenerates 80 Core + 80 typed wrapper files. - compile green * fix(java): ast-transpiler forward-reference reassignment fix Bumps ast-transpiler to include fix for forward-reference final-var hoisting: when a variable is used inside an object literal BEFORE being reassigned later in the same function body, analyzeFinalVars's usageToFinalName (set in a pre-walk) wasn't being consulted by getVarListFromObjectLiteralAndUpdateInPlace, which only checked the lazily-populated ReassignedVars. Result: no final shadow was emitted, Java compile failed with 'local variables referenced from an inner class must be final or effectively final'. Hit by Blofin.createTpslOrderRequest after merging upstream/master (new blofin.ts content from #28432). Pointing at a tagged commit on pcriadoperez/ast-transpiler until a new 0.0.85 npm release can be cut. Fix also sent upstream. * chore: bump ast-transpiler to ^0.0.85 (published release) The forward-reference reassignment fix is now in the official 0.0.85 release. Switching from the temporary fork-branch pointer back to the npm registry. Verified: clean retranspile green, compile green, request 4403/4403, response 1411/1411. * fix(ci): regenerate package-lock with npm registry resolution The lockfile still had "resolved": "../ast-transpiler" from the earlier file-path install, so CI's npm ci couldn't resolve the dep and every language build failed with "Cannot find package 'ast-transpiler'". Rebuilding the lockfile from scratch after the ^0.0.85 bump points at the npm tarball instead. * ci: retrigger after transient github HTTP 500 on wiki clone * fix: PHP sync regex + Go regex match patterns + 2 typos - build/transpile.ts: getPHPSyncRegexes / Promise\all replace now strip the `\React\` FQN prefix that ast-transpiler 0.0.79+ emits, so the sync PHP files (e.g. test_fetch_tickers.php) parse again. - build/goTranspiler.ts: 9 regexes still used `interface\{\}` as their match pattern but the transpiler now emits `any`. They were silently no-op'ing, leaving `client any,` / `sourceExchange any` in the generated Go and breaking the build with errors like "type any has no field or method Futures". Updated all match patterns to `any`. - go/v4/exchange_types.go: two casualties of the bulk `interface{}` → `any` replace where `Interface{}` got stripped: return Marketany -> return MarketInterface{} return TradingFeeany, ... -> return TradingFeeInterface{}, ... Verified locally: JS build, Python+PHP syntax, C# build, Go build, go vet ./tests/... all green. * chore: upgrade ast-transpiler 0.0.79 → 0.0.85 * fix(java): pass currencies to setMarkets + numeric sortBy comparator Two base-class fixes that turn ~40 live-test failures green. 1) Exchange.loadMarketsHelper (Exchange.java:1452-1462) The promise chain captured `currencies` in the first lambda but the second lambda called `this.setMarkets(markets)` without the currencies arg. With currencies=null, setMarkets reconstructs the dict from market base/quote only — silently dropping currencies that don't appear in any market (AGLD/WBTC for bigone/apex, USDC for aftermath). Fix: nest the second lambda inside the first so closure captures currencies and passes it through: return setMarkets(markets, currencies); Restores parity with ts/src/base/Exchange.ts:1144. Confirmed on bigone/apex/aftermath/krakenfutures/bitvavo/coinex live tests — all 6 newly green after this single change. 2) Generic.sortBy (Generic.java) Comparator coerced every value via .toString() and used Comparator.naturalOrder() — i.e. lexicographic. Numeric strings like "53.0" lex-sorted before "78301.0" because '3' > '2' at the second char. Order books came out shuffled — sortBy(bids, 0, true) put 53.0 before 78301.0 instead of after, breaking parseOrderBook for ~28 exchanges. Fix: introduce toComparable(v, default) that returns Double when the value parses as a number and String otherwise, mirroring TS `<`/`>` semantics. Coerce both the indexed-list-element variant and the String-key variant. Verified via standalone unit check: sortBy of [53, 78301, 78302, 126186.5] descending now returns [126186.5, 78302, 78301, 53]. Confirmed on krakenfutures/bigone live tests. Local sample of 98 exchanges (excluding 3 binance variants under geo-restriction): pre-fix had 58 failures; post-fix has ~17 failures - a 70%+ reduction in real failures from these two changes alone. Static suites unchanged: 4403/4403 request, 1411/1411 response, base tests green. * fix(java): unbreak loadMarkets for 8 more exchanges Two follow-up fixes after the previous setMarkets/sortBy commit. Both addressed regressions that the previous fix EXPOSED in deeper code paths: 1) Exchange.loadMarketsHelper has[] flag check (Exchange.java:1443-1449) `(Boolean) this.has.get("fetchCurrencies")` threw ClassCastException for bit2c, bitbns, coincheck — they don't override has[] and inherit the base value "emulated" (a String). Mirror TS `=== true` semantics: only treat as true when the value is actually a Boolean. Affected: bit2c, bitbns, coincheck → all 3 now pass loadMarkets. 2) Generic.sortBy mixed-type ClassCastException (Generic.java) The previous toComparable+naturalOrder() approach returned a Comparable that was either Double (for numeric strings) or String (for non-numeric). When a single sort saw both — which the new setMarkets(markets, currencies) merge path does for currency precision/fee fields — naturalOrder() tried to cast across types and threw "String cannot be cast to Double". Replaced with a single Comparator (compareJsLike) that handles each pair: both numeric → Double.compare, both string → lex, mixed → coerce both to String for lex compare. Mirrors TS `<`/`>` coercion well enough for ccxt's sort use-cases. Affected: bingx, mexc, yobit, bybit → all 4 now pass loadMarkets. coinsph passes loadMarkets but fails fetchTickers on a separate URL-encoding issue (out of scope here). Verification (local, 8 previously-failing + spot-check on previously-passing for regression): bingx OK (was: ClassCastException Double) mexc OK (was: ClassCastException Double) yobit OK (was: ClassCastException Double) bybit OK (was: ClassCastException Double) coinsph now passes loadMarkets, fetchTickers fails on URL-encoding (unrelated) bit2c OK (was: ClassCastException Boolean) bitbns OK (was: ClassCastException Boolean) coincheck OK (was: ClassCastException Boolean) bigone, apex, krakenfutures, bitvavo, coinex, alpaca, arkham, coinbase still OK Static suites unchanged: 4403/4403 request, 1411/1411 response, base green. * java: follow HTTP redirects in base HttpClient (fixes gemini contractSize) Java's java.net.http.HttpClient defaults to Redirect.NEVER, so any 3xx response surfaces with an empty body. exchange.gemini.com responds 303 to / (with a session cookie), so fetchCurrenciesFromWeb received an empty body, this.options['tradingPairs'] never got populated, and gemini perp markets (e.g. xrpusdcperp) fell through to the string-parse branch of parseMarket, leaving contractSize undefined and failing the swap contractSize assertion in live tests. Match TS/Node fetch behavior by configuring the builder with Redirect.NORMAL — this also unblocks any other Java-side webApi endpoint that 3xx-redirects through a session boundary. * fix(gate): guard option market create_time=0 against >2009 assertion gate's options API occasionally returns create_time: "0" for newly listed contracts (seen on 2026-04-28 for BTC_USDT-20260508-76000-P). safeTimestamp(market, 'create_time') returns 0 for that input, which trips the loadMarkets validator's 'timestamp must be >= 2009' check. Treat 0 as missing — same intent as the existing omitZero pattern, but without the temporary string detour. Verified with live-tests-rest-ts. * java: tolerate browser/Node-legal but URI-illegal chars in request URLs java.net.URI.create enforces RFC 3986 strictly. coinsph builds ?symbols=%5B"BTCUSDT"%5D — partially-encoded by design, with literal double-quotes the exchange API expects. Node/Axios accept this; URI.create throws URISyntaxException on the bare ". Same hazard applies to |, {, }, ^, `, <, >, space when they appear in query strings produced by exchange- specific encoders. Pre-pass the URL byte-by-byte and percent-encode that handful of chars before URI.create. Existing %XX escapes are left alone — we only touch raw illegal chars, so already-encoded queries round-trip unchanged. * test: skip swap suite when getValidSymbol returns undefined Some exchanges advertise has['swap']=true via describe() but expose no swap markets at runtime (e.g. bequant inherits hitbtc's swap support flag but the live symbol list is spot-only). getValidSymbol returns undefined in that case, and the test framework crashed on `undefined.replace('BTC', 'ETH')`. Guard the secondary-symbol derivation; if no primary swap symbol exists, leave swapSymbols undefined and the test framework will correctly skip the swap suite. Java TestMain.java is the transpile of the same fix from ts/src/test/tests.ts. * java(transpile): null-safe Array.isArray via Helpers.isArrayJs ast-transpiler emits `(X instanceof java.util.List) || (X.getClass().isArray())` for `Array.isArray(X)`, but X.getClass() NPEs when X is null. JS Array.isArray(null) is false; mirror that here. - Add Helpers.isArrayJs(Object) with explicit null branch - Post-transpile regex in build/javaTranspiler.ts rewrites the broken pattern across all REST/WS/test outputs (~140 sites). Three regex passes mirror the three transpile entry points (transformExchange, transpileBaseMethods, transpileTests, transpileMainTest). Surfaced via aftermath fetchTrades — TestSharedMethods.assertType called `Array.isArray(entryKeyVal)` where entryKeyVal came from safeValue() returning undefined, NPE'd in Java. Also: skip-tests.json — aftermath fetchCurrencies activeMajorCurrencies. The /currencies endpoint reports BTC with deposit=false, withdraw=false (aftermath is a Sui DEX, BTC is not natively bridgeable there); the major-currency assertion does not apply. * fix(bitfinex,upbit): live test failures (TS upstream) bitfinex: parseTicker length === 17 / === 16 hard-coded for funding currency shape, but bitfinex now appends a millisecond timestamp to multi-ticker responses (length 18) and singular fetchTicker (length 17). fUSD was misparsed as a trading pair, putting index 8 into baseVolume which is negative DAILY_CHANGE — failed the >=0 ticker assertion. Accept both lengths. upbit: fetchTickers joined all ~700 ids into one ?markets= query (~7KB) — Tomcat rejected with HTTP 400. Chunk into batches of 100, fetch via Promise.all, merge via arrayConcat. Used this.arraySlice rather than Array.prototype.slice because the Java transpiler maps .slice on Object to Helpers.slice (String) and ClassCastExceptions on List input. * java: rebuild HttpClient when proxy fields change after construction Tests (and library users) set httpProxy/httpsProxy/socksProxy AFTER exchange construction, but initHttpClient was only invoked once at construct time. The HttpClient captured the empty proxy state, so a later assignment had no effect — bullish (which requires httpsProxy per skip-tests.json) hit upstream APIs directly and got 403 even though the test framework had configured the proxy. Track a proxy-fingerprint string and rebuild the HttpClient lazily on the first fetch() after the fingerprint changes. Cheap (one rebuild per config change) and keeps connection reuse otherwise. Resolves bullish 403 (Java only — TS/Node fetch reads proxy at request time so it never had this issue). * fix(base): avoid params={} mutation in createTrigger/StopLoss/TakeProfit wrappers The six wrapper methods mutated the caller's params dict in place. After Python transpilation that becomes the well-known mutable-default-argument anti-pattern: params={} is bound once at def time, so a triggerPrice set in one call leaks into the next call that omits params, breaking createOrder's "exactly one of trigger/stopLoss/takeProfit/trailing" guard. Replace in-place assignment with this.extend(...) so each call gets a fresh dict. * java: 3 java-lang transpile-compat fixes + 4 missing Exchange fields ts/src/test/tests.ts — move 3 `// skip for java for now` inline comments off the `return false;` line. ast-transpiler 0.0.85 drops `return false;` when an inline trailing comment follows on the same line, leaving an empty `if` body in the Python output (IndentationError). ts/src/bydfi.ts — revert `fetchTransactionsHelper` signature back to all required params. Adding `params = {}` makes bydfi (alphabetically before dydx/poloniex) the first exchange to register a single-optional struct shape; the goTranspiler's `if (capName in goTypeOptions)` early-exit then shadows dydx/poloniex contributing Code/Since/Limit fields. Build fails with `opts.Code undefined (FetchTransactionsHelperOptionsStruct has no field or method Code)`. ts/src/bitget.ts — join two multi-line `multipleTriggers` boolean expressions onto single lines. ast-transpiler 0.0.85 emits unparenthesized `||` line continuations in Python (`x = a\n or b\n or c`) which is an IndentationError. java/lib/src/main/java/io/github/ccxt/Exchange.java — add 4 missing fields (`name`, `countries`, `certified`, `pro`) referenced by the transpiled `describe()` after upstream commit 4abefad819d added the exchange-properties tests. Also includes the transpiled diff for the already-existing `createTrigger/StopLoss/TakeProfit` wrappers using `extend(params, ...)` from PR #28508. * revert change * fix python linting * java props Co-authored-by: Copilot <copilot@github.com> * java(base): coerce Boolean→double in Generic.sum, fix encodeURIComponent + urlencodeNested Generic.toDouble: bitget createOrderRequest does `this.sum(isTriggerOrder, isStopLoss…) > 1` with Boolean inputs. JS coerces booleans (true→1, false→0) in arithmetic; Java's strict typing fell through to Double.parseDouble(String.valueOf(false)) → NumberFormatException. Add an explicit Boolean → 1.0/0.0 branch. Surfaced as 5× bitget createOrder/editOrder static-response failures. Encode.encodeURIComponent: previously included `[` `]` in the unreserved set to mimic C# HttpUtility output, but TS sources call encodeURIComponent expecting JS semantics. Aster's cancelOrders does `encodeURIComponent(this.json([orderId]))` and the test expects `orderIdList=%5B...%5D`; with brackets unencoded the value came out `orderIdList=[…]` (literal). Match the actual JS unreserved set: A-Z a-z 0-9 - _ . ~ ! * ' ( ) Encode.urlencodeNested: now uses urlEncode (java.net.URLEncoder, the qs-style stricter encoder that escapes `(` `)`) for both name segments and the value, with literal brackets concatenated between key segments. Mirrors qs.stringify(obj, {encodeValuesOnly:true}) — which kraken's private body relies on for fields like `method=Polygon%20%28MATIC%29`. Exchange.java: drop 4 duplicate field declarations (`name`, `countries`, `certified`, `pro`) — `f2cc20d91e9 java props` from upstream/master already added them with stricter typing (`List<Object>` rather than `Object`). Keep that version. TestMain.java: transpile artifact — inline `// skip for java for now` comments were moved off the `return false;` line in ts/src/test/tests.ts; the regen drops the comments entirely in the Java output. All 4407 static-request + 1415 static-response Java tests now pass. * add md explaining * add versioning Co-authored-by: Copilot <copilot@github.com> * add java cli helper and package.json command Co-authored-by: Copilot <copilot@github.com> * export exchanges list Co-authored-by: Copilot <copilot@github.com> * instantiate ws exchange * cli: support for ws methods Co-authored-by: Copilot <copilot@github.com> * java: fix ws data races on shared exchange state Three changes that together eliminate the ConcurrentModificationException seen during high-frequency watchTrades and the silent corruption of orderbook/balance state under concurrent frame handling: - Exchange.options is now a ConcurrentHashMap (matches C# Client.Options). Helpers.addElementToObject translates put(key, null) to remove(key) for ConcurrentHashMap so existing TS code that assigns null keeps working. - Generic.Extend snapshots the source maps under their own monitor before iterating, so callers passing options-like shared maps no longer race. - WsClient now serializes handleMessageCallback through a per-client single-thread virtual-thread executor. Frames from one connection are processed in arrival order (matches C# Receiving loop / JS event loop); different clients still run in parallel. Adds SharedStateRaceTest (deterministically reproduces the CME) and WsClientMessageOrderingTest (proves at-most-one handler runs per client). * java: parse8601 accepts trailing UTC/GMT zone suffix Aftermath returns datetime fields like "2025-12-29 22:43:54.639 UTC". JS's Date.parse and Python's dateutil accept the trailing zone word; java.time's strict parsers do not, so parse8601 was returning null on those values. The live test harness then fed the null through Helpers.subtract / Helpers.toString / Double.parseDouble and crashed with a misleading `Cannot invoke "String.trim()" because "in" is null`. Fix: strip a trailing " UTC"/" GMT" (case-insensitive) before the date math. parseDate gets the same treatment for symmetry. * update cli and metadata * fix ping pong issue * add verbose log to onMessage * java: reply to inbound ws ping frames with pong Binance's WebSocket server sends server-initiated Ping frames every few minutes and closes the connection with `1008 Pong timeout` if the client doesn't echo the Ping's payload back as a Pong (RFC 6455 §5.5.3). Netty does not auto-reply to inbound Pings; that's the application's job, and WsClient was silently dropping them — every long-lived Binance stream was getting kicked within ~10 minutes. Add a PingWebSocketFrame branch in WsClientHandler.channelRead0 that writes a PongWebSocketFrame echoing the inbound payload. Adds a unit test that exercises the dispatch via Netty's EmbeddedChannel. Also rename the class inside MetaData.java to match the file name — the file was renamed from Exchanges.java upstream but the class declaration still said `Exchanges`, breaking compilation. * fix ping-pong ws level * add log * run ws tests pipeline * init ws tests transpiling Co-authored-by: Copilot <copilot@github.com> * fix static calls * java: emit truncation overloads for typed wrappers Previously the generator emitted exactly two overloads per method: the full typed signature and a required-only convenience overload. Users who wanted any intermediate shape (e.g. fetchTrades(symbol, since)) had to fall back to the full signature with explicit nulls. Now genMethod emits an overload at every arity from required-only up through full-1, each delegating to the full method with declared defaults (or typed null) for the trailing args. Each truncation has a unique arity, so Java's overload resolution stays unambiguous at every call site. Skipped when requiredParams.length === 0: those overloads would shadow internal `this.fetchPositions()` / `this.fetchPositions(null)` calls in transpiled WS Core code (which extends typed REST and inherits these overloads), causing the inherited call to bind to the typed return instead of the parent's `Object...` returning CompletableFuture. Audited by grepping pro/*Core.java for self-calls — fetchPositions is the only zero-required wrapper called internally. Adds TruncationOverloadTest pinning the contract: each truncation forwards the right defaults to the full method. * java: transpile pro/test WS test suite into tests.exchange.ws Carlos's previous commits wired up the WS test pipeline scaffolding (uncommented transpileWsExchangeTests, fixed the csharp/java key typo, extended the cross-file static-call regex to recognise watch filenames) but the resulting Java still didn't compile across the 17 WS tests. This commit fills in the remaining pieces: - mkdir the ws/ output folder if missing (was crashing on first run) - skip the cross-file qualification rewrite when the called name ends with "Helper" (testWatchTickersHelper, testWatchBidsAsksHelper are declared inline on the same class as the public test, so they must resolve as instance method calls — not nonexistent helper classes) - collapse multi-arg System.out.println(a, b, c) to a single String.valueOf-concat string. Walks paren depth respecting string literals so messages containing "()" don't break the matcher. - replace the broken `exchange.spawn(fn, new object[]{args})` C#-style output with a Runnable lambda that calls the method and .join()s its CompletableFuture, mirroring the lib's spawn pattern. Lowercase `object[]` was a pure typo holdover from the C# transpile. - add a paren-counting fallback for `Helpers.callDynamically(...).join()` that the existing 1-level-nested regex couldn't reach (WS tests pass arrays-of-lists which cleared the threshold). Also drops the leftover commented-out duplicate Ping handler in WsClient.java that the merge of Carlos's parallel ping/pong fix and mine left behind. Live sweep: 80/80 exchanges OK (65 tested, 15 [Skipped] per skip-list). :lib:test, :tests:compileJava, request-java (4410), response-java (1415) all pass. * java: post-merge build fixes After merging upstream/master, two compile errors surfaced: - UpbitCore.fetchOrderBooks uses `String.join(",", (List<String>) this.ids)` on the typed field. Java's strict generics refuse the cast from `List<Object>` to `List<String>`. Change Exchange.ids to a raw List with @SuppressWarnings, so unchecked-but-legal casts work — matches how the field is already used (assigned from a Set<String> keySet at marketsSortedById). - CLI Main.java still imported `io.github.ccxt.Exchanges` after Carlos's earlier rename to `MetaData`. Update both the import and the ProExchanges reference. * build: update export-exchanges.js path after Java Exchanges→MetaData rename The build pre-step writes the metadata exchange list into the Java package, but still pointed at io/github/ccxt/Exchanges.java which was renamed to MetaData.java upstream. Local builds had the file because of git history, but a fresh CI checkout doesn't, breaking every build job with ENOENT before transpile even starts. Same fix as the earlier CLI Main.java import update — closes the last reference to the old filename. * ws tests compiling Co-authored-by: Copilot <copilot@github.com> * fix class instantiation * java: ws audit fixes — close(), reload-collapse, permissive deflate Three independent items from the WS API audit, each pinned by a TDD test: #1+#7 Exchange.close() + typed ExchangeClosedByUser Mirrors the TS Exchange.close() at ts/src/base/Exchange.ts:1537 (above the transpile delimiter, so it's hand-written in Java too). Iterates this.clients, tags each WsClient with an ExchangeClosedByUser, calls close() on each, clears the map. WsClient.close() now prefers a typed closeReason set by the caller over the bare RuntimeException("Connection closed by the user"), so consumers can `catch (ExchangeClosedByUser)` and distinguish deliberate shutdown from a remote-side disconnect. #11 loadMarkets(reload=true) collapses to one fetch under load The guard at Exchange.java:1515 was `if (!this.reloadingMarkets || reload)`, which short-circuits the in-flight check whenever reload is true — 20 concurrent reload callers used to spawn 20 sequential helper invocations, each overwriting this.marketsLoading. Drop the `|| reload` so concurrent callers always join the in-flight future. The cache-vs-fetch decision still respects reload via the outer `marketsLoaded && !reload` guard. #2 Permissive permessage-deflate handshaker WsClient.java used WebSocketClientCompressionHandler.INSTANCE, whose default PerMessageDeflateClientExtensionHandshaker rejects both server_no_context_takeover and client_no_context_takeover. Coinbase advertises both on every connect → CodecException on handshake, every time. Replace the default with a custom WebSocketClientExtensionHandler built from PerMessageDeflateClientExtensionHandshaker(6, true, 15, true, true) (allowClientNoContext=true, requestedServerNoContext=true) — the maximally-permissive defaults used by gorilla/websocket, browser native WebSocket, etc. Wire-protocol semantics are unchanged so exchanges that don't advertise these extensions keep working. Tests: - ExchangeCloseTest: clients map drained; in-flight future rejects with ExchangeClosedByUser. - LoadMarketsConcurrencyTest: 20 vthreads call loadMarkets(true) against a counting helper; assert exactly 1 fetch. - WsClientCompressionHandshakeTest: localhost Netty server hand-crafts the handshake response with both no-context params (mimicking Coinbase exactly); assert WsClient handshake completes. Validated: - :lib:test green - Coinbase WS live test now passes (was failing every connect) - 80/80 WS exchanges OK in full sweep * test: cross-language test.close.ts for Exchange.close() lifecycle Replaces the manual ts/src/pro/test/base/test.close.ts harness (binance- hardcoded, runs only via tsx) with a per-exchange test that the existing WS sweep picks up automatically across all language ports — closes the gap the user flagged where Exchange.close() had no automated coverage. What it tests (3 scenarios on every WS-capable exchange): 1. close() on an exchange with no active subscriptions — must not error 2. open watchTicker, drain it, close — must not error 3. close() while a watch is awaiting — must terminate the awaiter (no infinite hang, no uncaught crash). The exact rejection type is locked down by ExchangeCloseTest (Java unit test); the cross-language test is end-to-end coverage, not type-strict. Wiring: - tests.helpers.ts auto-loads test.close.ts when ws=true (alongside 'features'). Same change in BaseTest.java for Java's hand-written test loader. - tests.ts: isCloseTest flag bypasses the exchange.has gate; close isn't advertised through that map. - tests.ts: close runs as the WS test EPILOGUE, after both spot and swap rounds finish — not in the per-round parallel test list, since it tears down the WS clients other tests share. This keeps each batch finishing on a live channel. - exchange.has['watchTicker'] gates scenarios 2 and 3 (skips on exchanges that only expose orderbook/trades over WS, e.g. aftermath). - Exchange.close() now returns CompletableFuture<Object> so transpiled `(exchange.close()).join()` matches the TS `await exchange.close()` pattern across ports. Validated: aftermath + bybit close[] no longer surface in the per-exc…

lishawnl added 15 commits October 1, 2024 17:25

Add Alephx Rest APIs

f56fd10

Auto generated code after build

937ea74

Fix auth

efacef4

Fix errors and transpile

aef7bd2

Add async tests

84f8708

Add ws support for alephx

230fd32

Add alephx ws examples

58e5296

Fix fee cost key

a51b0d4

Merge branch 'master' into alephx-support

a897a09

Merge conflicts

9ebf7d8

Add more APIs, balance/status/trades for order

97874ea

Fix order id in trade

b7c915b

Merge branch 'master' into alephx-support

6daea62

Allow sandbox URLs in alephx

bd373e2

Update gateio limit and interval for watch_orderbook

9df56c0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Alephx support#2

Alephx support#2
lishawnl wants to merge 15 commits into
masterfrom
alephx-support

lishawnl commented Oct 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

lishawnl commented Oct 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant