Unbounded: core/src/exchanges/myriad/fetcher.ts — allMarkets pagination buffer grows proportional to caller-supplied limit with no server-side cap

[realfishsam]

Location

core/src/exchanges/myriad/fetcher.ts:118-141 (fetchRawMarkets pagination loop)

Code

// Paginate through multiple pages
const allMarkets: MyriadRawMarket[] = [];
let page = 1;
const maxPages = Math.ceil(limit / MAX_PAGE_SIZE);  // limit is CALLER-SUPPLIED

while (page <= maxPages) {
    queryParams.page = page;
    queryParams.limit = MAX_PAGE_SIZE;

    const response = await this.ctx.http.get(`${this.baseUrl}/markets`, { ... });
    const markets: MyriadRawMarket[] = data.data || data.markets || [];
    allMarkets.push(...markets);   // ← accumulates all pages

    const pagination = data.pagination;
    if (!pagination?.hasNext || markets.length === 0) break;

    page++;
}

return allMarkets.slice(0, limit);

MAX_PAGE_SIZE = 100. limit comes from params.limit passed by the caller with no enforcement of an upper bound in fetchRawMarkets().

Growth Pattern

• A caller passing limit: 10000 triggers Math.ceil(10000 / 100) = 100 pages
• allMarkets accumulates 10,000 MyriadRawMarket objects before slice(0, 10000) returns all of them
• Each MyriadRawMarket contains outcomes with price charts (price_charts?: Record<string, { timeframe: string; prices: { value: number; timestamp: number }[] }>) — potentially large nested objects
• No server-side hard cap exists in the fetcher; only maxPages computed from the unchecked limit controls the loop

OOM Estimate

Requested limit	Pages fetched	Est. size per market	Total
1,000	10	2 KB	~2 MB
10,000	100	2 KB	~20 MB
100,000 (no guard)	1,000	2 KB	~200 MB

With price chart data included, each market object can be 5–20 KB. Multiple concurrent large fetchMarkets() calls compound the peak.

Suggested Fix

• Enforce a hard maximum in fetchRawMarkets():

  const MAX_FETCH_LIMIT = 5000;
  const limit = Math.min(params?.limit || 100, MAX_FETCH_LIMIT);

• Expose a paginated variant that yields pages rather than accumulating them all into a buffer.
• Document the effective limit on fetchMarkets() via MarketFetchParams.limit.

---

Found by automated unbounded operations audit