Skip to content

added a new admin endpoint - /logs to fetch log files - #491

Merged
afrind merged 2 commits into
mainfrom
qlogs/admin-endpoint
Aug 17, 2026
Merged

added a new admin endpoint - /logs to fetch log files#491
afrind merged 2 commits into
mainfrom
qlogs/admin-endpoint

Conversation

@akash-a-n

@akash-a-n akash-a-n commented Jul 8, 2026

Copy link
Copy Markdown
Contributor
  • Implement GET /logs?type=<mlog|qlog>&connection_id=<hex> route.
  • Offload blocking file I/O (std::ifstream::read) to folly::getGlobalCPUExecutor() to avoid stalling the admin event loop.

This change is Reviewable

@akash-a-n
akash-a-n force-pushed the qlogs/admin-endpoint branch from 9bed433 to 74ba272 Compare July 9, 2026 06:47
@akash-a-n
akash-a-n force-pushed the qlogs/admin-endpoint branch 2 times, most recently from c161002 to 02c1462 Compare July 22, 2026 15:12

@afrind afrind left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@afrind reviewed 8 files and all commit messages, and made 6 comments.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on akash-a-n).


src/admin/ConnectionLogsHandler.cpp line 54 at r1 (raw file):

// Read an entire file into an IOBuf. Returns nullptr if the file cannot be
// opened, is empty, or exceeds maxBytes.
std::unique_ptr<folly::IOBuf> readFileToIOBuf(const std::string& path, size_t maxBytes) {

There's probably a folly helper


src/admin/ConnectionLogsHandler.cpp line 70 at r1 (raw file):

}

folly::coro::Task<std::unique_ptr<folly::IOBuf>> readFileTask(std::string path, size_t maxBytes) {

Is co_invoke a different way to wrap a sync task as a coro on another exec?


src/admin/ConnectionLogsHandler.cpp line 125 at r1 (raw file):

              .status(503, proxygen::HTTPMessage::getDefaultReason(503))
              .header("Content-Type", "application/json")
              .body(folly::IOBuf::copyBuffer("{\"error\":\"that log type is not configured\"}\n"))

Or maybe there just haven't been any connections yet?


src/admin/ConnectionLogsHandler.cpp line 175 at r1 (raw file):

                    if (!token.isCancellationRequested()) {
                      proxygen::ResponseBuilder(ds)
                          .status(500, proxygen::HTTPMessage::getDefaultReason(500))

I wonder if we want a generic utility in a common header like sendErrorResponse(code, body)

Also I see you are using json bodies -- do we do that elsewhere?


test/CMakeLists.txt line 6 at r1 (raw file):

# Increase the timeout for gtest_discover_tests to prevent flaky build failures
# when linking heavy executables under load.
set(CMAKE_TEST_DISCOVERY_TIMEOUT 60)

Why does test discovery do linking and why does it take 60s? What is the current timeout?


test/CMakeLists.txt line 53 at r1 (raw file):

  moqx_test_main
)
gtest_discover_tests(moqx_relay_test DISCOVERY_TIMEOUT 120)

This seems quite hefty.

@afrind

afrind commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Also, needs update and test failures

@akash-a-n
akash-a-n force-pushed the qlogs/admin-endpoint branch from 02c1462 to 2d47576 Compare August 11, 2026 16:31

@akash-a-n akash-a-n left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akash-a-n made 6 comments.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on afrind).


src/admin/ConnectionLogsHandler.cpp line 54 at r1 (raw file):

Previously, afrind wrote…

There's probably a folly helper

Updated it.


src/admin/ConnectionLogsHandler.cpp line 70 at r1 (raw file):

Previously, afrind wrote…

Is co_invoke a different way to wrap a sync task as a coro on another exec?

simplified to use co_invoke


src/admin/ConnectionLogsHandler.cpp line 125 at r1 (raw file):

Previously, afrind wrote…

Or maybe there just haven't been any connections yet?

'dir' here is the string, and we are checking if a dir name exists in the config is set


src/admin/ConnectionLogsHandler.cpp line 175 at r1 (raw file):

Previously, afrind wrote…

I wonder if we want a generic utility in a common header like sendErrorResponse(code, body)

Also I see you are using json bodies -- do we do that elsewhere?

Json is used in a few other API responses. I have updated it to use the new send_error util in the common file


test/CMakeLists.txt line 6 at r1 (raw file):

Previously, afrind wrote…

Why does test discovery do linking and why does it take 60s? What is the current timeout?

I don't think i wanted to commit any of this code, the tests were flaky and I let copilot fix it on my system back then. I have reverted.


test/CMakeLists.txt line 53 at r1 (raw file):

Previously, afrind wrote…

This seems quite hefty.

removed

@akash-a-n
akash-a-n force-pushed the qlogs/admin-endpoint branch 9 times, most recently from 6456710 to 98739a6 Compare August 12, 2026 14:35
@afrind

afrind commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

The signals here are still red

@afrind afrind mentioned this pull request Aug 12, 2026
@akash-a-n
akash-a-n force-pushed the qlogs/admin-endpoint branch 2 times, most recently from f5acd13 to f48d84e Compare August 17, 2026 10:59

@akash-a-n akash-a-n left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed it

@akash-a-n made 1 comment.
Reviewable status: 2 of 8 files reviewed, 6 unresolved discussions (waiting on afrind).

@afrind afrind left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@afrind reviewed 7 files and all commit messages, and resolved 6 discussions.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on akash-a-n).

@afrind afrind left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I rewrote the handler to stream the file in 64 kb chunks instead of reading it all into memory

@afrind made 1 comment.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on akash-a-n).

akash-a-n and others added 2 commits August 17, 2026 13:09
…ed bugfix

- Implement `GET /logs?type=<mlog|qlog>&connection_id=<hex>` route.
- Offload blocking file I/O (`std::ifstream::read`) to
  `folly::getGlobalCPUExecutor()` to avoid stalling the admin event loop.
- Bugfix: create log dir if not exists
streamLogFile reads 64 KB chunks on the global CPU pool and sends each
from the admin event base, so a disk read overlaps the previous chunk's
network write and the body never lands in the process whole. Headers go
out before the first read, with no Content-Length: the log is still being
appended to, so a length from fstat(2) would be stale by EOF.

kMaxDownloadBytes is now a policy limit on what a client may pull rather
than a memory bound.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@afrind
afrind force-pushed the qlogs/admin-endpoint branch from 37f33ff to 1b044bb Compare August 17, 2026 20:09
@afrind
afrind merged commit 7e4dce4 into main Aug 17, 2026
12 of 13 checks passed
@afrind
afrind deleted the qlogs/admin-endpoint branch August 17, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants