Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
0a5952b
feat(genai): add thinking mode, web search, multimodal attachments, a…
Water-Buckets Oct 20, 2026
bfc95dd
feat(genai): record microphone audio for STT with live waveform and W…
Water-Buckets Oct 20, 2026
edadaf5
fix(genai): prevent image upload failures from triggering unwanted se…
Water-Buckets Oct 20, 2026
727c704
fix(genai): preserve image attachment in user bubble and chat request…
Water-Buckets Oct 20, 2026
f5cab37
fix(genai): fix image upload multipart parameters, prevent sending fa…
Water-Buckets Oct 20, 2026
5d7d8c4
feat(genai): add image thumbnail in composer attachment chip with ful…
Water-Buckets Oct 20, 2026
1677d88
feat(genai): add retry, inline edit, turn revert, syntax highlighting…
Water-Buckets Oct 20, 2026
1c2960c
fix(genai): handle empty group IDs in history and renew group ID on r…
Water-Buckets Oct 20, 2026
899dbbb
fix(genai): allow adequate timeout for synchronous image generation
Water-Buckets Oct 20, 2026
3242005
feat(genai): bundle RobotoMono font and configure monospace family fo…
Water-Buckets Oct 20, 2026
bcef45d
fix(genai): improve blockquote readability in dark mode and add markd…
Water-Buckets Oct 20, 2026
667ddd2
feat(genai): show realtime search results in session search
Water-Buckets Oct 21, 2026
e05f31e
feat(genai): disable message deletion, reversion, and inline edits
Water-Buckets Oct 21, 2026
815fca1
feat(genai): enlarge default font size, disable regeneration, and str…
Water-Buckets Oct 21, 2026
2bc2405
chore(genai): simplify controller, remove dead code, and clean up red…
Water-Buckets Oct 21, 2026
135c19e
Merge branch 'HeZeBang:master' into codex/genai-foundation
Neo-Super-Express Oct 21, 2026
b915a52
feat(genai): add campus agent workflows and durable conversations
Water-Buckets Oct 23, 2026
4089948
Merge origin/codex/genai-foundation into codex/genai-foundation
Water-Buckets Oct 23, 2026
9f1dc73
chore(genai): merge upstream and align Flutter compatibility
Water-Buckets Oct 24, 2026
7df5ed1
feat(genai): add recoverable compaction and refine chat workspace
Water-Buckets Oct 24, 2026
58e8028
refactor(genai): align chat UI with TechPie and simplify image controls
Water-Buckets Oct 24, 2026
c06ace0
refactor(genai): compact responsive image generation controls
Water-Buckets Oct 25, 2026
9dd1210
feat(genai): refine workspace and image studio UX
Water-Buckets Oct 25, 2026
4e48582
fix(genai): guard reservations and refine responsive layouts
Water-Buckets Oct 25, 2026
b4ae086
refactor(genai): recover chat and agent state from school history
Water-Buckets Oct 27, 2026
f5038b4
fix(genai): preserve stopped turns and simplify runtime state
Water-Buckets Oct 27, 2026
034c50c
chore: merge upstream master into genai foundation
Water-Buckets Oct 27, 2026
ccf92cd
feat(genai): streamline recovery records and image actions
Water-Buckets Oct 27, 2026
1898247
fix(genai): verify timetable ownership and simplify code rendering
Water-Buckets Oct 27, 2026
7821ad6
chore(genai): remove bundled demo image fixture
Water-Buckets Oct 27, 2026
78a5f1f
chore(genai): remove preview wiring and prune test scaffolding
Water-Buckets Oct 27, 2026
9d73546
feat(genai): harden agent sessions and conversation controls
Water-Buckets Oct 29, 2026
b5d8470
Add GenAI foundation flow and native platform setup
Water-Buckets Oct 30, 2026
72ec35c
feat(genai): integrate native draft A workspace and interactions
Water-Buckets Nov 1, 2026
6e5e0df
merge: sync GenAI with upstream master
Water-Buckets Nov 1, 2026
723a630
merge: sync GenAI with upstream master
Water-Buckets Nov 3, 2026
c91f8fd
refactor(genai): harden agent recovery and unify adaptive UI
Water-Buckets Nov 3, 2026
978a2b0
feat(genai): streamline chat controls and harden agent recovery
Water-Buckets Nov 4, 2026
03eea3f
merge: sync GenAI with upstream master
Water-Buckets Nov 4, 2026
920d115
merge: sync GenAI with upstream master
Water-Buckets Nov 13, 2026
f0edef2
refactor(genai): simplify state and trim redundant tests
Water-Buckets Nov 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/linux-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ jobs:
cmake \
ninja-build \
pkg-config \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
libgtk-3-dev \
liblzma-dev \
libwebkit2gtk-4.1-dev \
Expand Down
10 changes: 10 additions & 0 deletions android/app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.CAMERA"/>
<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-feature android:name="android.hardware.microphone" android:required="false"/>
<uses-permission android:name="android.permission.VIBRATE"/>
<uses-permission android:name="android.permission.READ_CALENDAR"/>
<uses-permission android:name="android.permission.WRITE_CALENDAR"/>
Expand All @@ -12,6 +14,14 @@
android:icon="@mipmap/ic_launcher"
android:allowBackup="false"
android:usesCleartextTraffic="false">
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.provider"
android:exported="false"
android:grantUriPermissions="true">
<meta-data android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/genai_clipboard_paths"/>
</provider>
<activity
android:name=".MainActivity"
android:exported="true"
Expand Down
4 changes: 4 additions & 0 deletions android/app/src/main/res/xml/genai_clipboard_paths.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<paths>
<cache-path name="clipboard_images" path="."/>
</paths>
314 changes: 314 additions & 0 deletions docs/genai-agent-harness.md

Large diffs are not rendered by default.

153 changes: 153 additions & 0 deletions docs/genai-api-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
# GenAI integration contract

The school API is not versioned by this client. These contracts reflect the
implementation and observed server behavior; model availability, pricing and
limits can change independently. The [agent guide](genai-agent-harness.md) covers
framing, tools, context and server-only recovery.

## Integration and identity

`GenAiRuntime` is wired lazily through `ServiceProvider`. Reuse existing TechPie
services for authentication, timetable, assignments and booking. Leaving the page
cancels active agent work while retaining the transient workspace.

GenAI requires the primary TechPie identity and a campus binding. The verified
profile username must match the binding's student ID, not the primary GeekPie ID.
Ownership includes primary user ID, campus ID and binding time; renewal preserves
it, while rebinding/account changes invalidate it. Async requests, uploads, pickers,
audio and history reads must not apply results to a different owner or chat.

`CampusGenAiSessionIssuer` reuses CpDaily/IDS and `SessionTree.withCookie`. Native
SSO uses an isolated cookie jar; CASTGC stays on IDS and redirects require allowed
university HTTPS origins. Profile verification precedes token persistence. Trusted
IDS exchange may resolve an initially unknown SMS-binding ID; pasted tokens cannot
bootstrap identity. TechPie's existing binding UI handles credentials.

Credentials use the OHOS-aware secure store, outside cloud sync. Cached tokens are
reverified; concurrent issuance is shared and store operations serialized. Logout
clears memory immediately and queues deletion, surfacing failures. A stale rejected
request cannot invalidate a newer login. Authentication errors never automatically
replay a generation.

## Requests and streaming

Base: `https://genai.shanghaitech.edu.cn/htk/`. Injected bases require HTTPS except
loopback fixtures. API calls do not follow redirects. Exclude payloads, credential
URLs and raw upstream errors from logs.

| Operation | Wire contract |
| --- | --- |
| Profile | `GET user/info/{token}`, `X-Access-Token`; verify `userInfo.id` and `username` |
| Models | Merge `GET ai/aiModel/list` with `ai/aiModel/list/V2`; keep routes/status and refreshed thinking/pricing metadata |
| Chat | `POST chat/start/chat`; stable `chatGroupId`, latest input in `chatInfo`, context in `messages`, `type: "3"`, `stream: true` |
| Document | `POST chat/upload/chat/file?chatGroupId=...&aiType=...`; multipart `biz=temp`, `file` |
| Vision upload | Image host `/sys/common/upload`; canonical campus CDN URL |
| Generation | JSON `chat/start/chat`, `aiSecType: "1"`, `count: "1"`, PNG, size/quality and optional references |
| Usage | Signed `ai-api-gateway/my/summary` and `ai-trade-order/my/list` |
| Session metadata | `POST ai/history/group/rename` (`chatGroupId`, `title`), `group/pin` (`chatGroupIds`, `pinned`), `group/delete` (`chatGroupIds`) |
| Message deletion | `GET ai/history/delete/batch?ids=...`; comma-separated server row `id` values, not logical turn IDs |
| Deletion | `GET ai/history/delete/groupId?id=...` or `DELETE ai-image-history/delete?id=...`; explicit confirmation |

Catalog V2 may return the full registry rather than a curated subset. Skip rows
without explicit routing identifiers, retain valid legacy metadata when optional
V2 fields are null, and admit new V2 routes only for chat/image statuses 3/2.
Utility models do not belong in the chat picker; never infer a missing provider.

JSON envelopes require successful status/code; image generation also accepts its
raw response variant. Authentication expiry, rejection, transport, protocol and
storage failures remain distinguishable and sanitized. The signing salt is public
protocol configuration (`TECHPI_GENAI_SIGNING_SALT` override), never an account secret.

An HTTP 404 HTML response displaying `该服务暂不开放外网访问,仅支持校内访问`
means the campus ingress blocked the request: preserve the login and ask the user
to connect to the campus network or university VPN. Ordinary 404s remain request
failures; HTTP 401 remains authentication expiry. Never replay a generation POST.

Host instructions precede prior context; agent requests also append a concise
protocol guide immediately before the current input. Both modes use the catalog's
`enableDeepThink` capability and selected thinking flag, and force `netGo: false`.
There is no search preflight/tool; historical citations remain readable.

SSE supports incremental UTF-8, CR/LF/CRLF, comments and multiline data with bounded
event sizes. Reasoning and answer deltas stay separate. A model finish reason is
confirmed by `[DONE]`, a school `other` receipt, `isEnd`/`is_end`, or a finish reason at EOF for non-Azure providers. Azure accepts moderation-only
events and waits beyond `[DONE]` for the school `other` receipt with `answerGroupId`.
EOF before that receipt is interruption; an intervening save-error envelope fails
the turn. Non-Azure EOF without a finish reason is interruption. Malformed events fail visibly. Cancellation aborts local work without promising server rollback or
avoided billing. No automatic generation POST retry or stream reconnection occurs.

The server's `chatInfo` ceiling is 900,000 UTF-16 characters, including the recovery
envelope; it does not limit `messages`. Attachment-only prompts use `请分析所附内容。`.
The official frontend's `messageRound` window is not applied. Known model windows
follow the updated reference: GPT-6.x 1,050,000, Kimi 1,024,000, DeepSeek/GLM/Qwen
524,288 tokens. Unknown deployments retain the configurable 256k fallback.
Compaction keeps original records on the server. A V2 `longContextTokenThreshold`
is a billing threshold, never an input cap. History JSON reads retain a 64 MiB
resource bound rather than the previous 2 MiB ceiling.

The external `genaiapi.shanghaitech.edu.cn` gateway is a separate, API-key-authenticated
contract. IDS credentials and `isWeb: 1` keys must not be reused there; model aliases
and native tool protocols also differ. TechPie's IDS-backed conversations continue
through `/htk/` and the school's history API. No silent gateway migration occurs.

## Documents and images

Documents accept TXT, PDF, DOCX, XLSX, XLS and CSV. Recognized UTF-8 code/Markdown
keeps its display name and bytes but uses a `.txt` wire name. Legacy DOC,
PowerPoint and disguised binary text are rejected. Upload limits are 10 MiB and
500,000 extracted characters. Native `file_selector` checks size before and during
reading; web uses the same filename/content validation. Cancellation returns no
attachment. macOS requires the user-selected read-only entitlement.

Upload responses require string `content`: empty means known-empty extraction;
missing/non-string means protocol failure. Failed uploads block sending until
removed/retried. Recovered file IDs without extraction require a new chat and
re-upload: re-uploading into the same group cannot remove unknown earlier IDs.
Both modes carry deduplicated references through follow-ups, retries and reopening.
Extraction counts in full toward context, without duplicate instruction copies.
Archived IDs do not guarantee server availability; expired files need re-upload.

Vision history accepts URL arrays and legacy comma-separated `imageUrl`. Store
canonical validated CDN URLs in the turn record and tool receipts. For inference,
fetch the active image selection without credentials and put base64 image content
blocks only in `messages`, before the tool guide. Do not put bytes in `chatInfo`
or the top-level image fields: those fields can fail server persistence after
inference succeeds. Historical image-field extensions are not reliable vision
input. Downloads are cancellable and bounded to 30 seconds, 10 MiB per image and
25 MiB per request; failure stops before the chat POST. Reopening rebuilds this
request from server records without local image storage.

Chat, reasoning and tool images share the trusted-URL policy; external/data/file images
do not load automatically. Chat, gallery and viewer share explicit image actions and desktop right-click
menus; long-press does not open a menu. Copy/share capability varies by platform; Android
uses a non-exported cache FileProvider and iPad sharing needs source bounds.

Image dimensions must be positive integers. Presets are conveniences; the school
frontend's pixel range, experimental-size warning, four-reference count and
reference-byte ceilings are not imposed on generation. The selected service
validates model-specific constraints. References remain local until generation:
validated PNG/JPEG/WebP. One uses `imageBase64`, several use
`imageBase64List`; unused fields are null. Confirmation includes parameters and
quota implications. Ambiguous outcomes are never automatically regenerated.

## Platform boundaries and verification

Native and web implement chat, history, images, usage and agents. Native document
selection is implemented; native audio capture/playback remains unavailable.
Web audio depends on browser capability and permission. Production web SSO needs
a bridge: browser JavaScript cannot inject IDS cookies. No development login
proxy or seeded academic data is bundled.

Run `flutter analyze --no-pub` and `flutter test --no-pub` with the repository SDK.
Keep regression coverage for framing/stream boundaries, ownership races,
confirmations, unknown mutations, document extraction, server-only recovery and
compaction provenance. Keep one-off screenshots, demo fixtures, diagnostics and
live-run reports outside production changes.

The browser transport test needs an explicit local fixture: run
`python3 test/genai/support/stream_fixture.py --port 18765`, then
`flutter test --no-pub --platform chrome --dart-define=GENAI_TEST_ORIGIN=http://127.0.0.1:18765 test/genai/platform/http_genai_transport_browser_test.dart`.
Synthetic tests do not establish current school-service availability.

The vision upload gateway token can be overridden at build time with
`--dart-define=TECHPI_GENAI_VISION_TOKEN=...`; existing installations require an update.
24 changes: 24 additions & 0 deletions docs/genai-networking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# GenAI networking

The GenAI branch uses Flutter 3.41.9 / Dart 3.11.5 in CI. The Apple
networking dependencies require Dart 3.10 or newer.

On iOS and macOS, `createGenAiTransport` selects an ephemeral URLSession
through the official `cupertino_http` bindings. The IDS exchange and GenAI API
use the same transport factory. Other native platforms keep IOClient; web
keeps BrowserClient.

Each operation owns its native task/session. Response headers and data chunks
are delivered as they arrive; cancellation stops the native task even before
headers. Separate metadata, chat-prefill and stream-idle timers retain the
existing budgets. Sessions are invalidated after completion or cancellation.

Automatic redirects and cookie injection are disabled. The IDS exchange
validates every hop, maintains a per-exchange cookie jar, scopes CASTGC to IDS,
and intercepts the GenAI token callback before requesting it. Apple networking
uses the operating system proxy/VPN configuration and standard TLS verification.

Machine-specific launch files, proxy endpoints, native probes, live-test entry
points, credentials and recordings belong outside the repository. They must
not be staged or committed. Production regression tests use fakes or loopback
servers, never real credentials or campus API calls.
42 changes: 33 additions & 9 deletions ios/Podfile.lock
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
PODS:
- audioplayers_darwin (0.0.1):
- Flutter
- FlutterMacOS
- CocoaLumberjack (3.10.0):
- CocoaLumberjack/Core (= 3.10.0)
- CocoaLumberjack/Core (3.10.0)
Expand All @@ -7,6 +10,8 @@ PODS:
- cupertino_native_better (0.0.1):
- Flutter
- SVGKit (~> 3.0)
- file_selector_ios (0.0.1):
- Flutter
- Flutter (1.0.0)
- flutter_inappwebview_ios (0.0.1):
- Flutter
Expand All @@ -27,11 +32,14 @@ PODS:
- OrderedSet (6.0.3)
- package_info_plus (0.4.5):
- Flutter
- path_provider_foundation (0.0.1):
- pasteboard (0.0.1):
- Flutter
- record_ios (1.2.1):
- Flutter
- FlutterMacOS
- screen_brightness_ios (0.1.0):
- Flutter
- share_plus (0.0.1):
- Flutter
- shared_preferences_foundation (0.0.1):
- Flutter
- FlutterMacOS
Expand All @@ -44,17 +52,21 @@ PODS:
- FlutterMacOS

DEPENDENCIES:
- audioplayers_darwin (from `.symlinks/plugins/audioplayers_darwin/darwin`)
- connectivity_plus (from `.symlinks/plugins/connectivity_plus/ios`)
- cupertino_native_better (from `.symlinks/plugins/cupertino_native_better/ios`)
- file_selector_ios (from `.symlinks/plugins/file_selector_ios/ios`)
- Flutter (from `Flutter`)
- flutter_inappwebview_ios (from `.symlinks/plugins/flutter_inappwebview_ios/ios`)
- flutter_secure_storage (from `.symlinks/plugins/flutter_secure_storage/ios`)
- image_picker_ios (from `.symlinks/plugins/image_picker_ios/ios`)
- mobile_scanner (from `.symlinks/plugins/mobile_scanner/darwin`)
- open_filex (from `.symlinks/plugins/open_filex/ios`)
- package_info_plus (from `.symlinks/plugins/package_info_plus/ios`)
- path_provider_foundation (from `.symlinks/plugins/path_provider_foundation/darwin`)
- pasteboard (from `.symlinks/plugins/pasteboard/ios`)
- record_ios (from `.symlinks/plugins/record_ios/ios`)
- screen_brightness_ios (from `.symlinks/plugins/screen_brightness_ios/ios`)
- share_plus (from `.symlinks/plugins/share_plus/ios`)
- shared_preferences_foundation (from `.symlinks/plugins/shared_preferences_foundation/darwin`)
- url_launcher_ios (from `.symlinks/plugins/url_launcher_ios/ios`)
- webview_flutter_wkwebview (from `.symlinks/plugins/webview_flutter_wkwebview/darwin`)
Expand All @@ -66,10 +78,14 @@ SPEC REPOS:
- SVGKit

EXTERNAL SOURCES:
audioplayers_darwin:
:path: ".symlinks/plugins/audioplayers_darwin/darwin"
connectivity_plus:
:path: ".symlinks/plugins/connectivity_plus/ios"
cupertino_native_better:
:path: ".symlinks/plugins/cupertino_native_better/ios"
file_selector_ios:
:path: ".symlinks/plugins/file_selector_ios/ios"
Flutter:
:path: Flutter
flutter_inappwebview_ios:
Expand All @@ -84,10 +100,14 @@ EXTERNAL SOURCES:
:path: ".symlinks/plugins/open_filex/ios"
package_info_plus:
:path: ".symlinks/plugins/package_info_plus/ios"
path_provider_foundation:
:path: ".symlinks/plugins/path_provider_foundation/darwin"
pasteboard:
:path: ".symlinks/plugins/pasteboard/ios"
record_ios:
:path: ".symlinks/plugins/record_ios/ios"
screen_brightness_ios:
:path: ".symlinks/plugins/screen_brightness_ios/ios"
share_plus:
:path: ".symlinks/plugins/share_plus/ios"
shared_preferences_foundation:
:path: ".symlinks/plugins/shared_preferences_foundation/darwin"
url_launcher_ios:
Expand All @@ -96,22 +116,26 @@ EXTERNAL SOURCES:
:path: ".symlinks/plugins/webview_flutter_wkwebview/darwin"

SPEC CHECKSUMS:
audioplayers_darwin: 835ced6edd4c9fc8ebb0a7cc9e294a91d99917d5
CocoaLumberjack: 12545b2899dcaeec69b82c27b174042259c1ec27
connectivity_plus: cb623214f4e1f6ef8fe7403d580fdad517d2f7dd
cupertino_native_better: 48a6a9cd45efc9ea9e1eec30755b5a8ebe92f1f2
file_selector_ios: f92e583d43608aebc2e4a18daac30b8902845502
Flutter: cabc95a1d2626b1b06e7179b784ebcf0c0cde467
flutter_inappwebview_ios: b89ba3482b96fb25e00c967aae065701b66e9b99
flutter_secure_storage: 1ed9476fba7e7a782b22888f956cce43e2c62f13
image_picker_ios: 7fe1ff8e34c1790d6fff70a32484959f563a928a
image_picker_ios: e0ece4aa2a75771a7de3fa735d26d90817041326
mobile_scanner: 9157936403f5a0644ca3779a38ff8404c5434a93
open_filex: 432f3cd11432da3e39f47fcc0df2b1603854eff1
OrderedSet: e539b66b644ff081c73a262d24ad552a69be3a94
package_info_plus: 566e1b7a2f3900e4b0020914ad3fc051dcc95596
path_provider_foundation: 080d55be775b7414fd5a5ef3ac137b97b097e564
pasteboard: 49088aeb6119d51f976a421db60d8e1ab079b63c
record_ios: 980fd386a97a35987d0fce3dfda4b26a38c90f4c
screen_brightness_ios: 9953fd7da5bd480f1a93990daeec2eb42d4f3b52
shared_preferences_foundation: 9e1978ff2562383bd5676f64ec4e9aa8fa06a6f7
share_plus: 50da8cb520a8f0f65671c6c6a99b3617ed10a58a
shared_preferences_foundation: 7036424c3d8ec98dfe75ff1667cb0cd531ec82bb
SVGKit: 1ad7513f8c74d9652f94ed64ddecda1a23864dea
url_launcher_ios: 694010445543906933d732453a59da0a173ae33d
url_launcher_ios: 7a95fa5b60cc718a708b8f2966718e93db0cef1b
webview_flutter_wkwebview: 1821ceac936eba6f7984d89a9f3bcb4dea99ebb2

PODFILE CHECKSUM: 73a16e1b485e02cb0dd114d4fd0bdfbd1811e49c
Expand Down
2 changes: 2 additions & 0 deletions ios/Runner/Info.plist
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
<dict>
<key>TechPieDeadlineAppGroup</key><string>$(TECHPIE_DEADLINE_APP_GROUP)</string>
<key>TechPieWidgetURLScheme</key><string>$(TECHPIE_WIDGET_URL_SCHEME)</string>
<key>NSMicrophoneUsageDescription</key>
<string>使用麦克风将语音转为聊天草稿。</string>
<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>
<key>CFBundleDevelopmentRegion</key>
Expand Down
Loading