diff --git a/README.md b/README.md index 04d586a..916c14d 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,90 @@ # StudyOS Agent -StudyOS Agent is a University of Tuebingen study assistant shell. The Flutter -app provides the cross-platform UI, while native runners expose platform -features such as Android services, sensors, speech/TTS, reminders, local model -execution, and iOS-safe native APIs where available. +StudyOS Agent is a mobile study assistant for students at the University of +Tübingen. It brings university information into a Flutter application with a +personal overview, a study plan, and a conversational assistant that can use +university tools and display results as inline cards. + +## Product Artifact — Course Submission + +Developed for **Practical Machine Learning — Build your own Study OS**, +summer semester 2026. This repository documents the product; the group journey +portfolio and individual reflections are separate submissions. + +**Start here:** [Product walkthrough](docs/product-walkthrough.md). It explains +the intended workflow and current limitations without requiring installation. +Source code is included for inspection; running the app is optional. + +### Who it is for and what problem it addresses + +The intended users are University of Tübingen students, particularly those still +learning which university systems contain the information they need. Course +information, schedules, tasks, and deadlines are spread across ALMA, ILIAS, +Moodle, and other services. The product aims to reduce both the navigation +between these systems and the complexity of finding relevant information. + +### Main user workflow + +1. Connect a university account and complete the student profile. +2. Open **Home** for the next lecture and a personalised information feed. +3. Open **Plan** to inspect the timetable and academic registration overview. +4. Use **Assistant** to ask about schedules, tasks, deadlines, or Mensa options. + The agent can retrieve information through tools and present supported results + as inline cards, alongside its answer. +5. Revisit conversations and edit personal context in **Notes**; manage model + preferences in **Settings**. + +The interface was simplified during the project: more functionality moved from +separate pages into assistant tools and inline cards. Home and Plan remain +explicit navigation destinations. See the [walkthrough](docs/product-walkthrough.md) +for a concrete example and links to the corresponding implementation. + +### Demo recording + +[Watch or download the app demo (MP4, about 2 minutes, 13 MB)](docs/assets/app-demo.mp4). +The recording is included in the repository ZIP and can be opened locally. + +### Product screenshots + +Captured from the iPhone 17 Pro simulator on 15 September 2026. Click a screen +for the full-size image. These show the prototype's interface and account state; +the Assistant image contains an example conversation dated 2 July 2026. + +| Home — next lecture and personal feed | Plan — calendar item and academic status | +| --- | --- | +| [StudyOS Home with the next lecture and personalised feed](docs/assets/screenshots/home.png) | [StudyOS Plan showing an ALMA calendar item and no registrations returned](docs/assets/screenshots/plan.png) | +| **Assistant — example conversation and tool indicators** | **Settings — profile and model choice** | +| [StudyOS Assistant with schedule and mail tool indicators and a day-summary table](docs/assets/screenshots/assistant.png) | [StudyOS Settings with account, appearance, and on-device or custom assistant choices](docs/assets/screenshots/settings.png) | + +### Current state and limitations + +This is a course prototype with implemented university integrations and native +Android/iOS code, not a claim of complete feature parity across platforms. + +- Personal data requires a university account and working portal sessions. + Changes to university pages or authentication can break integrations. +- Local inference depends on supported devices, operating systems, and model + availability. Cloud inference requires the user's configured provider and key. +- University requests execute on the device. With cloud inference, selected + study context and tool results are sent to the configured AI provider; + choosing cloud mode is not an entirely local data flow. +- Web and desktop builds expose the Flutter interface but do not provide all + native capabilities. A web build is not equivalent to the mobile experience. +- iOS distribution requires signing; macOS builds are not notarized. See the + installation notes below for the available routes. +- Broader user testing, setup simplification, and distribution remain important + next steps. This documentation is based on source inspection and the team's + project account; it does not certify a fresh end-to-end test on every platform. + +### Contents of this repository + +| Material | Purpose | +| --- | --- | +| [Product walkthrough](docs/product-walkthrough.md) | User scenario, screen descriptions, and limitations | +| [Flutter app](flutter_app/) | Product implementation and native platform runners | +| [Developer instructions](flutter_app/README.md) | Running and inspecting the source | +| [Download page](https://tue-studyos.github.io/StudyOS_Agent/) | Installation entry point | +| [GitHub releases](https://github.com/Tue-StudyOS/StudyOS_Agent/releases) | Published build assets | ## Install A Release diff --git a/docs/assets/app-demo.mp4 b/docs/assets/app-demo.mp4 new file mode 100644 index 0000000..0cda557 Binary files /dev/null and b/docs/assets/app-demo.mp4 differ diff --git a/docs/assets/screenshots/assistant.png b/docs/assets/screenshots/assistant.png new file mode 100644 index 0000000..1b0eaa7 Binary files /dev/null and b/docs/assets/screenshots/assistant.png differ diff --git a/docs/assets/screenshots/home.png b/docs/assets/screenshots/home.png new file mode 100644 index 0000000..3c2e328 Binary files /dev/null and b/docs/assets/screenshots/home.png differ diff --git a/docs/assets/screenshots/plan.png b/docs/assets/screenshots/plan.png new file mode 100644 index 0000000..36ac2eb Binary files /dev/null and b/docs/assets/screenshots/plan.png differ diff --git a/docs/assets/screenshots/settings.png b/docs/assets/screenshots/settings.png new file mode 100644 index 0000000..0735afa Binary files /dev/null and b/docs/assets/screenshots/settings.png differ diff --git a/docs/product-walkthrough.md b/docs/product-walkthrough.md new file mode 100644 index 0000000..79c3099 --- /dev/null +++ b/docs/product-walkthrough.md @@ -0,0 +1,142 @@ +# StudyOS Agent — Product Walkthrough + +This guide accompanies the course Product Artifact submission. It describes the +implemented interface and an intended user scenario; the example questions are +not a transcript of a recorded test. Actual results depend on the student's +account, available university data, device capabilities, and configured model. + +## Demo recording + +[Watch or download the app demo](assets/app-demo.mp4) (MP4, about 2 minutes, +13 MB). The file is included in the repository ZIP for offline viewing. + +## A student preparing for the day + +The student wants to find their next lecture, check upcoming coursework, and +choose somewhere to eat without navigating several university portals. + +### 1. Connect the student workspace + +The opening screen is titled **StudyOS**, with the subtitle **Connect your +student workspace**. The student enters their university ID or email and +password, selects **Continue**, and completes onboarding. The app uses a +student profile to personalise the experience. + +University integrations run from the device rather than through a shared +StudyOS aggregation backend. A university login is needed for personal portal +data; public information such as Mensa menus has separate public sources. + +### 2. See what is relevant now + +**Home** presents a greeting, the next lecture when timetable data is available, +and a **For you** feed. It also provides access to Profile, Assistant setup, +Notes, Tübingen Talks, and University Mail. + +This is the entry point for discovering information without first knowing the +name of a university service or the exact question to ask. + +[Home screen with next lecture and personalised feed](assets/screenshots/home.png) + +*Home: the next scheduled lecture and a preparation suggestion.* + +### 3. Inspect the study plan + +The bottom navigation offers **Home**, **Plan**, and **Assistant**. In **Plan**, +the student can inspect timetable information and academic registration status, +refresh data, and access supported calendar/report actions. + +This is an overview of information retrieved from university sources, not a +promise that the assistant can enrol a student in a course or submit coursework. +Missing data or a failed refresh should be checked against the source portal. + +[Plan screen with calendar item and academic registration status](assets/screenshots/plan.png) + +*Plan: a selected ALMA calendar item. The academic status panel explicitly reports +that no registrations were exposed in this overview; this is not proof that the +student has no registrations.* + +### 4. Ask a question and inspect the result + +The student opens **Assistant** and asks, for example: + +> What deadlines do I have in the next seven days? + +The assistant can use the ILIAS/Moodle deadline tools to retrieve relevant data +and answer in the conversation. Supported results can appear as inline UI cards; +the exact presentation depends on the model's response. Calling a tool does not +automatically guarantee that a card will be shown. + +A second example is: + +> What vegan Mensa options are available today? + +The assistant can query public Mensa information rather than relying on a +student to locate and navigate a separate menu page. Timetable, study planner, +and campus information are further examples of tool-backed capabilities. + +The central design decision is to expose many capabilities through a compact +conversation instead of giving every tool its own top-level page. Dedicated +views remain available where they make recurring information easier to inspect. + +[Assistant conversation showing tool indicators and a day summary](assets/screenshots/assistant.png) + +*Assistant: an example conversation displaying schedule and mail-deadline tool +indicators. Its answer is dated 2 July 2026, not the screenshot capture date. +This screen illustrates the conversation and table rendering, not an inline tool +card or independent verification of the answer's correctness.* + +### 5. Maintain personal context and choose the model + +The student can revisit conversations and edit the local memory document through +**Notes**. **Settings → Assistant setup** controls the model configuration. + +- **On device:** local model execution requires a compatible device and available + model. Android and iOS use different native integrations. +- **Custom/cloud:** the student configures a provider and API key. Relevant + conversation context and tool results are sent to that provider. Portal + credentials and raw portal sessions are not intended to enter model prompts. + +Local inference avoids sending prompts to a cloud model, but retrieving current +university information still requires network access to the university services. + +[Settings with profile and assistant provider selection](assets/screenshots/settings.png) + +*Settings: account details, compact-chat preference, and the choice between +on-device and custom model configuration.* + +## Current boundaries + +The repository contains a Flutter implementation, native Android/iOS runners, +and web/desktop targets. These are not interchangeable deployment experiences: +native actions and local inference depend on the platform. Web builds also run +under browser networking restrictions, so they should not be assumed to support +every direct university integration available on mobile. + +The main remaining product questions concern setup complexity, reliability +across student accounts and devices, and wider distribution. Local models add +setup requirements; cloud models introduce provider dependencies and a different +data-sharing choice. Portal parsing needs maintenance when external sites change. + +The team simplified the first prototype after feedback that a combined interface +was still too complex. More user testing is needed to assess how well the revised +workflow works for students outside the project group. An assistant plugin is a +possible future direction, not the product submitted in this repository. + +## Implementation references + +These links let a reviewer connect the described workflow to the source without +having to install the application. + +| Part of the experience | Source | +| --- | --- | +| Login | [Login screen](../flutter_app/lib/src/login_page.dart) | +| Navigation | [App routes](../flutter_app/lib/src/app_router.dart) | +| Home overview | [Home view](../flutter_app/lib/src/views/home_view.dart) | +| Study plan | [Schedule view](../flutter_app/lib/src/views/schedule_view.dart) | +| Assistant capabilities | [Tool catalog](../flutter_app/lib/src/studyos_tool_catalog.dart) | +| Inline cards | [Assistant UI payload handling](../flutter_app/lib/src/generated_ui_message.dart) | +| Tasks and deadlines | [University capabilities](../flutter_app/lib/src/private_study_capabilities.dart) | +| Model configuration | [Settings](../flutter_app/lib/src/views/settings_view.dart) | + +For installation routes and build requirements, return to the +[repository README](../README.md). diff --git a/docs/studyos-tool-testing-session.pptx b/docs/studyos-tool-testing-session.pptx index da04bda..f7c8dc9 100644 Binary files a/docs/studyos-tool-testing-session.pptx and b/docs/studyos-tool-testing-session.pptx differ diff --git a/flutter_app/README.md b/flutter_app/README.md index 6f32877..3deb8f1 100644 --- a/flutter_app/README.md +++ b/flutter_app/README.md @@ -65,6 +65,7 @@ mock responses. The shared Dart catalog exposes public Mensa and Tübingen location tools to both local and cloud assistants. Authenticated `get_tasks` and `get_deadlines` -tools use ephemeral on-device ILIAS/Moodle sessions and are advertised only to -the local assistant. Passwords, cookies, SAML fields, Moodle session keys, and +tools use ephemeral on-device ILIAS/Moodle sessions and are available to local +and cloud assistants. In cloud mode, sanitized tool results are returned to the +configured model provider. Passwords, cookies, SAML fields, Moodle session keys, and raw portal HTML must never enter prompts, tool results, traces, or caches.