Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 85 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,90 @@
# StudyOS Agent

StudyOS Agent is a University of Tuebingen study assistant shell. The Flutter
app provides the cross-platform UI, while native runners expose platform
features such as Android services, sensors, speech/TTS, reminders, local model
execution, and iOS-safe native APIs where available.
StudyOS Agent is a mobile study assistant for students at the University of
Tübingen. It brings university information into a Flutter application with a
personal overview, a study plan, and a conversational assistant that can use
university tools and display results as inline cards.

## Product Artifact — Course Submission

Developed for **Practical Machine Learning — Build your own Study OS**,
summer semester 2026. This repository documents the product; the group journey
portfolio and individual reflections are separate submissions.

**Start here:** [Product walkthrough](docs/product-walkthrough.md). It explains
the intended workflow and current limitations without requiring installation.
Source code is included for inspection; running the app is optional.

### Who it is for and what problem it addresses

The intended users are University of Tübingen students, particularly those still
learning which university systems contain the information they need. Course
information, schedules, tasks, and deadlines are spread across ALMA, ILIAS,
Moodle, and other services. The product aims to reduce both the navigation
between these systems and the complexity of finding relevant information.

### Main user workflow

1. Connect a university account and complete the student profile.
2. Open **Home** for the next lecture and a personalised information feed.
3. Open **Plan** to inspect the timetable and academic registration overview.
4. Use **Assistant** to ask about schedules, tasks, deadlines, or Mensa options.
The agent can retrieve information through tools and present supported results
as inline cards, alongside its answer.
5. Revisit conversations and edit personal context in **Notes**; manage model
preferences in **Settings**.

The interface was simplified during the project: more functionality moved from
separate pages into assistant tools and inline cards. Home and Plan remain
explicit navigation destinations. See the [walkthrough](docs/product-walkthrough.md)
for a concrete example and links to the corresponding implementation.

### Demo recording

[Watch or download the app demo (MP4, about 2 minutes, 13 MB)](docs/assets/app-demo.mp4).
The recording is included in the repository ZIP and can be opened locally.

### Product screenshots

Captured from the iPhone 17 Pro simulator on 15 September 2026. Click a screen
for the full-size image. These show the prototype's interface and account state;
the Assistant image contains an example conversation dated 2 July 2026.

| Home — next lecture and personal feed | Plan — calendar item and academic status |
| --- | --- |
| [<img src="docs/assets/screenshots/home.png" width="260" alt="StudyOS Home with the next lecture and personalised feed">](docs/assets/screenshots/home.png) | [<img src="docs/assets/screenshots/plan.png" width="260" alt="StudyOS Plan showing an ALMA calendar item and no registrations returned">](docs/assets/screenshots/plan.png) |
| **Assistant — example conversation and tool indicators** | **Settings — profile and model choice** |
| [<img src="docs/assets/screenshots/assistant.png" width="260" alt="StudyOS Assistant with schedule and mail tool indicators and a day-summary table">](docs/assets/screenshots/assistant.png) | [<img src="docs/assets/screenshots/settings.png" width="260" alt="StudyOS Settings with account, appearance, and on-device or custom assistant choices">](docs/assets/screenshots/settings.png) |

### Current state and limitations

This is a course prototype with implemented university integrations and native
Android/iOS code, not a claim of complete feature parity across platforms.

- Personal data requires a university account and working portal sessions.
Changes to university pages or authentication can break integrations.
- Local inference depends on supported devices, operating systems, and model
availability. Cloud inference requires the user's configured provider and key.
- University requests execute on the device. With cloud inference, selected
study context and tool results are sent to the configured AI provider;
choosing cloud mode is not an entirely local data flow.
- Web and desktop builds expose the Flutter interface but do not provide all
native capabilities. A web build is not equivalent to the mobile experience.
- iOS distribution requires signing; macOS builds are not notarized. See the
installation notes below for the available routes.
- Broader user testing, setup simplification, and distribution remain important
next steps. This documentation is based on source inspection and the team's
project account; it does not certify a fresh end-to-end test on every platform.

### Contents of this repository

| Material | Purpose |
| --- | --- |
| [Product walkthrough](docs/product-walkthrough.md) | User scenario, screen descriptions, and limitations |
| [Flutter app](flutter_app/) | Product implementation and native platform runners |
| [Developer instructions](flutter_app/README.md) | Running and inspecting the source |
| [Download page](https://tue-studyos.github.io/StudyOS_Agent/) | Installation entry point |
| [GitHub releases](https://github.com/Tue-StudyOS/StudyOS_Agent/releases) | Published build assets |

## Install A Release

Expand Down
Binary file added docs/assets/app-demo.mp4
Binary file not shown.
Binary file added docs/assets/screenshots/assistant.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/screenshots/home.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/screenshots/plan.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/screenshots/settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
142 changes: 142 additions & 0 deletions docs/product-walkthrough.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# StudyOS Agent — Product Walkthrough

This guide accompanies the course Product Artifact submission. It describes the
implemented interface and an intended user scenario; the example questions are
not a transcript of a recorded test. Actual results depend on the student's
account, available university data, device capabilities, and configured model.

## Demo recording

[Watch or download the app demo](assets/app-demo.mp4) (MP4, about 2 minutes,
13 MB). The file is included in the repository ZIP for offline viewing.

## A student preparing for the day

The student wants to find their next lecture, check upcoming coursework, and
choose somewhere to eat without navigating several university portals.

### 1. Connect the student workspace

The opening screen is titled **StudyOS**, with the subtitle **Connect your
student workspace**. The student enters their university ID or email and
password, selects **Continue**, and completes onboarding. The app uses a
student profile to personalise the experience.

University integrations run from the device rather than through a shared
StudyOS aggregation backend. A university login is needed for personal portal
data; public information such as Mensa menus has separate public sources.

### 2. See what is relevant now

**Home** presents a greeting, the next lecture when timetable data is available,
and a **For you** feed. It also provides access to Profile, Assistant setup,
Notes, Tübingen Talks, and University Mail.

This is the entry point for discovering information without first knowing the
name of a university service or the exact question to ask.

[<img src="assets/screenshots/home.png" width="320" alt="Home screen with next lecture and personalised feed">](assets/screenshots/home.png)

*Home: the next scheduled lecture and a preparation suggestion.*

### 3. Inspect the study plan

The bottom navigation offers **Home**, **Plan**, and **Assistant**. In **Plan**,
the student can inspect timetable information and academic registration status,
refresh data, and access supported calendar/report actions.

This is an overview of information retrieved from university sources, not a
promise that the assistant can enrol a student in a course or submit coursework.
Missing data or a failed refresh should be checked against the source portal.

[<img src="assets/screenshots/plan.png" width="320" alt="Plan screen with calendar item and academic registration status">](assets/screenshots/plan.png)

*Plan: a selected ALMA calendar item. The academic status panel explicitly reports
that no registrations were exposed in this overview; this is not proof that the
student has no registrations.*

### 4. Ask a question and inspect the result

The student opens **Assistant** and asks, for example:

> What deadlines do I have in the next seven days?

The assistant can use the ILIAS/Moodle deadline tools to retrieve relevant data
and answer in the conversation. Supported results can appear as inline UI cards;
the exact presentation depends on the model's response. Calling a tool does not
automatically guarantee that a card will be shown.

A second example is:

> What vegan Mensa options are available today?

The assistant can query public Mensa information rather than relying on a
student to locate and navigate a separate menu page. Timetable, study planner,
and campus information are further examples of tool-backed capabilities.

The central design decision is to expose many capabilities through a compact
conversation instead of giving every tool its own top-level page. Dedicated
views remain available where they make recurring information easier to inspect.

[<img src="assets/screenshots/assistant.png" width="320" alt="Assistant conversation showing tool indicators and a day summary">](assets/screenshots/assistant.png)

*Assistant: an example conversation displaying schedule and mail-deadline tool
indicators. Its answer is dated 2 July 2026, not the screenshot capture date.
This screen illustrates the conversation and table rendering, not an inline tool
card or independent verification of the answer's correctness.*

### 5. Maintain personal context and choose the model

The student can revisit conversations and edit the local memory document through
**Notes**. **Settings → Assistant setup** controls the model configuration.

- **On device:** local model execution requires a compatible device and available
model. Android and iOS use different native integrations.
- **Custom/cloud:** the student configures a provider and API key. Relevant
conversation context and tool results are sent to that provider. Portal
credentials and raw portal sessions are not intended to enter model prompts.

Local inference avoids sending prompts to a cloud model, but retrieving current
university information still requires network access to the university services.

[<img src="assets/screenshots/settings.png" width="320" alt="Settings with profile and assistant provider selection">](assets/screenshots/settings.png)

*Settings: account details, compact-chat preference, and the choice between
on-device and custom model configuration.*

## Current boundaries

The repository contains a Flutter implementation, native Android/iOS runners,
and web/desktop targets. These are not interchangeable deployment experiences:
native actions and local inference depend on the platform. Web builds also run
under browser networking restrictions, so they should not be assumed to support
every direct university integration available on mobile.

The main remaining product questions concern setup complexity, reliability
across student accounts and devices, and wider distribution. Local models add
setup requirements; cloud models introduce provider dependencies and a different
data-sharing choice. Portal parsing needs maintenance when external sites change.

The team simplified the first prototype after feedback that a combined interface
was still too complex. More user testing is needed to assess how well the revised
workflow works for students outside the project group. An assistant plugin is a
possible future direction, not the product submitted in this repository.

## Implementation references

These links let a reviewer connect the described workflow to the source without
having to install the application.

| Part of the experience | Source |
| --- | --- |
| Login | [Login screen](../flutter_app/lib/src/login_page.dart) |
| Navigation | [App routes](../flutter_app/lib/src/app_router.dart) |
| Home overview | [Home view](../flutter_app/lib/src/views/home_view.dart) |
| Study plan | [Schedule view](../flutter_app/lib/src/views/schedule_view.dart) |
| Assistant capabilities | [Tool catalog](../flutter_app/lib/src/studyos_tool_catalog.dart) |
| Inline cards | [Assistant UI payload handling](../flutter_app/lib/src/generated_ui_message.dart) |
| Tasks and deadlines | [University capabilities](../flutter_app/lib/src/private_study_capabilities.dart) |
| Model configuration | [Settings](../flutter_app/lib/src/views/settings_view.dart) |

For installation routes and build requirements, return to the
[repository README](../README.md).
Binary file modified docs/studyos-tool-testing-session.pptx
Binary file not shown.
5 changes: 3 additions & 2 deletions flutter_app/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ mock responses.

The shared Dart catalog exposes public Mensa and Tübingen location tools to
both local and cloud assistants. Authenticated `get_tasks` and `get_deadlines`
tools use ephemeral on-device ILIAS/Moodle sessions and are advertised only to
the local assistant. Passwords, cookies, SAML fields, Moodle session keys, and
tools use ephemeral on-device ILIAS/Moodle sessions and are available to local
and cloud assistants. In cloud mode, sanitized tool results are returned to the
configured model provider. Passwords, cookies, SAML fields, Moodle session keys, and
raw portal HTML must never enter prompts, tool results, traces, or caches.
Loading