Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .agents/docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
Interoperable Signal products (see [issue #10](https://github.com/BreadchainCoop/sigstack-bot/issues/10)):

1. **Voice transcription** — own CVM (`BOT__ROLE=transcription`)
2. **In-chat group translation** — translation CVM (`BOT__ROLE=translation`)
3. **Parallel Translation** — planned on translation CVM
4. **Language Threads** — translation CVM (see [`docs/language-threads.md`](../docs/language-threads.md))
2. **In-chat group translation** — translation CVM (see [`docs/in-chat-translation.md`](../../docs/in-chat-translation.md))
3. **Parallel Translation** — translation CVM MVP (see [`docs/parallel-translation.md`](../../docs/parallel-translation.md))
4. **Language Threads** — translation CVM (see [`docs/language-threads.md`](../../docs/language-threads.md))

Architecture overview: [`docs/two-cvm-architecture.md`](../docs/two-cvm-architecture.md).
Architecture overview: [`docs/two-cvm-architecture.md`](../../docs/two-cvm-architecture.md).

Fork legacy removed: general AI chat, tool use (`crates/tools`), x402 payments, in-memory conversation store.

Expand Down
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ docker compose -f docker/compose.translation.yaml --env-file docker/translation.
|-----|-----|
| [`.agents/docs/DEVELOPMENT.md`](.agents/docs/DEVELOPMENT.md) | TEE trust model, `BOT__ROLE`, Phala dual-CVM ops |
| [`docs/two-cvm-architecture.md`](docs/two-cvm-architecture.md) | Architecture diagram and compose/Phala split |
| [`docs/in-chat-translation.md`](docs/in-chat-translation.md) | In-chat (group) bilingual auto/manual translate |
| [`docs/parallel-translation.md`](docs/parallel-translation.md) | Parallel Translation product behavior |
| [`docs/language-threads.md`](docs/language-threads.md) | Language Threads product behavior |
| [`.agents/skills/`](.agents/skills/) | Vendored skills (Rust, Docker, Stripe) |
| [`.agents/rules/`](.agents/rules/) | Agent rules (e.g. commit message subjects) |
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ Not a general AI chat assistant. Conversation history, tool-calling, and x402 cr
| Voice transcription | `BOT__ROLE=transcription` | Own CVM / Compose stack (with Whisper) |
| In-chat (group) translation | `BOT__ROLE=translation` | Shared translation CVM |
| Language Threads | `BOT__ROLE=translation` | Shared translation CVM |
| Parallel Translation | planned | Shared translation CVM |
| Parallel Translation | `BOT__ROLE=translation` (MVP) | Shared translation CVM |

Pair products by adding **both bots** (two phone numbers) to the same Signal group. Signal is the bus — there is no Docker network between CVMs.

Details: [docs/two-cvm-architecture.md](docs/two-cvm-architecture.md) · [docs/language-threads.md](docs/language-threads.md)
Details: [docs/two-cvm-architecture.md](docs/two-cvm-architecture.md) · [docs/in-chat-translation.md](docs/in-chat-translation.md) · [docs/parallel-translation.md](docs/parallel-translation.md) · [docs/language-threads.md](docs/language-threads.md)

## Architecture

Expand Down
104 changes: 7 additions & 97 deletions crates/signal-bot/src/commands/translate_service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,6 @@ use whatlang::Lang;

const MIN_DETECT_CONFIDENCE: f64 = 0.2;

/// Resolve source/target languages for `!translate-on` on a voice note.
pub fn resolve_translate_all_voice_pair(
mode: &GroupTranslateMode,
whisper_lang: Option<&str>,
transcript: &str,
) -> Option<(&'static Language, &'static Language)> {
for code in voice_language_candidates(whisper_lang, transcript) {
if let Some(normalized) = normalize_for_translate_all_pair(mode, &code) {
if let (Some(target), Some(source)) = (
mode.target_for_source(&normalized),
mode.source_language(&normalized),
) {
return Some((source, target));
}
}
}
None
}

/// Map a detected code into one side of the active pair when possible.
fn normalize_for_translate_all_pair(mode: &GroupTranslateMode, code: &str) -> Option<String> {
let code = code.to_lowercase();
Expand All @@ -43,27 +24,7 @@ fn normalize_for_translate_all_pair(mode: &GroupTranslateMode, code: &str) -> Op
None
}

fn voice_language_candidates(whisper_lang: Option<&str>, transcript: &str) -> Vec<String> {
let mut codes = Vec::new();
let mut push = |code: &str| {
if !codes.iter().any(|c| c == code) {
codes.push(code.to_string());
}
};

if let Some(lang) = whisper_lang {
push(lang);
}
if let Some(lang) = detect_text_language_voice(transcript) {
push(&lang);
}
for hint in casual_language_hints(transcript) {
push(hint);
}
codes
}

/// Like [`detect_text_language`] but tuned for short Whisper transcripts.
/// Like [`detect_text_language`] but tuned for short / casual messages.
fn detect_text_language_voice(text: &str) -> Option<String> {
const MIN_VOICE_CONFIDENCE: f64 = 0.08;

Expand Down Expand Up @@ -222,23 +183,6 @@ pub fn format_text_auto_translation(target: &Language, translation: &str) -> Str
format!("{} {}", target.flag, translation.trim())
}

/// Voice auto-translate reply: transcript + translation in one quote-reply.
pub fn format_voice_auto_translation(
source: &Language,
transcript: &str,
target: &Language,
translation: &str,
) -> String {
format!(
"📝 ({}) {}\n{} ({}) {}",
source.code,
transcript.trim(),
target.flag,
target.code,
translation.trim()
)
}

/// Resolve target language for a text message in translate-all mode.
pub fn target_for_message_text(
mode: &GroupTranslateMode,
Expand Down Expand Up @@ -291,55 +235,21 @@ mod tests {
}

#[test]
fn resolve_voice_pair_from_whisper_english() {
let mode = GroupTranslateMode::new(
resolve_language("es").unwrap(),
resolve_language("en").unwrap(),
);
let pair = resolve_translate_all_voice_pair(
&mode,
Some("en"),
"Hello, I'm speaking in English now. How was your day?",
);
let (source, target) = pair.expect("should resolve en -> es");
assert_eq!(source.code, "en");
assert_eq!(target.code, "es");
}

#[test]
fn resolve_voice_pair_spanish_from_whisper_or_hints() {
let mode = GroupTranslateMode::new(
resolve_language("es").unwrap(),
resolve_language("en").unwrap(),
);
let pair =
resolve_translate_all_voice_pair(&mode, Some("es"), "Como está? Como foi tu dia oi?")
.or_else(|| {
resolve_translate_all_voice_pair(&mode, None, "Como está? Como foi tu dia oi?")
});
let (source, target) = pair.expect("should resolve es -> en");
assert_eq!(source.code, "es");
assert_eq!(target.code, "en");
}

#[test]
fn resolve_voice_pair_maps_portuguese_to_spanish_in_es_en_pair() {
fn resolve_text_pair_maps_portuguese_to_spanish_in_es_en_pair() {
let mode = GroupTranslateMode::new(
resolve_language("es").unwrap(),
resolve_language("en").unwrap(),
);
let pair = resolve_translate_all_voice_pair(&mode, Some("pt"), "Como foi tu dia?");
let (source, target) = pair.expect("pt should map to es in es/en pair");
let pair = resolve_translate_all_text_pair(&mode, "Como foi tu dia?");
let (source, target) = pair.expect("pt-like text should map to es in es/en pair");
assert_eq!(source.code, "es");
assert_eq!(target.code, "en");
}

#[test]
fn format_voice_bilingual() {
fn format_text_auto_includes_flag() {
let es = resolve_language("es").unwrap();
let en = resolve_language("en").unwrap();
let out = format_voice_auto_translation(es, "Hola", en, "Hello");
assert!(out.contains("📝 (es) Hola"));
assert!(out.contains("🇺🇸 (en) Hello"));
let out = format_text_auto_translation(es, " Buenos días ");
assert_eq!(out, format!("{} Buenos días", es.flag));
}
}
128 changes: 1 addition & 127 deletions crates/signal-bot/src/commands/voice.rs
Original file line number Diff line number Diff line change
@@ -1,18 +1,13 @@
//! Implicit voice note handler — transcribe via Whisper and quote-reply.

use crate::commands::translate_service::{
format_voice_auto_translation, near_ai_translate, resolve_translate_all_voice_pair,
};
use crate::commands::CommandHandler;
use crate::error::AppResult;
use crate::group_preferences_store::GroupPreferencesStore;
use crate::transcribe_store::TranscribeStore;
use crate::voice_attachment_cache::VoiceAttachmentCache;
use async_trait::async_trait;
use near_ai_client::NearAiClient;
use signal_client::{Attachment, BotMessage, SignalClient};
use std::sync::Arc;
use tracing::{debug, info, instrument, warn};
use tracing::{info, instrument, warn};
use whisper_client::{WhisperClient, WhisperError};

#[cfg_attr(not(test), allow(dead_code))]
Expand All @@ -23,8 +18,6 @@ pub struct VoiceHandler {
signal: Arc<SignalClient>,
reply_prefix: String,
max_attachment_bytes: usize,
group_translate: Option<Arc<GroupPreferencesStore>>,
near_ai: Option<Arc<NearAiClient>>,
transcribe_store: Option<Arc<TranscribeStore>>,
voice_cache: Option<Arc<VoiceAttachmentCache>>,
}
Expand All @@ -41,8 +34,6 @@ impl VoiceHandler {
signal,
reply_prefix: reply_prefix.into(),
max_attachment_bytes,
group_translate: None,
near_ai: None,
transcribe_store: None,
voice_cache: None,
}
Expand All @@ -58,16 +49,6 @@ impl VoiceHandler {
self
}

pub fn with_translate_all(
mut self,
store: Arc<GroupPreferencesStore>,
near_ai: Arc<NearAiClient>,
) -> Self {
self.group_translate = Some(store);
self.near_ai = Some(near_ai);
self
}

pub fn format_transcript(text: &str, prefix: &str) -> String {
format!("{prefix}\n{text}")
}
Expand Down Expand Up @@ -97,81 +78,6 @@ impl VoiceHandler {
}
}
}

async fn translate_all_voice(
&self,
_message: &BotMessage,
audio: &Attachment,
bytes: &[u8],
group_id: &str,
) -> AppResult<String> {
let store = self
.group_translate
.as_ref()
.expect("translate-all store required");
let near_ai = self
.near_ai
.as_ref()
.expect("near_ai required for translate-all");
let mode = store
.get(group_id)
.expect("translate-all mode must be active");

let filename = Self::attachment_filename(audio);
let content_type = &audio.content_type;

let transcript = self
.whisper
.transcribe(bytes, &filename, content_type)
.await?;
let transcript_text = transcript.trimmed_text();
let whisper_lang = transcript.language.as_deref();

let (source, target) =
match resolve_translate_all_voice_pair(&mode, whisper_lang, transcript_text) {
Some(pair) => pair,
None => {
info!(
group_id,
whisper_lang,
"Voice transcript language not in translate-all pair — transcript only"
);
return Ok(Self::format_transcript(transcript_text, &self.reply_prefix));
}
};

debug!(
group_id,
whisper_lang,
source_lang = source.code,
target_lang = target.code,
whisper_translate = target.code == "en" && source.code != "en",
"translate-all voice pipeline"
);

let translation = if target.code == "en" && source.code != "en" {
match self
.whisper
.translate_to_english(bytes, &filename, content_type)
.await
{
Ok(r) => r.trimmed_text().to_string(),
Err(e) => {
warn!("Whisper translate failed, falling back to NEAR AI: {}", e);
near_ai_translate(near_ai, transcript_text, target).await?
}
}
} else {
near_ai_translate(near_ai, transcript_text, target).await?
};

Ok(format_voice_auto_translation(
source,
transcript_text,
target,
&translation,
))
}
}

#[async_trait]
Expand Down Expand Up @@ -232,38 +138,6 @@ impl CommandHandler for VoiceHandler {
return Ok("Voice note too long (max 5 min). Send a shorter clip.".into());
}

let translate_all_active = message.group_id.as_ref().is_some_and(|gid| {
self.group_translate
.as_ref()
.is_some_and(|s| s.is_active(gid))
});

if translate_all_active {
let group_id = message.group_id.as_deref().unwrap();
let store = self.group_translate.as_ref().unwrap();
if store.allow_message(group_id) {
match self
.translate_all_voice(message, audio, &bytes, group_id)
.await
{
Ok(response) => {
info!(
source = %message.source,
chars = response.len(),
"Voice note translated (translate-all)"
);
return Ok(response);
}
Err(e) => {
warn!("translate-all voice failed: {}", e);
return Ok("Could not transcribe voice note. Try again later.".into());
}
}
} else {
warn!(group_id, "translate-all rate limited — transcript only");
}
}

let filename = Self::attachment_filename(audio);
let result = self
.whisper
Expand Down
2 changes: 1 addition & 1 deletion docker/phala.translation.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Phala Cloud TEE — translation CVM (4 GB / tdx.medium)
# Hosts in-chat translation + Language Threads (+ Parallel Translation later).
# Hosts in-chat translation + Parallel Translation + Language Threads.
#
# Build & push (linux/amd64):
# docker buildx build --platform linux/amd64 -t YOUR_DOCKERHUB/signal-bot-tee:latest -f docker/Dockerfile --push .
Expand Down
Loading
Loading