Rawaudio dev by dingodoppelt · Pull Request #3653 · jamulussoftware/jamulus

dingodoppelt · 2026-04-17T21:32:16Z

Add a new "raw" audio quality setting

This PR adds uncompressed audio ("raw") to the quality settings so there is no Opus compression along the way
Discussion in #3654

This feature improves latency as well. I gained 2ms by using uncompressed audio while having a better audio quality.

CHANGELOG: Add uncompressed audio transmission - dedicated to the memory of Hans Petter Selasky (1982 - 2023)

Does this change need documentation? What needs to be documented and how?
Corresponding PR in jamulussoftware/jamuluswebsite #1133

Checklist

I've verified that this Pull Request follows the general code principles
I tested my code and it does what I want
My code follows the style guide
I waited some time after this Pull Request was opened and all GitHub checks completed without errors.
I've filled all the content above

…s on audio quality

…d but works

…ber)

ann0see · 2026-04-20T15:56:10Z

I'd prefer not to check for the Jamulus version number but rather based on capabilities - we don't have 4.0.0 out yet and it might break during the dev process.

dingodoppelt · 2026-04-20T16:11:19Z

I'd prefer not to check for the Jamulus version number but rather based on capabilities - we don't have 4.0.0 out yet and it might break during the dev process.

I wanted to reuse information already available as much as possible so I just added the code where there were version checks already implemented. (For sequence number and pan feature)
Capabilities would be nice but also would require more changes to client, channel, server and protocol which I don't really have an idea on how to make that backwards compatible. We should rather replace all version checks with some capabilities struct that client and server can agree upon so everything lands in one place. I just don't feel like the right person to take on that challenge and rather pursue my hacky approach, as long as it works for everybody.
The version check with 4.0.0 could be replaced by a point release 3.11.1 and would work right away.

ann0see · 2026-04-20T17:13:55Z

Tested it and yes, the noise would be unacceptable. What is our fallback if max is selected but the server doesn't support it?

dingodoppelt · 2026-04-20T17:22:01Z

Tested it and yes, the noise would be unacceptable. What is our fallback if max is selected but the server doesn't support it?

I just noticed that if you connect to a server with Max selected you get the noise unless you switch audio quality again while connected. The server code is fine and doesn't need changes, I misplaced the check for my introduced bRawAudioSupported in the client code. I'll have a closer look
Edit: Funny, the noise doesn't happen on legacy servers, only on rawaudio :D

pljones · 2026-04-29T14:14:05Z

It's definitely interesting that this lack of backwards compatibility without

  -F, --fastupdate        use 64 samples frame size mode

on the server when the client is using hasn't been spotted before -- I've certainly been using Small Network Buffers on every server I connect to and I can't say I've ever noticed anything. Here's my full settings:

Is this happening only on rawaudio servers not using -F?

softins · 2026-04-29T16:21:58Z

It's definitely interesting that this lack of backwards compatibility without
  -F, --fastupdate        use 64 samples frame size mode
on the server when the client is using Small Network Buffers hasn't been spotted before -- I've certainly been using Small Network Buffers on every server I connect to and I can't say I've ever noticed anything.
...
Is this happening only on rawaudio servers not using -F?

Yes, it seems so. I've just done some tests with Wireshark capturing the traffic between a client and a server running without -F:

Server 3.11.0, client 3.11.0 - no problem.
Server latest main, client recent main - no problem.
Server latest rawaudio-dev 3f059b3, client recent main - 64 byte buffers with Small Buffers enabled gives the distorted audio, even though the client isn't a rawaudio build.

I have saved the Wireshark captures, but not yet examined them.

pljones · 2026-04-29T17:36:50Z

I asked VSCode CoPilot:

Likely cause identified: a packet-send regression in server mix/encode on this branch.

The server can require two sub-blocks per client frame when server runs without fastupdate (128-sample server frame) and a client is on small buffers (64-sample transport). That is set here: server.cpp:771.
In this branch, encode/copy still iterates over both sub-blocks, but packet send is done only once after the loop: server.cpp:1191, server.cpp:1205, server.cpp:1213.
Result: when two blocks are needed, only the last encoded/copied block is transmitted each cycle (the first is overwritten before send). That produces the garbled/alternating distortion pattern users report.
Why this matches your symptom:

It is triggered specifically by non-fastupdate server mode (128-frame server path) with 64-frame clients.
That is exactly the compatibility path that depends on two conversion blocks and is sensitive to send placement.

It supplied this patch

@@
-        if ( iCeltNumCodedBytes != static_cast<int> ( iClientFrameSizeSamples * vecNumAudioChannels[iChanCnt] * sizeof ( int16_t ) ) )
+        if ( iCeltNumCodedBytes != static_cast<int> ( iClientFrameSizeSamples * vecNumAudioChannels[iChanCnt] * sizeof ( int16_t ) ) )
         {
             // OPUS encoding
             if ( pCurOpusEncoder != nullptr )
             {
@@
                 for ( int iB = 0; iB < vecNumFrameSizeConvBlocks[iChanCnt]; iB++ )
                 {
                     const int iOffset = iB * SYSTEM_FRAME_SIZE_SAMPLES * vecNumAudioChannels[iChanCnt];
 
                     iUnused = opus_custom_encode ( pCurOpusEncoder,
                                                    &vecsSendData[iOffset],
                                                    iClientFrameSizeSamples,
                                                    &vecvecbyCodedData[iChanCnt][0],
                                                    iCeltNumCodedBytes );
+
+                    // send each encoded sub-block (important when vecNumFrameSizeConvBlocks == 2)
+                    vecChannels[iCurChanID].PrepAndSendPacket ( &Socket, vecvecbyCodedData[iChanCnt], iCeltNumCodedBytes );
                 }
             }
         }
         else
         {
             for ( int iB = 0; iB < vecNumFrameSizeConvBlocks[iChanCnt]; iB++ )
             {
                 const int iOffset = iB * SYSTEM_FRAME_SIZE_SAMPLES * vecNumAudioChannels[iChanCnt];
 
                 memcpy ( &vecvecbyCodedData[iChanCnt][0], &vecsSendData[iOffset], iCeltNumCodedBytes );
+
+                // send each raw sub-block (important when vecNumFrameSizeConvBlocks == 2)
+                vecChannels[iCurChanID].PrepAndSendPacket ( &Socket, vecvecbyCodedData[iChanCnt], iCeltNumCodedBytes );
             }
         }
-        // send separate mix to current clients
-        vecChannels[iCurChanID].PrepAndSendPacket ( &Socket, vecvecbyCodedData[iChanCnt], iCeltNumCodedBytes );

My fault for saying put the PrepAndSendPacket outside the if, I think...

softins · 2026-04-29T20:22:05Z

Wow, that's impressive! I haven't tried copilot yet.

Although I can't see what is different between the two versions of the first line?

I'll try the patch a little later and repeat my tests.

…ent settings for small network buffers

dingodoppelt · 2026-04-30T08:52:07Z

I can confirm the issue being fixed with the last commit in which I applied the suggested patch. (actually a revert)

softins · 2026-04-30T16:40:08Z

Yes, I can confirm the problem with small buffers enabled connected to a server without -F appears to be fixed.

Finally understood that the only difference between having -F or not is that with 64-byte packets, when -F is specified, the server sends them one at a time at 1.33ms intervals, and without -F, it sends them in pairs at 2.66ms intervals. The issue before the last commit was that only one of each pair was being sent.

dingodoppelt · 2026-04-30T17:53:55Z

Furthermore I can't reproduce the crash on closing anymore. On linux I was getting crashes very rarely in the first place but I could trigger it by connecting to a server with -c <IP> but this doesn't crash anymore when closing the client.
This should be tested on macos and windows, too since I've had reports (mostly from mac users) of the client crashing on close.

softins

Fixes for the CodeQL warnings

@softins

Thanks @softins Co-authored-by: Tony Mountifield <tony@mountifield.org>

dingodoppelt · 2026-05-07T19:42:12Z

Fixes for the CodeQL warnings

Applied! Thank you very much for the suggestions @softins

pljones · 2026-05-08T17:02:35Z


-    opus_custom_encoder_ctl ( CurOpusEncoder,
-                              OPUS_SET_BITRATE ( CalcBitRateBitsPerSecFromCodedBytes ( iCeltNumCodedBytes, iOPUSFrameSizeSamples ) ) );
+    if ( !bRawAudioIsSupported )


Shouldn't this happen only if AQ_RAW is also selected? Or is bRawAudioIsSupported never true in that case?

(I mean, if AQ_RAW is not selected, bRawAudioIsSupported shouldn't matter, the control setting should be done. Like line 1468.)

pljones · 2026-05-08T17:06:52Z

+            {
+                pCurCodedData = &vecbyNetwData[0];
+            }
            // on any valid received packet, we clear the initialization phase flag


Keep the blank lines before the comments to make them more visibly related to the line(s) below, please.

pljones · 2026-05-08T17:11:40Z


-        // OPUS decoding
-        if ( CurOpusDecoder != nullptr )
+        if ( ( eAudioQuality != AQ_RAW || !bRawAudioIsSupported ) && CurOpusDecoder != nullptr )


Will CurOpusDecoder != nullptr be true any time other than when ( eAudioQuality != AQ_RAW || !bRawAudioIsSupported ) is true? That is, isn't the first part redundant? If not, maybe it should be.

pljones · 2026-05-08T17:18:19Z

+                if ( bIsRawAudio )
+                {
+                    memcpy ( &vecvecsData[iChanCnt][iOffset], &vecvecbyCodedData[iChanCnt][0], iCeltNumCodedBytes );
+                    pCurCodedData = nullptr;


Maybe set this outside the if so it's _definitely nullptr unless eGetStat == GS_BUFFER_OK and not bIsRawAudio. Essentially, it would mean moving (new) lines 909-910 to after (new) line 889.

pljones · 2026-05-08T17:19:46Z


-            // OPUS decode received data stream
-            if ( CurOpusDecoder != nullptr )
+            if ( !bIsRawAudio && CurOpusDecoder != nullptr )


Again, the !bIsRawAudio should be redundant.

pljones · 2026-05-08T17:24:03Z

        }

-        // OPUS encoding
-        if ( pCurOpusEncoder != nullptr )


So why can't this check be relied on here, like it has been elsewhere?

pljones · 2026-05-08T17:30:39Z

    // length. Since that is usually not the case but the buffers are usually
    // a bit larger than necessary, we introduce some factor for compensation.
    // Consider the jitter buffer on the client and on the server side, too.
    const float fTotalJitterBufferDelayMs = fSystemBlockDurationMs * ( GetSockBufNumFrames() + GetServerSockBufNumFrames() ) * 0.7f;


I'd like that 0.7f moved to global.h and documented, really... It should tie in with the Auto-Jitter sizing calculation, rather than being separately hard-coded here.

dingodoppelt added 10 commits April 17, 2026 19:59

Introduce raw audio setting in UI. Opus encoding in client now depend…

a45aded

…s on audio quality

addition to last commit

73d983f

client still sends raw audio to stock servers and causes garbled soun…

03f7ab3

…d but works

make client opus decoding depend on audio quality (raw audio)

9a05dd1

no crashes but bad noise on raw audio setting

0f22ee9

first working version (needs testing)

7f47dd8

fix style

7734d75

fix style

fa61b55

Merge branch 'jamulussoftware:main' into rawaudio-dev

68deea9

fix mute myself

5a467e0

dingodoppelt marked this pull request as ready for review April 19, 2026 06:54

remove redundant include

8f53817

ann0see added this to the Release 4.0.0 milestone Apr 20, 2026

ann0see added this to Tracking Apr 20, 2026

github-project-automation Bot moved this to Triage in Tracking Apr 20, 2026

ann0see reviewed Apr 20, 2026

View reviewed changes

Comment thread src/clientsettingsdlg.cpp Outdated

Comment thread src/util.h

dingodoppelt commented Apr 20, 2026

View reviewed changes

Comment thread src/client.cpp

dingodoppelt added 2 commits April 20, 2026 15:47

Change audio quality option from 'Raw' to 'Max'

c77029d

Only send raw audio if the server supports it (checked by version num…

d36448a

…ber)

dingodoppelt added 5 commits April 20, 2026 18:12

fix style

8079eed

fix style

5c73b6a

change version

a597843

Update changelog

7fa20b7

fix changelog

b602efa

dingodoppelt added 2 commits April 20, 2026 20:27

Fix noise issue with rawaudio servers when Max audio quality is selected

42af9ab

Merge branch 'rawaudio-dev' into autobuild-rawaudio

1cb8a22

Fix issue with internal buffering for clients and servers with differ…

ca1b74f

…ent settings for small network buffers

Merge branch 'main' into rawaudio-dev

6705495

github-advanced-security AI found potential problems May 3, 2026

View reviewed changes

Comment thread src/client.cpp Fixed

Comment thread src/client.cpp Fixed

Comment thread src/client.cpp Fixed

Comment thread src/client.cpp Fixed

Comment thread src/server.cpp Fixed

Comment thread src/server.cpp Fixed

pljones moved this from Triage to Backlog in Tracking May 3, 2026

pljones moved this from Backlog to In Progress in Tracking May 3, 2026

pljones assigned dingodoppelt May 3, 2026

dingodoppelt mentioned this pull request May 6, 2026

Update server bandwith table with correct values for existing and new audio quality settings jamulussoftware/jamuluswebsite#1137

Open

3 tasks

Merge branch 'jamulussoftware:main' into rawaudio-dev

27faaa7

softins reviewed May 7, 2026

View reviewed changes

Comment thread src/client.cpp Outdated

Comment thread src/client.cpp Outdated

Comment thread src/client.cpp Outdated

Comment thread src/client.cpp Outdated

Comment thread src/server.cpp Outdated

Comment thread src/server.cpp Outdated

Apply suggestions from code review to fix CodeQL warnings

5c1e8ef

Thanks @softins Co-authored-by: Tony Mountifield <tony@mountifield.org>

pljones reviewed May 8, 2026

View reviewed changes

Comment thread src/server.cpp

pljones reviewed May 8, 2026

View reviewed changes

Comment thread src/server.cpp

pljones reviewed May 8, 2026

View reviewed changes

Comment thread src/client.cpp

dingodoppelt added 2 commits May 9, 2026 00:29

Add missing comment

129386c

Merge branch 'jamulussoftware:main' into rawaudio-dev

fb2b822

Conversation

dingodoppelt commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ann0see commented Apr 20, 2026

Uh oh!

dingodoppelt commented Apr 20, 2026

Uh oh!

ann0see commented Apr 20, 2026

Uh oh!

dingodoppelt commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pljones commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

softins commented Apr 29, 2026

Uh oh!

pljones commented Apr 29, 2026

Uh oh!

softins commented Apr 29, 2026

Uh oh!

dingodoppelt commented Apr 30, 2026

Uh oh!

softins commented Apr 30, 2026

Uh oh!

dingodoppelt commented Apr 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

softins left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dingodoppelt commented May 7, 2026

Uh oh!

pljones May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pljones May 8, 2026

Choose a reason for hiding this comment

Uh oh!

pljones May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pljones May 8, 2026

Choose a reason for hiding this comment

Uh oh!

pljones May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pljones May 8, 2026

Choose a reason for hiding this comment

Uh oh!

pljones May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

dingodoppelt commented Apr 17, 2026 •

edited

Loading

dingodoppelt commented Apr 20, 2026 •

edited

Loading

pljones commented Apr 29, 2026 •

edited

Loading

pljones May 8, 2026 •

edited

Loading