Bug#507248: realtime settings

[Reinhard Tartler]

adi at drcomp.erfurt.thur.de (Adrian Knoth) writes:

> On Sat, Mar 07, 2009 at 02:47:28PM -0800, Steve Langasek wrote:
>
>> ardour uses mlock() on its ring buffers to ensure they're available in
>> physical memory.  What if in order to meet this requirement, the kernel
>> instead swaps out the process stack and has to page that back in?  What if
>> it drops the pages for the program itself out of the disk cache, and has to
>> read /that/ back from disk?
>
> I won't discuss this. The kernel RT wiki is very clear on that point,
> and all the audio developers do it for years.
>
> http://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application#Latencies_caused_by_Page-faults

This article does not discuss Steve's arguments at all. While it is
moderatly well written, I think that especially the part about page
faulting needs more thought for general purpose systems like Debian.

>>> > Increasming the mlock limit is a must and number one recommendation all
>> > over the net for pro-audio. jackd won't even start with realtime
>> > priority with the kernel's 32k limit.
>> Then that's a bug in the jackd package.
>
> Ok, it starts, but it won't have RT prio. And without RT prio, it's
> mostly pointless (especially in case of firewire audio interfaces
> handled by FFADO, resulting in many underruns and therefore crackle)

Perhaps we can do something with filesystem capabilities? Or perhaps
even some suid wrapper that drops priviledges immediatly after setting
the right priority? I cannot imagine that this hasn't been discussed yet
upstream. Could you (or anyone) provide pointers here?

>> > > Removing the memlock limit is still wrong.  That needs to be fixed in
>> > > whatever app is abusing locked memory this way.
>> > No. 
>> Yes, it does, and I will file a critical bug report on jackd if I find it
>> removing the memlock limit on my systems.
>
> You're sure you have ever used a pro-audio system? Then you won't call
> all the apps broken. Feel free to file the appropriate bugs upstream.

Please do that yourself, as you seem to be more involved with
"professional" audio applications.

> However, even if 99.9% of jackd users would be fine with "it's my
> machine, I want it all" (read: optimize the common case), I'd use
> debconf and ask whether to touch memlock limits and to which degree.
>
> So it's up to the user when things break, either failing RT performance
> or ruined system stability due to too much mlock().

If it's up to the user, then so be it. I don't think we should add
debconf screens bugging users about this as I feel the matter is too
complicated for an average system administrator to understand and make
an informed decision at installation time.

However I'd be more than happy to add an explanatory README file. Or
even some shell script tweaking the system to the needs. But I will
object to breaking or weakinging system security in a package's
postinst.

> JFTR: ubuntustudio has a settings package for tweaking limits.conf.
> Since there are non-jackified RT audio applications, it's probably a
> good idea.

That's most probably fine for ubuntustudio, FTR, I'm not convinced that
this was a good idea for the debian jack package.

> Just a last remark to make you help understanding why the pro-audio apps
> need so much memlock: I have a virtual piano. It uses a 970MB sample
> file. When sending MIDI data to it, it has to output the right sample
> within 8-12ms, and there's no time for reading it from disk or swapping
> it in. That's why all the data is mlock()ed, all the 970MB, and there
> are even bigger samples out there (i.e. Vienna Symphonic Library).
>
> Note that also commercial products on Windows or OSX lock the sample
> data in memory. It's the only way to get decent performance, and
> crackle is surely the last thing you want to get when being on stage.
>
> Also note that many audio distributions (ubuntustudio, studio64 a.s.o.)
> raise memlock limits. Take either as a hint that it's just a must, not a
> bug.

I fully agree to Steve, your proposed approach opens a wide door for
DoS'ing the complete system. This might be an acceptable risk for
"pro-audio users" (whatever that means), but I don't consider it
acceptable for Debian.

-- 
Gruesse/greetings,
Reinhard Tartler, KeyID 945348A4