[HN Gopher] Native Matrix VoIP with Element Call
___________________________________________________________________
Native Matrix VoIP with Element Call
Author : Sami_Lehtinen
Score : 353 points
Date : 2022-03-05 15:06 UTC (7 hours ago)
(HTM) web link (element.io)
(TXT) w3m dump (element.io)
| youerbt wrote:
| > And, the big one _drum roll, please_... we will be integrating
| this into Element so you can have voice and video rooms, and hold
| group video calls inside the Element app natively over Matrix.
|
| I'd love to replace my teamspeak server with this. Discord is
| tempting, but I'd rather host it myself.
| Taywee wrote:
| That's the exciting bit for me. I love Matrix, but video and
| voice being primarily "call" based is painful, when what I
| really want is a dynamic voice/video room that people can
| quickly and easily drop in and out of, like in Discord.
| Arathorn wrote:
| MSC3401 specifically provides for this UX, with the
| `m.intent` field used when you instantiate an `m.call` within
| a room (https://github.com/matrix-org/matrix-spec-
| proposals/blob/mat...):
|
| * `m.intent` to describe the intended UX for handling the
| call. One of:
|
| 1. `m.ring` if the call is meant to cause the room
| participants devices to ring (e.g. 1:1 call or group call)
|
| 2. `m.prompt` is the call should be presented as a conference
| call which users in the room are prompted to connect to
|
| 3. `m.room` if the call should be presented as a voice/video
| channel in which the user is immediately immersed on
| selecting the room.
|
| Element Call implements `m.room` effectively (in that you get
| slung directly into the call once you click on the link).
| Once we integrate this into Element itself, then the plan is
| to support all three different intent types: `m.ring` for
| "group calls", `m.prompt` for "conferences" and `m.room` for
| discord-style voice/video rooms.
| panick21_ wrote:
| So will the room have a setting to make a voice video only
| room, a mixed room or a chat only room?
|
| I assume m.ring would not be needed if its a something like
| a Teamspeak room.
| Arathorn wrote:
| It wouldn't be a setting; the intent would be set based
| on how you initiated the call. If you hit a big 'call
| Alice and Bob!' button you'd get a call which rings them
| both; if you hit a 'start a conference' you'd get a
| Teams/Zoom-style conference; if you hit 'create a
| voice/video room' button you'd get a Discord-style
| voice/video room.
|
| (N.B. we might not bother with the group-calling option
| in Element; haven't decided yet - but Matrix needs to
| support it for folks who want those semantics :)
| Taywee wrote:
| Thanks for the extra info. I'm really looking forward to
| the fruition of this, and will happily be trying out this
| current Element Call beta quite soon.
| kitkat_new wrote:
| you can choose one of the three, and if you choose option
| 3, the room will become a voice video only room (this is
| how I understand it).
|
| So m.ring existing isn't a problem.
|
| How it is exposed in the UI, is a completely different
| thing, although I don't think it would be impossible to
| make a button that creates a room with a m.room call aka
| voice/video room
| jeroenhd wrote:
| For what it's worth, Matrix / Element already have something
| like this with the built-in Jitsi integration in (some)
| clients. The UX isn't great right now in my opinion (which will
| hopefully change when this project gets merged into Element)
| but if all you want is rooms, text and voice, then you're
| already set with the current setup.
|
| You can self-host a Jitsi instance and use it in Matrix rooms
| as an integration. Depending on your teamspeak setup you can
| enable guest access to your server, enable registration for
| server user accounts (or manually register it for specific
| members), or only let people with existing accounts from other
| servers join in (their own, matrix.org ones, you name it). You
| might also need to tweak the ACLs a bit, but other than that
| you'll have all the core features Teamspeak and friends
| provide.
|
| I think the biggest difference between the existing Matrix
| group call system and what's been announced here today is the
| new UI/UX, and the fact that the native group call API and
| reference implementation are nearing completion. Both are great
| news, but don't necessarily add new user-facing features to the
| Matrix ecosystem.
| GeckoEidechse wrote:
| Yes but the current UX for that is terrible. With the
| proposed MSC linked in the blog post, Matrix will
| additionally gain Teamspeak/Discord like voice channels [1].
|
| With this we could finally have a proper Teamspeak/Mumble
| bridge for voice that gets properly represented on the Matrix
| side which is amazing :D
|
| Also kinda funny that Teamspeak only recently started using
| Matrix for their global chat feature [2].
|
| Maybe now after the gitter acquisition, Element should
| consider acquiring a certain voice-focused company ;)
|
| **
|
| [1] https://github.com/matrix-org/matrix-spec-
| proposals/blob/mat...
|
| [2] https://community.teamspeak.com/t/teamspeak-5-beta-bug-
| repor...
| georgyo wrote:
| The UX is more than terrible, it is basically unusable.
|
| The jitsi stuff is bolted on, you join a link and your user
| name does not come with you.
|
| No discoverablity. Does this channel already have a jitsi
| room attached, no way to know. I see a link to a jitsi room,
| is there anyone inside? No way to know.
|
| Recommending this as a alternative to TeamSpeak, mumble, or
| discord today is a way to get people to try a horrible
| experience and then say Matrix sucks forever.
|
| Only when this call stuff merges will Matrix start being
| viable for those communities.
| anticensor wrote:
| TeamSpeak 5 actually uses Matrix under the hood, with a
| proprietary voice protocol and TeamSpeak-specific room
| version.
| chayleaf wrote:
| Mumble works great for me in the meantime
| Gigachad wrote:
| Mumble doesn't really work on mobile making it not usable for
| most users
| [deleted]
| panick21_ wrote:
| This is really cool. I really feel like this ecosystem is hitting
| its stride.
|
| The list of upcoming features is long and sometimes I don't
| understand how they can have so many things going on at once.
|
| Features tend to take a few years to mature, but that is not a
| bad thing.
|
| I will be setting up my own server shortly, something that I have
| been thinking about for a while and maybe try to develop
| something.
|
| Congrats to the team.
|
| Question:
|
| The OpenID part, would it not be cool if Matrx would offer
| something like a federated SSO experience? So that on other
| webpages you could use your matrix login instead of Github/Google
| and co. Hosting that server yourself would be fantastic.
|
| There are of course some issues with Attribute retrieval if user
| can host these servers themselves.
| Arathorn wrote:
| > The OpenID part, would it not be cool if Matrix would offer
| something like a federated SSO experience?
|
| Yes, hopefully we'll be able to use the OIDC server bundled
| with your Matrix server to auth users in general if wanted (or
| alternatively you can of course point at an existing OIDC
| server).
| Godel_unicode wrote:
| Has anyone looked into their implementation of E2EE for calls? My
| naive assumption is they are brokering some kind of shared key
| exchange as opposed to requiring everyone to send an individually
| encrypted stream to everyone else.
| jfkimmes wrote:
| The post says there is no E2EE at the moment. They're probably
| talking about signalling, since with the current implementation
| of full mesh you get E2EE 'for free' in WebRTC.
|
| Also the post mentions:
| https://2021.commcon.xyz/talks/extending-matrix-s-e2ee-calls...
| Arathorn wrote:
| So in the initial beta we haven't turned on E2EE, purely
| because it will make it way harder to debug any problems
| which surface.
|
| However, at this rate, things are looking pretty stable and
| i'd expect us to enable it in the next few weeks. It uses
| normal Matrix E2EE, which means a Double Ratchet between the
| pairs of devices participating in a given conversation, which
| is then used to secure the signalling which is used to set up
| the calls between the devices. The call media is transport-
| layer encrypted via DTLS and SRTP, so as long as the
| signalling with the DTLS fingerprints is secured by Matrix
| E2EE, the whole call fabric will be E2EE.
|
| https://github.com/matrix-org/matrix-js-sdk/pull/2002 is the
| PR we actually need to merge in order to enable E2EE on it,
| ftr.
| arianvanp wrote:
| Mesh doesn't scale for large conference calls though. Usually
| some SFU needs to be in play. But that's only possible if you
| have Insertable Streams support which for now only Chrome has
| Arathorn wrote:
| Safari DP 141 actually released RTCRtpScriptTransform
| support 2 days ago (thanks to
| https://trac.webkit.org/changeset/270107/webkit/). But
| we're waiting eagerly for Firefox to add it in
| https://bugzilla.mozilla.org/show_bug.cgi?id=1631263.
|
| Hopefully by the time we've sorted out the SFU component,
| all the browsers will have RTCRtpScriptTransform
| implemented and so we'll get good cross-platform E2EE for
| larger calls (rather than it being limited to Chrome +
| Safari + Desktop)
| giancarlostoro wrote:
| My only concern is does this expose your IP to participants? This
| is one benefit of Discord Voice channels vs Discord Calls (which
| are end to end but expose your IP to everyone on the call, and
| theirs to you, so much so, you just open a console log and you
| can see them all...).
| Arathorn wrote:
| Currently yes, but it's trivial to fix and we'll do so on
| Monday - see https://news.ycombinator.com/item?id=30570436
| mattmerr wrote:
| Sounds like hiding IP is going to be opt-in? I'm not sure
| what the implications are of TURN but what would be the
| downside of making IP-hiding the default?
| Arathorn wrote:
| it would increase latency badly (the turn server for the
| call.element.io instance is in the UK, so all traffic would
| bounce through it, often unnecessarily), and it would cost
| us loads on bandwidth as a result. It also means that the
| turn server sees all the IPs and metadata of who is calling
| who, which may not be an improvement if you trust your
| caller more than your server admins!
|
| For instance, two users on the same LAN calling each other
| would end up bounced via the UK, which is a bit unfortunate
| if they are in Australia.
|
| The solution is really to switch to using SFUs everywhere,
| which then solves both firewall traversal, scalability and
| privacy (assuming you're happy for your SFU to know your IP
| - but if you're happy for your TURN to know it, then it's
| probably fine).
| puyoxyz wrote:
| Why is this an Element thing and not a Matrix thing?
| Arathorn wrote:
| The protocol, spec (MSC3401) and underlying client heavy
| lifting (matrix-js-sdk) are all Matrix; contributed by a mix of
| Element employees (Robert, me, Dave) and community contribs
| (Simon B). The app skin itself on top is Element, given most of
| the work and the original idea is from Element folks - same
| reason that Element itself is a product from Element-the-
| company even though 95% of the code is underlying Matrix
| projects. The Matrix Foundation itself is a non-profit to look
| after all the Matrix protocol and reference implementations but
| doesn't ship user-facing products, just as W3C and Linux
| Foundation don't.
| [deleted]
| ve5eta wrote:
| Impressive
| abnercoimbre wrote:
| I run a hybrid conference [0] with a physical track and an online
| track. All tracks communicate using Matrix.
|
| Element Call sounds like something I should integrate into the
| conference experience, but logistics and audience size is a
| concern. We'll be entering 1k+ territory this year.
|
| [0] https://handmade-seattle.com
| Arathorn wrote:
| please wait until we're out of beta! :)
| 2Gkashmiri wrote:
| this does not seem to have changeable room addresses right now
| like jitsi.
|
| we use jitsi common url to have weekly meetings.
| meet.jit.si/meeting2. everyone knows the url and time. we just
| join and talk. this is a nice concept
| jeroenhd wrote:
| That seems to be available, though?
|
| https://call.element.io/meetingwiththegang opens a meeting for
| a room named "meetingwiththegang".
|
| Ideally, you'd want to use such a system with accounts so you
| can apply some kind of ACL, but if you want to use randomly
| generated room names for secret links then everything you need
| is there already.
|
| Looks like you can't change the room name after starting the
| call, but you can start a new one at the new address if you
| want to change the name.
| kitkat_new wrote:
| I think it does in theory (Matrix supports changing room
| addresses), however what is the benefit of this as opposed to
| using the same address for all meetings?
| 3np wrote:
| The person you're replying to wants to set it to something
| custom, to make it memorable and meaningful.
| Arathorn wrote:
| We have this today. https://call.element.io/whateveryouwant
| does what you'd expect.
| jacobmischka wrote:
| I think that's what they want, I believe by "changeable" they
| meant "customizable"?
| robobro wrote:
| Did anyone else try to install this? It just refreshes non stop
| for me when I access its domain.
| Arathorn wrote:
| works fine for me. if the app had launched i'd ask you to
| submit feedback (which ends up in our bugtracker). as it
| hasn't, can you file a bug on github.com/vector-im/element-
| call/issues with a copy of the javascript console so we can see
| what's breaking? you might also want to disable any weird
| browser extensions to see if that's to blame.
| robobro wrote:
| sure, appreciate the comment! :)
|
| The main instance @ call.element.io also doesn't seem to work
| for me, but I chalked that up to (assumed) heavy server load.
| lijogdfljk wrote:
| A bit off topic, but anyone know if Threads are something Matrix
| currently supports, or ever wants to support?
|
| My family uses Signal for _a lot_, likely way more than we
| should, and i feel like we'd benefit from threads immensely.
|
| Maybe i should toy with making my own Matrix client or something,
| heh.
|
| _edit_ : https://github.com/vector-im/element-web/issues/2349
| looks like it might exist in some beta form. Depending on the
| client heh
| kevincox wrote:
| Threads are in beta on the Element clients. IIUC the spec isn't
| quite solidified (they are waiting for more clients to
| experiment first).
| jeroenhd wrote:
| Threads are supported on Matrix as of a week or two ago.
| They're no longer in beta as far as I can tell. I don't know
| for sure how well alternative clients support them, but desktop
| and mobile versions of Element do threads just fine.
|
| Alternatives (Fluffychat, for example) are usually a while
| behind Element because of manpower. For example, native polls
| are in Element but not in Fluffychat yet. So, if you want to
| risk the transition, best to recommend the official clients
| first.
|
| That said, I don't know how well threads will actually work in
| a family context. I like them as a concept, but outside Google
| Chat (where every message in a room is a response to or the
| start of a thread) I haven't seen them get picked up naturally.
| Arathorn wrote:
| yup, it's currently in testing on element web/ios/android and
| you can play with it in beta.
| jeroenhd wrote:
| I wonder if this setup can be leveraged to create a Twitch-like
| streaming solution based on Matrix. The group call setup is
| clearly not built for such a use case, but if you only stream to
| a small audience (say, people paying for a certain perk on
| Patreon?) you could probably use this quite easily.
|
| The client would need better support for alternative inputs, of
| course, such as the RTMP suggestion found on the Github issues
| page, but it can and should work.
|
| What I'm also curious about is how this fits in with the
| experiments on the peer-to-peer mesh system (running a homeserver
| on every client). Forwarding video across a web of clients would
| become bandwidth and CPU intensive real fast and the added
| latency would be non-negligible.
|
| I'll be watching this play out with great interest, that's for
| sure.
| Arathorn wrote:
| > I wonder if this setup can be leveraged to create a Twitch-
| like streaming solution based on Matrix.
|
| Yes, we've built it with this in mind. There are three possible
| approaches:
|
| * You can broadcast a headless client via HLS or RTMP, similar
| to how Jibri works for Jitsi, and how we broadcast FOSDEM
| (https://matrix.org/blog/2021/02/15/how-we-hosted-
| fosdem-2021...). Basically you run a headless Chrome against a
| virtual X server and pipe X's framebuffer into ffmpeg. It's a
| pretty blunt approach and obviously uses a lot of serverside
| resources, but ensures that the broadcasted stream is
| completely faithful given that it's literally a recording of
| what a client would be seeing.
|
| * Alternatively, you could do the same approach, but pipe it
| into a WebRTC broadcasting platform rather than
| HLS/RTMP/RTSP/DASH. This could give much lower latency, but
| more bandwidth on the server given you're effectively setting
| up a 1:1 VoIP call with each viewer, and you'll be unlikely to
| be able to use CDNs (unless we end up in some crazy world where
| freeform multicast on the internet finally comes to pass thanks
| to CDNs fanning out your RTP packets for you :P)
|
| * Finally, you could composite the streams together serverside
| using an MSC3401 compliant MCU and broadcast the result via one
| or other approach. This might not be quite as high fidelity as
| running a 'real' Element Call instance serverside, but could be
| way more efficient in terms of resources, and could prove
| perfectly adequate. The only catch is that we haven't bolted
| MSC3401 onto any MCUs yet. (Clearly we should get back in touch
| with our friends at FreeSWITCH / Signalwire :D)
|
| > What I'm also curious about is how this fits in with the
| experiments on the peer-to-peer mesh system (running a
| homeserver on every client). Forwarding video across a web of
| clients would become bandwidth and CPU intensive real fast and
| the added latency would be non-negligible.
|
| MSC3401 is pretty much orthogonal to P2P Matrix. The way it
| works is that it'll use whatever conferencing nodes ('foci')
| are advertised in the room to mix together the calls in a
| decentralised fashion. If there are none (as per today) then it
| just goes full mesh. If there is one, then everyone will
| converge on it. If there are two, then folks will pick the one
| with the lowest latency. But critically, it doesn't matter
| whether the clients are talking to a serverside homeserver or
| are running P2P Matrix. Finally, the foci could run either
| serverside or clientside (much as skype supernodes were
| effectively clientside foci), but we need to get serverside
| foci working first before we add clientside ones in for the P2P
| crew ;)
| anticensor wrote:
| > If there are two, then folks will pick the one with the
| lowest latency.
|
| No it needs to be deterministic. "Just picking the lowest
| latency node" would result in sync failures i.e. not
| everybody hearing the same thing.
| encryptluks2 wrote:
| > We're also working with the Tauri core team to provide cross-
| platform lightweight desktop apps as soon as possible!
|
| How about a lightweight cross-platform Matrix client that doesn't
| rely on Electron?
| Jack5500 wrote:
| Well, it doesn't use Electron, it uses tauri, which doesn't
| bundle chrome but uses the native browser and is therefor much
| smaller, faster und more lightweight.
| GeckoEidechse wrote:
| Why not just use FluffyChat?
|
| https://fluffychat.im/
| nani8ot wrote:
| I personally can't stand how big the chat bubbles in
| FluffyChat 1.0+ are. They were always big but now they are
| huge. I would have preferred it if they got smaller...
|
| But with Element now shipping chat bubbles as an configurable
| option, I'll stay with that. For me, the new chat bubbles
| size is perfect.
|
| Aside of that, I'm generally really fond of FluffyChat, they
| are doing a great job!
| DoItToMe81 wrote:
| I used Fluffychat for a few months. I found its performance
| was rather lacking. It had slowdown issues and general
| problems that I didn't find with other clients.
| Arathorn wrote:
| Tauri + Electron (or Hydrogen) provides a lightweight cross-
| platform Matrix client that doesn't rely on Electron...
|
| Meanwhile, Nheko, NeoChat, Fractal and others provide very
| usable lightweight native cross-platform Matrix clients too.
| Element is also working on its own experiments (TBA) based on
| matrix-rust-sdk.
| caslon wrote:
| Calling Fractal usable is maybe a bit of a stretch, but there
| are a lot of really good Matrix clients now!
| JCWasmx86 wrote:
| Fractal-Next is really usable for me at least. (The times
| I've tried it) The only thing that is hindering it for me
| is the missing SSO-Support, but as soon as my MR is merged,
| it has SSO-Support
| Arathorn wrote:
| Fractal-next is looking very usable (thanks to leveraging
| matrix-rust-sdk as its engine), tbh.
| karmanyaahm wrote:
| i think you meant Tauri + Element
| Arathorn wrote:
| bah, so i did. doh
| caslon wrote:
| There are already like a dozen Matrix clients that don't use
| Electron. It doesn't take much effort to go to the Clients list
| and pick one out.
| Gigachad wrote:
| The problem is the group of users who have a problem with
| Electron also tend to have a problem with a billion other
| things (the size of the chat bubbles as another user
| commented here) so they end up never being satisfied with any
| client and any developer who tries to cater to them will be
| flooded with a wave of complaints about how the animation
| speed should be configurable.
| jpbernius wrote:
| There is some cool development going on with Matrix recently!
| Threaded messaging first and now group calls. I am especially
| excited for Safari Support as this is my personal pain-point with
| Jitsi Meet. Very excited to see this evolve into the Element
| client.
| jiffygist wrote:
| Just wondering, can I have fullband stereo sound to play music?
| Arathorn wrote:
| The media quality that you hear is entirely up to the sender's
| WebRTC implementation. If you're calling another Element Call
| user, we currently use the default audio constraints when
| capturing audio: https://github.com/matrix-org/matrix-js-
| sdk/blob/96ba061732b... - but if you wanted stereo, you could
| put { channelCount: 2, sampleRate: 48000 } or whatever into
| that line to achieve it.
|
| I'll file a bug to make this configurable.
|
| EDIT: https://github.com/vector-im/element-call/issues/249
| traspler wrote:
| Do the Homeservers act as Signaling, STUN and TURN servers or are
| there additional components necessary? Will the SFU part in the
| future also be part of Synapse or will/are these things split?
| qqcwerqwect wrote:
| As I understand it, signalling will happen over the matrix
| protocol. as matrix/element already had a 1:1 voice chat,
| synapse integrates well with coturn, so you typically run
| coturn along side synapse.
| Arathorn wrote:
| The homeservers act as signalling servers. For STUN/TURN you
| need to run a separate TURN server (typically coturn), as per
| https://matrix-org.github.io/synapse/latest/turn-howto.html.
|
| The future SFU will similarly be split from the homeserver,
| with the initial implementation based on either Signal-Calling-
| Service, ionsfu or mediasoup (we're evaluating all three). Of
| course, the point of being standards based is that you'll be
| able to mix & match SFUs and MCUs from other vendors.
|
| You can find out more about the architecture from my talk at
| Commcon (https://2021.commcon.xyz/talks/extending-matrix-
| s-e2ee-calls...) - or Robert's talk at FOSDEM:
| https://fosdem.org/2022/schedule/event/matrix_metaverse/
| traspler wrote:
| Thanks a lot for answering in detail!
| anticensor wrote:
| For court admissible conferencing, routing everything through
| the persistent room state is almost a hard requirement.
| georgyo wrote:
| Only signalling. Matrix users coturn for STUN and TURN.
| encryptluks2 wrote:
| I feel if XMPP just had some better tools this push for Matrix
| would be unnecessary.
| kitkat_new wrote:
| please read the article (and linked MSC [0]) first and check
| out how this decentralized(!) conferencing proposal works.
|
| I don't think tools that implement this exist in the XMPP
| ecosystem at all.
|
| [0] https://github.com/matrix-org/matrix-spec-
| proposals/blob/mat...
| pigeons wrote:
| Yeah, I read Matrix's reasoning for not just improving XMPP and
| instead starting a new protocol, and I gave them a few years,
| and have to use matrix quite a bit and am consistently
| convinced it was the wrong choice and XMPP is the way to go.
| GeckoEidechse wrote:
| https://matrix.org/faq/#what-is-the-difference-between-matri...
| zaik wrote:
| If you read https://github.com/matrix-org/matrix-spec-
| proposals/blob/mat... it becomes apparent that the
| 'eventually consistent' part of Matrix is more of a hindrance
| in this case.
|
| Also 'the more federation and interoperability the better' is
| kind of contradictory to constantly reinventing extisting
| Internet Standards. I suspect VoIP over Matrix is not
| compatible with SIP or XMPP A/V calls and looking at current
| state of the matrix.org XMPP chat implementation they
| probably will never be.
| Gigachad wrote:
| If IRC had some better tools then the push for XMPP would be
| unnecessary. In fact, why didn't we just build IM on top of
| email?
|
| Turns out its much much easier to start from scratch with the
| right ideas than to transform an existing protocol in to
| something usable and convince everyone else and update all
| software to match.
|
| "The market" has decided that Matrix works well and XMPP
| doesn't. If it made sense to fix XMPP then someone would have
| done it and we would all be using that.
| zaik wrote:
| As of recently, Dino already can do video conferences based on
| XMPP and other clients are working on it as well. There is also
| a bidirectional SIP/XMPP bridge, so it integrates with other
| existing Internet Standards: https://sip.cheogram.com/
| mawise wrote:
| > Element Call is built entirely on Matrix: it doesn't need any
| additional servers to get going. You can run it against your
| existing Matrix homeserver to provide complete self-
| sovereignty...
|
| > In the near future, we will support using the app with any
| homeserver
|
| I really hope this isn't an indicator how how Element the company
| will be focusing more on their paid offerings to the exclusion of
| supporting private homeservers. I've recently been experimenting
| with running a Matrix homeserver on a Raspberry PI in my basement
| --I'm excited about actual self-sovereignty in messaging but it
| isn't quite to the point where I can tell my family to rely on it
| instead.
| jfkimmes wrote:
| I feel like this is a misunderstanding. You can already self
| host 'Element Call' and point that to your own homeserver. The
| passage you're referencing just refers to their hosted Element
| Call client. I'm pretty sure they're currently trying to move
| away from credentials-based login on 3rd-party clients to
| token-based authentication. I would guess they open up their
| hosted client when they finished implementing this new
| authentication scheme to avoid the doubled effort and to
| discourage using your homeserver credentials on "random"
| clients.
| Arathorn wrote:
| Yup, the grandparent post is a misunderstanding. Basically,
| we want Element Call to work with a single click to a URL
| without requiring any account - much as Jitsi and similar do.
| Therefore it needs to pick a default homeserver. Currently it
| doesn't play well with existing Matrix accounts (as it will
| sync in loads of chatrooms which are irrelevant when you just
| want to do a call), so for expedience we just create accounts
| on its default homeserver to get going.
|
| However, we're in the process of moving Matrix over to OIDC
| for auth (as per https://matrix.org/blog/2021/12/22/the-mega-
| matrix-holiday-s...), at which point apps like Element Call
| should be able to easily hook into your existing account on
| your own homeserver using your existing server auth. In other
| words, the server will auth you, not the client. Combined
| with Sliding Sync (aka Sync v3)
| https://matrix.org/blog/2021/12/22/the-mega-matrix-
| holiday-s... this will then let you securely and efficiently
| use your existing Matrix account with apps like Element Call
| without it all getting bogged down with your existing
| chatrooms (or needlessly giving Element Call access to
| conversations it shouldn't care about).
|
| Finally, this is beta: the auth/reg stuff is very much
| placeholder - for instance, it doesn't even expose password
| reset, given the upcoming shift to OIDC.
| danShumway wrote:
| > using your existing server auth
|
| Complete sidenote, but just to expand on the above, the
| OIDC work is particularly exciting because (at least my
| understanding is) it opens the door for letting people use
| a single Matrix account for a lot of different "stuff" on
| top of Matrix.
|
| There have been some interesting conversations and demos
| I've seen about using Matrix rooms to help with P2P sync
| for apps, and there are already bots on Matrix that use
| Matrix rooms for notifications so you don't need to send
| emails. But setting that stuff up is kind of challenging if
| you don't want to make a dedicated new user for each app or
| grant access to your account -- and (again, assuming my
| understanding is correct) I vaguely suspect that we might
| start to see a lot more 3rd-party apps built to integrate
| with Matrix once it's easier to point someone at a webapp
| and say, "just log in with your Matrix account, and it'll
| only have access to one room that's specific to this app."
|
| So imagine potentially having a collaborative app that's
| hyper-focused on one task, like a collaborative D&D
| messaging platform or some crud. And in the backend,
| everyone using that app uses their regular Matrix account,
| and under the hood it's just sending messages to a shared
| room -- and then as a developer you don't need to worry
| about handling a bunch of encryption and user accounts and
| all of that stuff, you might not even need a backend at
| all.
| dane-pgp wrote:
| > you might not even need a backend at all.
|
| On the basis that "the best code is no code", this could
| revolutionise app architecture. Imagine if instead of
| using blockchains, Web3 apps were backed by Matrix
| servers.
| mawise wrote:
| I guess I still don't understand it. (And my apologies for
| coming across like I was trying to pick a fight)
|
| What does it mean that I can currently use it with my
| existing homeserver? The architectures for some of these
| federated systems can be difficult to understand, I feel
| like I didn't really understand what _Matrix_ was until I
| tried running my own homeserver.
| jfkimmes wrote:
| Federated systems tend to have different client and
| server implementations that can work in all combinations
| (as long as both are spec compliant).
|
| The confusing part comes into play when the client
| implementation is a web app that you can use to log in to
| _any_ server hosted anywhere. Because of how centralized
| services work, we are used to a paradigm where the
| website you visit hosts its own backend - you cannot
| choose. With these federated systems, on the other hand,
| you can go to website element.foo.com and log in at
| matrix.bar.com.
|
| The blog post was saying they limited the option to log
| into your homeserver at matrix.bar.com (because of the
| reasons outlined above). This does not, however, stop you
| from hosting their (FOSS) app at call.bar.com and
| pointing it to your homeserver at matrix.bar.com
| soupbowl wrote:
| I am excited about this, I hope it is stable and secure...
| Arathorn wrote:
| From my perspective, the really exciting thing about this that it
| works equally well in mobile web browsers as well as desktop web
| - clicking on a link on Mobile Safari should Do The Right Thing
| without having to install anything.
|
| Moreover, because it's built on Matrix, MSC3401
| (https://github.com/matrix-org/matrix-doc/blob/matthew/group-...)
| means that we'll finally have decentralised cascading video/voice
| conferences once the SFU (selective forwarding unit) component is
| added into the mix. So, for instance, users on the same
| homeserver will get their video feeds relayed locally with
| minimal latency... and then users on another remote homeserver
| will also get mixed locally with minimal latency, trunking the
| two together. If the link dies or one homeserver dies, the
| conference will keep going - i.e. precisely the same semantics as
| normal Matrix.
| jfkimmes wrote:
| Could you give some insights on how you estimate the amount of
| effort of making Element Call competitive, performance-wise,
| with say Discord. I heard that Discord threw a lot of time and
| money at optimizing voice in their product. Can you just jump
| in and realistically compete?
|
| There are a lot of performance/latency/sound quality
| comparisons online of Mumble vs TeamSpeak vs Discord and
| recently Jitsi vs MS Teams vs Zoom, etc. I feel like this is a
| problem-space that can be optimized to an arbitrarily deep
| extent. Just two examples that come to mind are SFU
| performance/efficiency and noise reduction, two things where
| e.g Jitsi notoriously lags behind Zoom.
| Arathorn wrote:
| The competitive gap with Discord in terms of media quality is
| probably something like:
|
| * Need a low-latency SFU. This should be very doable; not
| only are there a lot of good FOSS SFUs to build on top of
| these days, the history of the Matrix team is actually that
| we built VoIP stacks fulltime before we shifted focus to
| Matrix, and we've built MCUs and media servers of all
| flavours in the past. MSC3401 should also give us a
| competitive edge given latency will be automagically
| minimised by using the physically closest decentralised SFU,
| thus letting anyone bring their SFU to the party.
|
| * Needs a SFU with good rate control (and/or FEC). This is
| probably the single most important thing to get right in
| terms of quality. Signal wrote up a good overview of why:
| https://signal.org/blog/how-to-build-encrypted-group-calls/
|
| * Excellent noise cancellation (and background noise
| elimination, microphone scratch noise elimination etc).
| Ideally you need something like https://krisp.ai/ or https://
| workspaceupdates.googleblog.com/2021/06/background-n... in
| the mix - but doing this in an E2EE-friendly and privacy
| preserving manner is Hard. However, just like we solved E2EE
| full text search by doing it clientside and making the
| indexes gossipable between your clients
| (https://github.com/matrix-org/seshat), we'll have a go at
| doing something similar for this problem too.
|
| * Excellent automatic gain control. The importance of
| normalising/compressing everyone's audio so they're
| equivalent loudness is really important.
|
| We're also in the process of adding in spatial audio (unsure
| if Discord has that) which should help a tonne with
| distinguishing the different audio feeds.
|
| We can probably also be more bullish about supporting new
| audio codecs like Lyra.
|
| EDIT: oh, the other obvious thing is echo cancellation - but
| we're currently at the mercy of the browser's WebRTC stack
| for that. However, we could ship a tweaked WebRTC in Element
| Desktop (or other native Matrix clients) in future to do
| better than plain old WebRTC.
| anticensor wrote:
| Discord voice is actually a server muxing MCU emulating an
| SFU.
| dopa42365 wrote:
| Something wrong with simply using RNNoise?
| Arathorn wrote:
| mmm, https://github.com/jitsi/rnnoise-wasm looks like it
| might help. wonder if jitsi actually uses it.
| Cu3PO42 wrote:
| The repository is under the Jitsi organization, so I'd
| say chances of them using it or at least planning to use
| it are quite high.
| jfkimmes wrote:
| Very insightful! I'll have a look at the links. Thanks for
| answering.
| spockz wrote:
| Nice elaborate answer. Discord has noise cancellation by
| krisp. I believe it is offline only. That would be good
| enough I suppose. But I'm not sure whether it can be loaded
| in webasm so it works in the browser. (To meet the cross
| platform needs of Matrix/Element.)
| GeckoEidechse wrote:
| Maybe a bit off-topic but will upcoming Matrix Live also be
| hosted via Matrix VoIP now? ^^
|
| (At least the smaller ones like interviews where the number of
| participants is <=8 until compatibility with SFU is done)
| Arathorn wrote:
| sure! :D
| stevenicr wrote:
| great post, animation helps a bunch too.
|
| wondering if ip addys are exposed to the other users like with
| webrtc and if so, could we force conencts to be only through
| coturn to hide the ips?
|
| Also wonder if there will be a warning for such, especially of
| encryption is turned on - some may think they are truly
| anonymous, and where ips are exposed of course that's not true.
|
| Also wonder about moderation, hopefully this does not become a
| target for the sickening trolls soon - but moderation needs will
| be coming, so who gets the ip logs to consider blocking?
| homeserver runners?
|
| I expect the future will need whitelists/blocklists subscription
| options for clients at some point.
| Arathorn wrote:
| Currently we don't force TURN, so in practice this means that
| voice packets go direct between the clients if possible, and so
| the IP addresses of the clients are necessarily exposed to each
| other.
|
| However, this is utterly trivial to fix: matrix-js-sdk already
| exposes https://github.com/matrix-org/matrix-js-
| sdk/blob/96ba061732b... and we simply haven't exposed it as a
| setting in Element Call yet. I've filed a bug for it at
| https://github.com/vector-im/element-call/issues/251 - thanks
| for bringing it up!
|
| In terms of moderation: this is no different to moderation in
| Matrix as a whole, where we're already busy working on shared
| greylists (MSC2313 and friends) -
| https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix...
| has more details at the end.
___________________________________________________________________
(page generated 2022-03-05 23:00 UTC)